2023-11-27T14:53:09.714 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-27T14:53:09.714 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_choffman@teuthology --name choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/misc}} --archive /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809 -- /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809/orig.config.yaml 2023-11-27T14:53:09.771 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-27T14:53:09.856 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809 branch: wip-choffman-62793 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/misc}} email: null first_in_suite: false job_id: '7467809' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-choffman-62793 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62793 sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 owner: scheduled_choffman@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8430 sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b sleep_before_teardown: 0 suite: fs suite_branch: wip-62793 suite_path: /home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa suite_relpath: qa suite_repo: https://github.com/chrisphoffman/ceph.git suite_sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi008.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi029.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-27_14:43:37 tube: smithi user: choffman verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-27T14:53:09.856 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa; will attempt to use it 2023-11-27T14:53:09.857 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks 2023-11-27T14:53:09.857 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-27T14:53:09.858 INFO:teuthology.task.internal:Checking packages... 2023-11-27T14:53:09.881 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '1c9565146c8647d4e00eb6c7ae626dc192680e7b' 2023-11-27T14:53:09.881 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-27T14:53:09.881 INFO:teuthology.packaging:ref: None 2023-11-27T14:53:09.881 INFO:teuthology.packaging:tag: None 2023-11-27T14:53:09.881 INFO:teuthology.packaging:branch: wip-choffman-62793 2023-11-27T14:53:09.881 INFO:teuthology.packaging:sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T14:53:09.882 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-choffman-62793 2023-11-27T14:53:10.030 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3906.g1c956514 2023-11-27T14:53:10.030 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-27T14:53:10.037 INFO:teuthology.task.internal:no buildpackages task found 2023-11-27T14:53:10.037 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-27T14:53:10.057 INFO:teuthology.task.internal:Saving configuration 2023-11-27T14:53:10.077 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-27T14:53:10.083 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T14:53:10.107 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:01.949055', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T14:53:10.130 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:01.950693', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T14:53:10.151 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:01.947281', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T14:53:10.151 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-27T14:53:10.157 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-11-27T14:53:10.157 INFO:teuthology.task.internal:roles: ubuntu@smithi029.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-11-27T14:53:10.158 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-11-27T14:53:10.158 INFO:teuthology.run_tasks:Running task console_log... 2023-11-27T14:53:10.259 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd8b179bf70>, signals=[15]) 2023-11-27T14:53:10.260 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-27T14:53:10.266 INFO:teuthology.task.internal:Opening connections... 2023-11-27T14:53:10.266 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi008.front.sepia.ceph.com 2023-11-27T14:53:10.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:53:10.336 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi029.front.sepia.ceph.com 2023-11-27T14:53:10.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:53:10.408 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2023-11-27T14:53:10.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:53:10.476 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-27T14:53:10.484 DEBUG:teuthology.orchestra.run.smithi008:> uname -m 2023-11-27T14:53:10.506 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2023-11-27T14:53:10.507 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2023-11-27T14:53:10.567 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="Red Hat Enterprise Linux" 2023-11-27T14:53:10.567 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="8.6 (Ootpa)" 2023-11-27T14:53:10.567 INFO:teuthology.orchestra.run.smithi008.stdout:ID="rhel" 2023-11-27T14:53:10.567 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="fedora" 2023-11-27T14:53:10.567 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="8.6" 2023-11-27T14:53:10.568 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el8" 2023-11-27T14:53:10.568 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-27T14:53:10.568 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2023-11-27T14:53:10.568 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-27T14:53:10.568 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://www.redhat.com/" 2023-11-27T14:53:10.568 INFO:teuthology.orchestra.run.smithi008.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-27T14:53:10.568 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-27T14:53:10.568 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T14:53:10.568 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-27T14:53:10.569 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-27T14:53:10.569 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-27T14:53:10.569 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-27T14:53:10.569 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2023-11-27T14:53:10.594 DEBUG:teuthology.orchestra.run.smithi029:> uname -m 2023-11-27T14:53:10.614 INFO:teuthology.orchestra.run.smithi029.stdout:x86_64 2023-11-27T14:53:10.615 DEBUG:teuthology.orchestra.run.smithi029:> cat /etc/os-release 2023-11-27T14:53:10.675 INFO:teuthology.orchestra.run.smithi029.stdout:NAME="Red Hat Enterprise Linux" 2023-11-27T14:53:10.675 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION="8.6 (Ootpa)" 2023-11-27T14:53:10.675 INFO:teuthology.orchestra.run.smithi029.stdout:ID="rhel" 2023-11-27T14:53:10.675 INFO:teuthology.orchestra.run.smithi029.stdout:ID_LIKE="fedora" 2023-11-27T14:53:10.675 INFO:teuthology.orchestra.run.smithi029.stdout:VERSION_ID="8.6" 2023-11-27T14:53:10.676 INFO:teuthology.orchestra.run.smithi029.stdout:PLATFORM_ID="platform:el8" 2023-11-27T14:53:10.676 INFO:teuthology.orchestra.run.smithi029.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-27T14:53:10.676 INFO:teuthology.orchestra.run.smithi029.stdout:ANSI_COLOR="0;31" 2023-11-27T14:53:10.676 INFO:teuthology.orchestra.run.smithi029.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-27T14:53:10.676 INFO:teuthology.orchestra.run.smithi029.stdout:HOME_URL="https://www.redhat.com/" 2023-11-27T14:53:10.676 INFO:teuthology.orchestra.run.smithi029.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-27T14:53:10.676 INFO:teuthology.orchestra.run.smithi029.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-27T14:53:10.676 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T14:53:10.676 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-27T14:53:10.676 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-27T14:53:10.677 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-27T14:53:10.677 INFO:teuthology.orchestra.run.smithi029.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-27T14:53:10.677 INFO:teuthology.lock.ops:Updating smithi029.front.sepia.ceph.com on lock server 2023-11-27T14:53:10.700 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2023-11-27T14:53:10.722 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2023-11-27T14:53:10.723 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2023-11-27T14:53:10.782 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="Red Hat Enterprise Linux" 2023-11-27T14:53:10.782 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="8.6 (Ootpa)" 2023-11-27T14:53:10.782 INFO:teuthology.orchestra.run.smithi060.stdout:ID="rhel" 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="fedora" 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="8.6" 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el8" 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://www.redhat.com/" 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T14:53:10.783 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-27T14:53:10.784 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-27T14:53:10.784 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-27T14:53:10.784 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-27T14:53:10.784 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2023-11-27T14:53:10.808 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-27T14:53:10.819 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-27T14:53:10.865 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-27T14:53:10.866 DEBUG:teuthology.orchestra.run.smithi008:> test '!' -e /home/ubuntu/cephtest 2023-11-27T14:53:10.868 DEBUG:teuthology.orchestra.run.smithi029:> test '!' -e /home/ubuntu/cephtest 2023-11-27T14:53:10.870 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2023-11-27T14:53:10.890 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-27T14:53:10.897 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-27T14:53:10.897 DEBUG:teuthology.orchestra.run.smithi008:> test -z $(ls -A /var/lib/ceph) 2023-11-27T14:53:10.927 DEBUG:teuthology.orchestra.run.smithi029:> test -z $(ls -A /var/lib/ceph) 2023-11-27T14:53:10.930 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2023-11-27T14:53:10.962 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-27T14:53:11.033 INFO:teuthology.run_tasks:Running task kernel... 2023-11-27T14:53:11.047 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-11-27T14:53:11.047 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-11-27T14:53:11.048 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-11-27T14:53:11.048 DEBUG:teuthology.orchestra.run.smithi008:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-27T14:53:11.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:53:11.068 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-27T14:53:11.309 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-27T14:53:11.310 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-27T14:53:11.310 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2023-11-27T14:53:11.330 INFO:teuthology.orchestra.run.smithi008.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-27T14:53:11.330 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-27T14:53:11.331 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-27T14:53:11.331 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum upgrade -y linux-firmware 2023-11-27T14:53:12.505 INFO:teuthology.orchestra.run.smithi008.stdout:Updating Subscription Management repositories. 2023-11-27T14:53:12.506 INFO:teuthology.orchestra.run.smithi008.stdout:Unable to read consumer identity 2023-11-27T14:53:13.917 INFO:teuthology.orchestra.run.smithi008.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2023-11-27T14:53:14.642 INFO:teuthology.orchestra.run.smithi008.stdout:Extra Packages for Enterprise Linux 23 MB/s | 16 MB 00:00 2023-11-27T14:53:18.551 INFO:teuthology.orchestra.run.smithi008.stdout:lab-extras 388 kB/s | 24 kB 00:00 2023-11-27T14:53:19.355 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-11-27T14:53:19.356 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2023-11-27T14:53:19.356 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-11-27T14:53:19.483 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-27T14:53:19.483 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-27T14:53:19.700 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ 2023-11-27T14:53:19.700 DEBUG:teuthology.orchestra.run.smithi008:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-27T14:53:19.702 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi008.front.sepia.ceph.com to complete... 2023-11-27T14:53:32.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27 14:53:32 URL:https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [383178732/383178732] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-27T14:53:32.793 INFO:teuthology.orchestra.run.smithi008.stderr:FINISHED --2023-11-27 14:53:32-- 2023-11-27T14:53:32.793 INFO:teuthology.orchestra.run.smithi008.stderr:Total wall clock time: 13s 2023-11-27T14:53:32.793 INFO:teuthology.orchestra.run.smithi008.stderr:Downloaded: 1 files, 365M in 13s (28.6 MB/s) 2023-11-27T14:53:32.795 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-27T14:53:32.795 DEBUG:teuthology.orchestra.run.smithi008:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-27T14:53:34.443 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying... ######################################## 2023-11-27T14:53:34.504 INFO:teuthology.orchestra.run.smithi008.stdout:Preparing... ######################################## 2023-11-27T14:53:34.618 INFO:teuthology.orchestra.run.smithi008.stdout:Updating / installing... 2023-11-27T14:54:15.466 INFO:teuthology.orchestra.run.smithi008.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-27T14:54:17.073 INFO:teuthology.orchestra.run.smithi008.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-27T14:54:39.484 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi008.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-27T14:54:39.484 DEBUG:teuthology.orchestra.run.smithi008:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-27T14:54:39.539 INFO:teuthology.orchestra.run.smithi008.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-27T14:54:39.539 INFO:teuthology.orchestra.run.smithi008.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-27T14:54:39.539 INFO:teuthology.orchestra.run.smithi008.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-27T14:54:39.539 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-27T14:54:39.539 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-27T14:54:39.539 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-27T14:54:39.539 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-27T14:54:39.540 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-27T14:54:39.541 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-27T14:54:39.542 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-27T14:54:39.542 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-27T14:54:39.542 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-27T14:54:39.542 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-27T14:54:39.542 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-27T14:54:39.542 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-27T14:54:39.542 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-27T14:54:39.542 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-27T14:54:39.542 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-27T14:54:39.543 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-27T14:54:39.544 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-27T14:54:39.545 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-27T14:54:39.546 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-27T14:54:39.547 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-27T14:54:39.548 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-27T14:54:39.549 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-27T14:54:39.550 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-27T14:54:39.551 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-27T14:54:39.552 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-27T14:54:39.553 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-27T14:54:39.554 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-27T14:54:39.555 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-27T14:54:39.556 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-27T14:54:39.557 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-27T14:54:39.558 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-27T14:54:39.559 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-27T14:54:39.560 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-27T14:54:39.561 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-27T14:54:39.562 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-27T14:54:39.563 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-27T14:54:39.564 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-27T14:54:39.565 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-27T14:54:39.566 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-27T14:54:39.567 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-27T14:54:39.568 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-27T14:54:39.569 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-27T14:54:39.570 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-27T14:54:39.571 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-27T14:54:39.572 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-27T14:54:39.573 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-27T14:54:39.574 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-27T14:54:39.575 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-27T14:54:39.576 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-27T14:54:39.577 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-27T14:54:39.578 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-27T14:54:39.579 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-27T14:54:39.580 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-27T14:54:39.581 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-27T14:54:39.582 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-27T14:54:39.585 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-27T14:54:39.585 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-27T14:54:39.585 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-27T14:54:39.585 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-27T14:54:39.585 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-27T14:54:39.585 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-27T14:54:39.585 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-27T14:54:39.585 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-27T14:54:39.585 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-27T14:54:39.586 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-27T14:54:39.587 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-27T14:54:39.588 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-27T14:54:39.589 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-27T14:54:39.590 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-27T14:54:39.591 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-27T14:54:39.592 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-27T14:54:39.593 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-27T14:54:39.594 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-27T14:54:39.595 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-27T14:54:39.596 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-27T14:54:39.597 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-27T14:54:39.598 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-27T14:54:39.598 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-27T14:54:39.598 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-27T14:54:39.598 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-27T14:54:39.598 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-27T14:54:39.598 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-27T14:54:39.598 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-27T14:54:39.598 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-27T14:54:39.598 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-27T14:54:39.599 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-27T14:54:39.600 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-27T14:54:39.601 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-27T14:54:39.602 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-27T14:54:39.603 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-27T14:54:39.604 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-27T14:54:39.605 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-27T14:54:39.606 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-27T14:54:39.607 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-27T14:54:39.608 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-27T14:54:39.609 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-27T14:54:39.610 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-27T14:54:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-27T14:54:39.612 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-27T14:54:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-27T14:54:39.614 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-27T14:54:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-27T14:54:39.616 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-27T14:54:39.617 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-27T14:54:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-27T14:54:39.619 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-27T14:54:39.620 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-27T14:54:39.621 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-27T14:54:39.622 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-27T14:54:39.623 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-27T14:54:39.624 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-27T14:54:39.625 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-27T14:54:39.626 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-27T14:54:39.627 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-27T14:54:39.628 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-27T14:54:39.629 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-27T14:54:39.630 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-27T14:54:39.631 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-27T14:54:39.632 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-27T14:54:39.633 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-27T14:54:39.634 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-27T14:54:39.635 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-27T14:54:39.636 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-27T14:54:39.637 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-27T14:54:39.638 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-27T14:54:39.639 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-27T14:54:39.640 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-27T14:54:39.641 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-27T14:54:39.642 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-27T14:54:39.643 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-27T14:54:39.644 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-27T14:54:39.645 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-27T14:54:39.646 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-27T14:54:39.647 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-27T14:54:39.648 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-27T14:54:39.649 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-27T14:54:39.650 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-27T14:54:39.651 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-27T14:54:39.652 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-27T14:54:39.653 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-27T14:54:39.654 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-27T14:54:39.655 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-27T14:54:39.656 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-27T14:54:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-27T14:54:39.658 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-27T14:54:39.659 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-27T14:54:39.660 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-27T14:54:39.661 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-27T14:54:39.662 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-27T14:54:39.663 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-27T14:54:39.664 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-27T14:54:39.665 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-27T14:54:39.666 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-27T14:54:39.667 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-27T14:54:39.668 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-27T14:54:39.669 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-27T14:54:39.670 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-27T14:54:39.671 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-27T14:54:39.672 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-27T14:54:39.673 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-27T14:54:39.674 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-27T14:54:39.675 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-27T14:54:39.676 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-27T14:54:39.677 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-27T14:54:39.678 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-27T14:54:39.679 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-27T14:54:39.680 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-27T14:54:39.681 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-27T14:54:39.682 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-27T14:54:39.683 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-27T14:54:39.684 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-27T14:54:39.685 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-27T14:54:39.686 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-27T14:54:39.687 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-27T14:54:39.688 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-27T14:54:39.689 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-27T14:54:39.690 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-27T14:54:39.691 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-27T14:54:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-27T14:54:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-27T14:54:39.693 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-27T14:54:39.693 DEBUG:teuthology.orchestra.run.smithi008:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-27T14:54:39.719 INFO:teuthology.orchestra.run.smithi008.stdout:postinstall scriptlet (using /bin/sh): 2023-11-27T14:54:39.720 INFO:teuthology.orchestra.run.smithi008.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-27T14:54:39.720 INFO:teuthology.orchestra.run.smithi008.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-27T14:54:39.720 INFO:teuthology.orchestra.run.smithi008.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-27T14:54:39.720 INFO:teuthology.orchestra.run.smithi008.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-27T14:54:39.720 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:fi 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:fi 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-27T14:54:39.721 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-27T14:54:39.722 INFO:teuthology.orchestra.run.smithi008.stdout:fi 2023-11-27T14:54:39.723 DEBUG:teuthology.orchestra.run.smithi008:> sudo rpm -qi grub2-tools 2023-11-27T14:54:39.811 INFO:teuthology.orchestra.run.smithi008.stdout:Name : grub2-tools 2023-11-27T14:54:39.811 INFO:teuthology.orchestra.run.smithi008.stdout:Epoch : 1 2023-11-27T14:54:39.811 INFO:teuthology.orchestra.run.smithi008.stdout:Version : 2.02 2023-11-27T14:54:39.811 INFO:teuthology.orchestra.run.smithi008.stdout:Release : 123.el8 2023-11-27T14:54:39.811 INFO:teuthology.orchestra.run.smithi008.stdout:Architecture: x86_64 2023-11-27T14:54:39.811 INFO:teuthology.orchestra.run.smithi008.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-11-27T14:54:39.811 INFO:teuthology.orchestra.run.smithi008.stdout:Group : System Environment/Base 2023-11-27T14:54:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:Size : 9265450 2023-11-27T14:54:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:License : GPLv3+ 2023-11-27T14:54:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-11-27T14:54:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-11-27T14:54:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-11-27T14:54:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-11-27T14:54:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:Relocations : (not relocatable) 2023-11-27T14:54:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:Packager : Red Hat, Inc. 2023-11-27T14:54:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:Vendor : Red Hat, Inc. 2023-11-27T14:54:39.812 INFO:teuthology.orchestra.run.smithi008.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-27T14:54:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:Summary : Support tools for GRUB. 2023-11-27T14:54:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:Description : 2023-11-27T14:54:39.813 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T14:54:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-27T14:54:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-27T14:54:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-27T14:54:39.813 INFO:teuthology.orchestra.run.smithi008.stdout:hardware devices. 2023-11-27T14:54:39.814 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T14:54:39.814 INFO:teuthology.orchestra.run.smithi008.stdout:This subpackage provides tools for support of all platforms. 2023-11-27T14:54:39.815 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-27T14:54:39.815 INFO:teuthology.task.kernel:Updating grub on smithi008 to boot 6.6.0-g8247bbb5b470 2023-11-27T14:54:39.815 DEBUG:teuthology.orchestra.run.smithi008:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-27T14:54:44.024 INFO:teuthology.orchestra.run.smithi008.stderr:Generating grub configuration file ... 2023-11-27T14:54:45.969 INFO:teuthology.orchestra.run.smithi008.stderr:done 2023-11-27T14:54:45.971 DEBUG:teuthology.orchestra.run.smithi008:> mktemp 2023-11-27T14:54:46.044 INFO:teuthology.orchestra.run.smithi008.stdout:/tmp/tmp.h31XFZEjfz 2023-11-27T14:54:46.044 DEBUG:teuthology.orchestra.run.smithi008:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.h31XFZEjfz 2023-11-27T14:54:46.123 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0666 /tmp/tmp.h31XFZEjfz 2023-11-27T14:54:46.272 DEBUG:teuthology.orchestra.remote:smithi008:/tmp/tmp.h31XFZEjfz is 6KB 2023-11-27T14:54:46.321 DEBUG:teuthology.orchestra.run.smithi008:> rm -fr /tmp/tmp.h31XFZEjfz 2023-11-27T14:54:46.338 DEBUG:teuthology.orchestra.run.smithi008:> sudo /bin/ls /boot/loader/entries || true 2023-11-27T14:54:46.420 INFO:teuthology.orchestra.run.smithi008.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-11-27T14:54:46.420 INFO:teuthology.orchestra.run.smithi008.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-11-27T14:54:46.420 INFO:teuthology.orchestra.run.smithi008.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470.conf 2023-11-27T14:54:46.421 DEBUG:teuthology.orchestra.run.smithi008:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470 2023-11-27T14:54:46.579 DEBUG:teuthology.orchestra.run.smithi008:> sudo shutdown -r now 2023-11-27T14:55:16.582 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-27T14:55:16.583 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-11-27T14:55:16.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:55:35.010 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.8 2023-11-27T14:55:44.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-11-27T14:55:44.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:55:47.074 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.8 2023-11-27T14:55:59.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-11-27T14:55:59.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:56:30.937 DEBUG:teuthology.orchestra.run.smithi008:> true 2023-11-27T14:56:31.329 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-11-27T14:56:31.329 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g8247bbb5b470"... 2023-11-27T14:56:31.329 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2023-11-27T14:56:31.371 INFO:teuthology.orchestra.run.smithi008.stdout:6.6.0-g8247bbb5b470 2023-11-27T14:56:31.371 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-27T14:56:31.371 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-27T14:56:31.372 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-11-27T14:56:32.372 DEBUG:teuthology.parallel:result is None 2023-11-27T14:56:32.372 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-27T14:56:32.380 INFO:teuthology.task.internal:Creating test directory... 2023-11-27T14:56:32.380 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-27T14:56:32.384 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-27T14:56:32.387 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-27T14:56:32.424 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-27T14:56:32.430 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-27T14:56:32.440 INFO:teuthology.task.internal:Creating archive directory... 2023-11-27T14:56:32.440 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-27T14:56:32.472 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-27T14:56:32.476 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-27T14:56:32.533 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-27T14:56:32.540 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-27T14:56:32.540 DEBUG:teuthology.orchestra.run.smithi029:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-27T14:56:32.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:56:32.565 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-27T14:56:32.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:56:32.585 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-27T14:56:32.589 DEBUG:teuthology.orchestra.run.smithi029:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-27T14:56:32.607 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-27T14:56:32.649 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:56:32.670 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:56:32.695 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:56:32.714 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:56:32.728 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:56:32.764 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:56:32.767 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-27T14:56:32.775 INFO:teuthology.task.internal:Configuring sudo... 2023-11-27T14:56:32.775 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-27T14:56:32.779 DEBUG:teuthology.orchestra.run.smithi029:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-27T14:56:32.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-27T14:56:32.845 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-27T14:56:32.854 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-27T14:56:32.854 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-27T14:56:32.858 DEBUG:teuthology.orchestra.run.smithi029:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-27T14:56:32.861 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-27T14:56:32.897 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:56:33.012 DEBUG:teuthology.orchestra.run.smithi008:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:56:33.114 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:56:33.184 DEBUG:teuthology.orchestra.run.smithi008:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:56:33.275 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T14:56:33.275 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-27T14:56:33.344 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:56:33.391 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:56:33.448 DEBUG:teuthology.orchestra.run.smithi029:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:56:33.536 DEBUG:teuthology.orchestra.run.smithi029:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:56:33.582 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T14:56:33.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-27T14:56:33.665 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:56:33.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:56:33.774 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:56:33.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:56:33.908 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T14:56:33.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-27T14:56:34.011 DEBUG:teuthology.orchestra.run.smithi008:> sudo service rsyslog restart 2023-11-27T14:56:34.016 DEBUG:teuthology.orchestra.run.smithi029:> sudo service rsyslog restart 2023-11-27T14:56:34.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2023-11-27T14:56:34.088 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T14:56:34.089 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T14:56:34.128 INFO:teuthology.orchestra.run.smithi008.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T14:56:34.656 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-27T14:56:34.664 INFO:teuthology.task.internal:Starting timer... 2023-11-27T14:56:34.664 INFO:teuthology.run_tasks:Running task pcp... 2023-11-27T14:56:34.675 INFO:teuthology.run_tasks:Running task selinux... 2023-11-27T14:56:34.683 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-27T14:56:34.683 DEBUG:teuthology.orchestra.run.smithi008:> sudo service auditd rotate 2023-11-27T14:56:34.794 INFO:teuthology.orchestra.run.smithi008.stdout:Rotating logs: 2023-11-27T14:56:34.798 DEBUG:teuthology.orchestra.run.smithi029:> sudo service auditd rotate 2023-11-27T14:56:34.852 INFO:teuthology.orchestra.run.smithi029.stdout:Rotating logs: 2023-11-27T14:56:34.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2023-11-27T14:56:34.920 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2023-11-27T14:56:34.922 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-27T14:56:34.922 DEBUG:teuthology.orchestra.run.smithi008:> /usr/sbin/getenforce 2023-11-27T14:56:34.976 INFO:teuthology.orchestra.run.smithi008.stdout:Permissive 2023-11-27T14:56:34.976 DEBUG:teuthology.orchestra.run.smithi029:> /usr/sbin/getenforce 2023-11-27T14:56:34.996 INFO:teuthology.orchestra.run.smithi029.stdout:Permissive 2023-11-27T14:56:34.997 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2023-11-27T14:56:35.022 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2023-11-27T14:56:35.023 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi008.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi029.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi060.front.sepia.ceph.com': 'permissive'} 2023-11-27T14:56:35.023 DEBUG:teuthology.orchestra.run.smithi008:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T14:56:35.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:56:35.093 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T14:56:35.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:56:35.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T14:56:35.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:56:35.151 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-27T14:56:35.151 DEBUG:teuthology.orchestra.run.smithi008:> sudo /usr/sbin/setenforce permissive 2023-11-27T14:56:35.230 DEBUG:teuthology.orchestra.run.smithi029:> sudo /usr/sbin/setenforce permissive 2023-11-27T14:56:35.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2023-11-27T14:56:35.309 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-27T14:56:35.319 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-27T14:56:35.319 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-27T14:56:35.340 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-27T14:56:35.341 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi008.front.sepia.ceph.com,smithi029.front.sepia.ceph.com,smithi060.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-27T15:10:51.610 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi008.front.sepia.ceph.com'), Remote(name='ubuntu@smithi029.front.sepia.ceph.com'), Remote(name='ubuntu@smithi060.front.sepia.ceph.com')] 2023-11-27T15:10:51.611 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-11-27T15:10:51.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T15:10:51.702 DEBUG:teuthology.orchestra.run.smithi008:> true 2023-11-27T15:10:51.812 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2023-11-27T15:10:51.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi029.front.sepia.ceph.com' 2023-11-27T15:10:51.813 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi029.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T15:10:51.881 DEBUG:teuthology.orchestra.run.smithi029:> true 2023-11-27T15:10:51.957 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi029.front.sepia.ceph.com' 2023-11-27T15:10:51.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-11-27T15:10:51.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T15:10:52.027 DEBUG:teuthology.orchestra.run.smithi060:> true 2023-11-27T15:10:52.098 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2023-11-27T15:10:52.099 INFO:teuthology.run_tasks:Running task clock... 2023-11-27T15:10:52.109 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-27T15:10:52.110 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-27T15:10:52.110 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:10:52.114 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-27T15:10:52.115 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:10:52.117 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-27T15:10:52.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:10:52.152 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-27T15:10:52.169 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-27T15:10:52.174 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-27T15:10:52.186 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-27T15:10:52.191 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-27T15:10:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-27T15:10:52.253 INFO:teuthology.orchestra.run.smithi029.stderr:sudo: ntpd: command not found 2023-11-27T15:10:52.266 INFO:teuthology.orchestra.run.smithi029.stdout:506 Cannot talk to daemon 2023-11-27T15:10:52.271 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2023-11-27T15:10:52.282 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-27T15:10:52.284 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2023-11-27T15:10:52.297 INFO:teuthology.orchestra.run.smithi029.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-27T15:10:52.300 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-27T15:10:52.315 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-27T15:10:52.318 INFO:teuthology.orchestra.run.smithi008.stderr:sudo: ntpd: command not found 2023-11-27T15:10:52.348 INFO:teuthology.orchestra.run.smithi008.stdout:506 Cannot talk to daemon 2023-11-27T15:10:52.375 INFO:teuthology.orchestra.run.smithi029.stderr:bash: ntpq: command not found 2023-11-27T15:10:52.379 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:10:52.379 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2023-11-27T15:10:52.379 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.379 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.380 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.380 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.383 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-27T15:10:52.385 INFO:teuthology.orchestra.run.smithi060.stderr:bash: ntpq: command not found 2023-11-27T15:10:52.389 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:10:52.389 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2023-11-27T15:10:52.389 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.389 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.389 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.389 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.418 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-27T15:10:52.515 INFO:teuthology.orchestra.run.smithi008.stderr:bash: ntpq: command not found 2023-11-27T15:10:52.523 INFO:teuthology.orchestra.run.smithi008.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:10:52.523 INFO:teuthology.orchestra.run.smithi008.stdout:=============================================================================== 2023-11-27T15:10:52.523 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.523 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.523 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.523 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:10:52.524 INFO:teuthology.run_tasks:Running task pexec... 2023-11-27T15:10:52.534 INFO:teuthology.task.pexec:Executing custom commands... 2023-11-27T15:10:52.534 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi008.front.sepia.ceph.com 2023-11-27T15:10:52.534 DEBUG:teuthology.orchestra.run.smithi008:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-27T15:10:52.535 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi029.front.sepia.ceph.com 2023-11-27T15:10:52.535 DEBUG:teuthology.orchestra.run.smithi029:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-27T15:10:52.535 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi060.front.sepia.ceph.com 2023-11-27T15:10:52.535 DEBUG:teuthology.orchestra.run.smithi060:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-27T15:10:52.928 INFO:teuthology.orchestra.run.smithi060.stdout:Updating Subscription Management repositories. 2023-11-27T15:10:52.931 INFO:teuthology.orchestra.run.smithi029.stdout:Updating Subscription Management repositories. 2023-11-27T15:10:53.052 INFO:teuthology.orchestra.run.smithi008.stdout:Updating Subscription Management repositories. 2023-11-27T15:10:54.584 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:05:16 ago on Mon 27 Nov 2023 03:05:38 PM UTC. 2023-11-27T15:10:54.634 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:04:57 ago on Mon 27 Nov 2023 03:05:57 PM UTC. 2023-11-27T15:10:54.962 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:08 ago on Mon 27 Nov 2023 03:08:46 PM UTC. 2023-11-27T15:10:56.529 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-11-27T15:10:56.530 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-11-27T15:10:56.530 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2023-11-27T15:10:56.530 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-11-27T15:10:56.531 INFO:teuthology.orchestra.run.smithi060.stdout:Resetting modules: 2023-11-27T15:10:56.531 INFO:teuthology.orchestra.run.smithi060.stdout: container-tools 2023-11-27T15:10:56.531 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:10:56.531 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-11-27T15:10:56.531 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2023-11-27T15:10:56.531 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:10:56.639 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2023-11-27T15:10:56.640 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2023-11-27T15:10:56.640 INFO:teuthology.orchestra.run.smithi029.stdout: Package Architecture Version Repository Size 2023-11-27T15:10:56.640 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2023-11-27T15:10:56.640 INFO:teuthology.orchestra.run.smithi029.stdout:Resetting modules: 2023-11-27T15:10:56.641 INFO:teuthology.orchestra.run.smithi029.stdout: container-tools 2023-11-27T15:10:56.641 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:10:56.641 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2023-11-27T15:10:56.641 INFO:teuthology.orchestra.run.smithi029.stdout:================================================================================ 2023-11-27T15:10:56.641 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:10:57.028 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-11-27T15:10:57.029 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-11-27T15:10:57.029 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2023-11-27T15:10:57.029 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-11-27T15:10:57.029 INFO:teuthology.orchestra.run.smithi008.stdout:Resetting modules: 2023-11-27T15:10:57.029 INFO:teuthology.orchestra.run.smithi008.stdout: container-tools 2023-11-27T15:10:57.030 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:10:57.030 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-11-27T15:10:57.030 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2023-11-27T15:10:57.030 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:10:57.604 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:05:19 ago on Mon 27 Nov 2023 03:05:38 PM UTC. 2023-11-27T15:10:57.646 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:05:00 ago on Mon 27 Nov 2023 03:05:57 PM UTC. 2023-11-27T15:10:58.322 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:11 ago on Mon 27 Nov 2023 03:08:46 PM UTC. 2023-11-27T15:11:04.926 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-11-27T15:11:04.933 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2023-11-27T15:11:05.428 INFO:teuthology.orchestra.run.smithi029.stdout:Updating Subscription Management repositories. 2023-11-27T15:11:05.544 INFO:teuthology.orchestra.run.smithi008.stdout:Updating Subscription Management repositories. 2023-11-27T15:11:06.433 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-11-27T15:11:06.944 INFO:teuthology.orchestra.run.smithi060.stdout:Updating Subscription Management repositories. 2023-11-27T15:11:07.002 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:05:09 ago on Mon 27 Nov 2023 03:05:57 PM UTC. 2023-11-27T15:11:07.191 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:20 ago on Mon 27 Nov 2023 03:08:46 PM UTC. 2023-11-27T15:11:08.477 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:05:30 ago on Mon 27 Nov 2023 03:05:38 PM UTC. 2023-11-27T15:11:08.996 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2023-11-27T15:11:09.096 INFO:teuthology.orchestra.run.smithi029.stdout:============================================================================================================ 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout:============================================================================================================ 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout:Installing group/module packages: 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2023-11-27T15:11:09.097 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout:Installing module profiles: 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: container-tools/common 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout:Enabling module streams: 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: container-tools rhel8 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout:============================================================================================================ 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout:Install 10 Packages 2023-11-27T15:11:09.098 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:11:09.099 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 18 M 2023-11-27T15:11:09.099 INFO:teuthology.orchestra.run.smithi029.stdout:Installed size: 65 M 2023-11-27T15:11:09.099 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2023-11-27T15:11:09.190 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-11-27T15:11:09.230 INFO:teuthology.orchestra.run.smithi029.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 719 kB/s | 94 kB 00:00 2023-11-27T15:11:09.297 INFO:teuthology.orchestra.run.smithi029.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 206 kB/s | 41 kB 00:00 2023-11-27T15:11:09.298 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================================================ 2023-11-27T15:11:09.298 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-11-27T15:11:09.298 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================================================ 2023-11-27T15:11:09.298 INFO:teuthology.orchestra.run.smithi008.stdout:Installing group/module packages: 2023-11-27T15:11:09.299 INFO:teuthology.orchestra.run.smithi008.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-27T15:11:09.299 INFO:teuthology.orchestra.run.smithi008.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-27T15:11:09.299 INFO:teuthology.orchestra.run.smithi008.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-27T15:11:09.299 INFO:teuthology.orchestra.run.smithi008.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-27T15:11:09.299 INFO:teuthology.orchestra.run.smithi008.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-27T15:11:09.299 INFO:teuthology.orchestra.run.smithi008.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-27T15:11:09.300 INFO:teuthology.orchestra.run.smithi008.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-27T15:11:09.300 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2023-11-27T15:11:09.300 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-27T15:11:09.300 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-27T15:11:09.300 INFO:teuthology.orchestra.run.smithi008.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-27T15:11:09.300 INFO:teuthology.orchestra.run.smithi008.stdout:Installing module profiles: 2023-11-27T15:11:09.301 INFO:teuthology.orchestra.run.smithi008.stdout: container-tools/common 2023-11-27T15:11:09.301 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:11:09.301 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling module streams: 2023-11-27T15:11:09.301 INFO:teuthology.orchestra.run.smithi008.stdout: container-tools rhel8 2023-11-27T15:11:09.301 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:11:09.302 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-11-27T15:11:09.302 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================================================ 2023-11-27T15:11:09.302 INFO:teuthology.orchestra.run.smithi008.stdout:Install 10 Packages 2023-11-27T15:11:09.302 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:11:09.303 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 18 M 2023-11-27T15:11:09.303 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 65 M 2023-11-27T15:11:09.303 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-11-27T15:11:09.322 INFO:teuthology.orchestra.run.smithi029.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 111 kB/s | 25 kB 00:00 2023-11-27T15:11:09.348 INFO:teuthology.orchestra.run.smithi029.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 418 kB/s | 49 kB 00:00 2023-11-27T15:11:09.457 INFO:teuthology.orchestra.run.smithi029.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.2 MB/s | 524 kB 00:00 2023-11-27T15:11:09.477 INFO:teuthology.orchestra.run.smithi008.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 532 kB/s | 94 kB 00:00 2023-11-27T15:11:09.494 INFO:teuthology.orchestra.run.smithi008.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 211 kB/s | 41 kB 00:00 2023-11-27T15:11:09.511 INFO:teuthology.orchestra.run.smithi008.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 118 kB/s | 25 kB 00:00 2023-11-27T15:11:09.524 INFO:teuthology.orchestra.run.smithi029.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 851 kB/s | 149 kB 00:00 2023-11-27T15:11:09.575 INFO:teuthology.orchestra.run.smithi029.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.8 MB/s | 213 kB 00:00 2023-11-27T15:11:09.661 INFO:teuthology.orchestra.run.smithi008.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 269 kB/s | 49 kB 00:00 2023-11-27T15:11:09.703 INFO:teuthology.orchestra.run.smithi008.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 2.4 MB/s | 524 kB 00:00 2023-11-27T15:11:09.733 INFO:teuthology.orchestra.run.smithi029.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 19 MB/s | 8.0 MB 00:00 2023-11-27T15:11:09.770 INFO:teuthology.orchestra.run.smithi008.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.3 MB/s | 149 kB 00:00 2023-11-27T15:11:09.825 INFO:teuthology.orchestra.run.smithi029.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 8.8 MB/s | 2.2 MB 00:00 2023-11-27T15:11:09.829 INFO:teuthology.orchestra.run.smithi008.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.7 MB/s | 213 kB 00:00 2023-11-27T15:11:09.979 INFO:teuthology.orchestra.run.smithi008.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 17 MB/s | 8.0 MB 00:00 2023-11-27T15:11:09.986 INFO:teuthology.orchestra.run.smithi029.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 15 MB/s | 7.0 MB 00:00 2023-11-27T15:11:09.986 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:11:09.986 INFO:teuthology.orchestra.run.smithi029.stdout:Total 21 MB/s | 18 MB 00:00 2023-11-27T15:11:10.090 INFO:teuthology.orchestra.run.smithi008.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 8.4 MB/s | 2.2 MB 00:00 2023-11-27T15:11:10.194 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2023-11-27T15:11:10.215 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2023-11-27T15:11:10.215 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2023-11-27T15:11:10.221 INFO:teuthology.orchestra.run.smithi008.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 16 MB/s | 7.0 MB 00:00 2023-11-27T15:11:10.222 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:11:10.222 INFO:teuthology.orchestra.run.smithi008.stdout:Total 20 MB/s | 18 MB 00:00 2023-11-27T15:11:10.408 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2023-11-27T15:11:10.412 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2023-11-27T15:11:10.503 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-11-27T15:11:10.567 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-11-27T15:11:10.592 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-11-27T15:11:10.592 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-11-27T15:11:10.605 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================================ 2023-11-27T15:11:10.605 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-11-27T15:11:10.605 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================================ 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout:Installing group/module packages: 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-27T15:11:10.606 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout:Installing module profiles: 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout: container-tools/common 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling module streams: 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout: container-tools rhel8 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================================================ 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout:Install 10 Packages 2023-11-27T15:11:10.607 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:11:10.608 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 18 M 2023-11-27T15:11:10.608 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 65 M 2023-11-27T15:11:10.608 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-11-27T15:11:10.750 INFO:teuthology.orchestra.run.smithi060.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 662 kB/s | 94 kB 00:00 2023-11-27T15:11:10.775 INFO:teuthology.orchestra.run.smithi060.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 149 kB/s | 25 kB 00:00 2023-11-27T15:11:10.801 INFO:teuthology.orchestra.run.smithi060.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 212 kB/s | 41 kB 00:00 2023-11-27T15:11:10.826 INFO:teuthology.orchestra.run.smithi060.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-11-27T15:11:10.885 INFO:teuthology.orchestra.run.smithi060.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.7 MB/s | 524 kB 00:00 2023-11-27T15:11:10.930 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2023-11-27T15:11:10.977 INFO:teuthology.orchestra.run.smithi060.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 993 kB/s | 149 kB 00:00 2023-11-27T15:11:11.002 INFO:teuthology.orchestra.run.smithi060.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.8 MB/s | 213 kB 00:00 2023-11-27T15:11:11.118 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-27T15:11:11.119 INFO:teuthology.orchestra.run.smithi060.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 25 MB/s | 8.0 MB 00:00 2023-11-27T15:11:11.154 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-11-27T15:11:11.159 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-11-27T15:11:11.211 INFO:teuthology.orchestra.run.smithi060.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-11-27T15:11:11.288 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:11:11.328 INFO:teuthology.orchestra.run.smithi060.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 20 MB/s | 7.0 MB 00:00 2023-11-27T15:11:11.329 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:11:11.329 INFO:teuthology.orchestra.run.smithi060.stdout:Total 25 MB/s | 18 MB 00:00 2023-11-27T15:11:11.485 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-27T15:11:11.537 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-11-27T15:11:11.557 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-11-27T15:11:11.558 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-11-27T15:11:11.689 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-27T15:11:11.720 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-11-27T15:11:11.751 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-11-27T15:11:11.756 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-11-27T15:11:11.984 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-27T15:11:12.070 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-27T15:11:12.162 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:11:12.328 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-11-27T15:11:12.452 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-27T15:11:12.504 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-27T15:11:12.648 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-27T15:11:12.668 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:11:12.873 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-27T15:11:12.875 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-27T15:11:13.050 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-27T15:11:13.114 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-27T15:11:13.469 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-27T15:11:13.827 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-27T15:11:13.942 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-27T15:11:14.055 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-27T15:11:14.244 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-27T15:11:14.262 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-27T15:11:14.454 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:11:14.738 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:11:14.739 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-27T15:11:14.739 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:11:14.739 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-27T15:11:14.739 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-27T15:11:14.739 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-27T15:11:14.739 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-27T15:11:14.739 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-27T15:11:14.739 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-27T15:11:14.739 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-27T15:11:14.942 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-27T15:11:15.202 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-27T15:11:15.206 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-27T15:11:15.433 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-27T15:11:15.437 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-27T15:11:15.590 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-27T15:11:15.624 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:11:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:11:15.943 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-27T15:11:15.943 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:05:18 ago on Mon 27 Nov 2023 03:05:57 PM UTC. 2023-11-27T15:11:16.006 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:11:16.006 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-27T15:11:16.006 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:11:16.006 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-27T15:11:16.006 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-27T15:11:16.006 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-27T15:11:16.006 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-27T15:11:16.006 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-27T15:11:16.007 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-27T15:11:16.007 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-27T15:11:16.090 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:11:16.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-27T15:11:16.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:11:16.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-27T15:11:16.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-27T15:11:16.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-27T15:11:16.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-27T15:11:16.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-27T15:11:16.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-27T15:11:16.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-27T15:11:17.339 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-27T15:11:17.339 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:05:38 ago on Mon 27 Nov 2023 03:05:38 PM UTC. 2023-11-27T15:11:17.433 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-27T15:11:17.433 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:30 ago on Mon 27 Nov 2023 03:08:46 PM UTC. 2023-11-27T15:11:22.767 INFO:teuthology.orchestra.run.smithi029.stdout:Installed products updated. 2023-11-27T15:11:22.983 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:11:22.983 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2023-11-27T15:11:22.983 INFO:teuthology.orchestra.run.smithi029.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:22.983 INFO:teuthology.orchestra.run.smithi029.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:11:22.983 INFO:teuthology.orchestra.run.smithi029.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:22.983 INFO:teuthology.orchestra.run.smithi029.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-27T15:11:22.983 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-27T15:11:22.983 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-27T15:11:22.983 INFO:teuthology.orchestra.run.smithi029.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:22.983 INFO:teuthology.orchestra.run.smithi029.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:22.984 INFO:teuthology.orchestra.run.smithi029.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:11:22.984 INFO:teuthology.orchestra.run.smithi029.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-27T15:11:22.984 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:11:22.984 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2023-11-27T15:11:23.147 DEBUG:teuthology.parallel:result is None 2023-11-27T15:11:23.555 INFO:teuthology.orchestra.run.smithi060.stdout:Installed products updated. 2023-11-27T15:11:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:11:23.758 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-11-27T15:11:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:11:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-27T15:11:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-27T15:11:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-27T15:11:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:11:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-27T15:11:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:11:23.759 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-11-27T15:11:23.926 DEBUG:teuthology.parallel:result is None 2023-11-27T15:11:26.210 INFO:teuthology.orchestra.run.smithi008.stdout:Installed products updated. 2023-11-27T15:11:26.489 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:11:26.489 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-11-27T15:11:26.489 INFO:teuthology.orchestra.run.smithi008.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:26.489 INFO:teuthology.orchestra.run.smithi008.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:11:26.489 INFO:teuthology.orchestra.run.smithi008.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:26.489 INFO:teuthology.orchestra.run.smithi008.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-27T15:11:26.489 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-27T15:11:26.489 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-27T15:11:26.489 INFO:teuthology.orchestra.run.smithi008.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:26.490 INFO:teuthology.orchestra.run.smithi008.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:11:26.490 INFO:teuthology.orchestra.run.smithi008.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:11:26.490 INFO:teuthology.orchestra.run.smithi008.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-27T15:11:26.490 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:11:26.490 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-11-27T15:11:26.962 DEBUG:teuthology.parallel:result is None 2023-11-27T15:11:26.963 INFO:teuthology.run_tasks:Running task install... 2023-11-27T15:11:26.975 DEBUG:teuthology.task.install:project ceph 2023-11-27T15:11:26.975 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b'}} 2023-11-27T15:11:26.975 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b'} 2023-11-27T15:11:26.975 INFO:teuthology.task.install:Using flavor: default 2023-11-27T15:11:26.981 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-27T15:11:26.981 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-27T15:11:26.981 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-27T15:11:26.982 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:11:26.984 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-27T15:11:26.984 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:11:26.985 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-27T15:11:26.985 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:11:27.137 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-27T15:11:27.137 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-27T15:11:27.202 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-27T15:11:27.202 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-27T15:11:27.207 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-27T15:11:27.207 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-27T15:11:27.304 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-27T15:11:27.305 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T15:11:27.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-27T15:11:27.332 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-27T15:11:27.332 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-27T15:11:27.371 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-27T15:11:27.371 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:11:27.371 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-27T15:11:27.379 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-27T15:11:27.379 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T15:11:27.379 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-27T15:11:27.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-27T15:11:27.406 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-27T15:11:27.407 DEBUG:teuthology.orchestra.run.smithi029:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-27T15:11:27.445 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-27T15:11:27.445 DEBUG:teuthology.orchestra.run.smithi008:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-27T15:11:27.475 DEBUG:teuthology.orchestra.run.smithi029:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-27T15:11:27.483 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-27T15:11:27.549 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2023-11-27T15:11:27.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2023-11-27T15:11:27.555 DEBUG:teuthology.orchestra.run.smithi029:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-27T15:11:27.558 DEBUG:teuthology.orchestra.run.smithi008:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-27T15:11:27.623 INFO:teuthology.orchestra.run.smithi029.stdout:check_obsoletes = 1 2023-11-27T15:11:27.624 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum clean all 2023-11-27T15:11:27.657 DEBUG:teuthology.orchestra.run.smithi008:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-27T15:11:27.728 INFO:teuthology.orchestra.run.smithi008.stdout:check_obsoletes = 1 2023-11-27T15:11:27.731 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum clean all 2023-11-27T15:11:27.947 INFO:teuthology.orchestra.run.smithi060.stdout:Updating Subscription Management repositories. 2023-11-27T15:11:28.020 INFO:teuthology.orchestra.run.smithi029.stdout:Updating Subscription Management repositories. 2023-11-27T15:11:28.270 INFO:teuthology.orchestra.run.smithi008.stdout:Updating Subscription Management repositories. 2023-11-27T15:11:28.986 INFO:teuthology.orchestra.run.smithi029.stdout:44 files removed 2023-11-27T15:11:29.022 INFO:teuthology.orchestra.run.smithi060.stdout:44 files removed 2023-11-27T15:11:29.036 DEBUG:teuthology.orchestra.run.smithi029:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-27T15:11:29.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-27T15:11:29.364 INFO:teuthology.orchestra.run.smithi008.stdout:44 files removed 2023-11-27T15:11:29.389 INFO:teuthology.orchestra.run.smithi029.stdout:Updating Subscription Management repositories. 2023-11-27T15:11:29.427 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-27T15:11:29.455 INFO:teuthology.orchestra.run.smithi060.stdout:Updating Subscription Management repositories. 2023-11-27T15:11:29.921 INFO:teuthology.orchestra.run.smithi008.stdout:Updating Subscription Management repositories. 2023-11-27T15:11:30.686 INFO:teuthology.orchestra.run.smithi029.stdout:ceph packages for x86_64 291 kB/s | 80 kB 00:00 2023-11-27T15:11:30.968 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 171 kB/s | 80 kB 00:00 2023-11-27T15:11:31.025 INFO:teuthology.orchestra.run.smithi029.stdout:ceph noarch packages 31 kB/s | 9.4 kB 00:00 2023-11-27T15:11:31.276 INFO:teuthology.orchestra.run.smithi029.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2023-11-27T15:11:31.279 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 34 kB/s | 9.4 kB 00:00 2023-11-27T15:11:31.346 INFO:teuthology.orchestra.run.smithi008.stdout:ceph packages for x86_64 257 kB/s | 80 kB 00:00 2023-11-27T15:11:31.445 INFO:teuthology.orchestra.run.smithi029.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-27T15:11:31.649 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 4.3 kB/s | 1.5 kB 00:00 2023-11-27T15:11:31.765 INFO:teuthology.orchestra.run.smithi008.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-11-27T15:11:31.833 INFO:teuthology.orchestra.run.smithi060.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-11-27T15:11:32.058 INFO:teuthology.orchestra.run.smithi008.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2023-11-27T15:11:32.208 INFO:teuthology.orchestra.run.smithi008.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-11-27T15:11:33.148 INFO:teuthology.orchestra.run.smithi029.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 16 MB 00:01 2023-11-27T15:11:33.337 INFO:teuthology.orchestra.run.smithi008.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-11-27T15:11:33.454 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-11-27T15:11:36.873 INFO:teuthology.orchestra.run.smithi029.stdout:lab-extras 398 kB/s | 24 kB 00:00 2023-11-27T15:11:37.227 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 304 kB/s | 24 kB 00:00 2023-11-27T15:11:37.822 INFO:teuthology.orchestra.run.smithi008.stdout:lab-extras 391 kB/s | 24 kB 00:00 2023-11-27T15:11:38.496 INFO:teuthology.orchestra.run.smithi060.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 MB/s | 47 MB 00:01 2023-11-27T15:11:38.593 INFO:teuthology.orchestra.run.smithi029.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 MB/s | 53 MB 00:01 2023-11-27T15:11:39.300 INFO:teuthology.orchestra.run.smithi008.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 MB/s | 47 MB 00:01 2023-11-27T15:11:47.314 INFO:teuthology.orchestra.run.smithi060.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 MB/s | 53 MB 00:01 2023-11-27T15:11:48.108 INFO:teuthology.orchestra.run.smithi029.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 MB/s | 47 MB 00:01 2023-11-27T15:11:49.007 INFO:teuthology.orchestra.run.smithi008.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 MB/s | 53 MB 00:01 2023-11-27T15:11:59.265 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:11:59.266 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:11:59.281 INFO:teuthology.orchestra.run.smithi060.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-27T15:11:59.282 INFO:teuthology.orchestra.run.smithi060.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-27T15:11:59.283 INFO:teuthology.orchestra.run.smithi060.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-27T15:11:59.285 INFO:teuthology.orchestra.run.smithi060.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-27T15:11:59.287 INFO:teuthology.orchestra.run.smithi060.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-27T15:11:59.290 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-27T15:11:59.291 INFO:teuthology.orchestra.run.smithi060.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-27T15:11:59.433 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2023-11-27T15:11:59.446 INFO:teuthology.orchestra.run.smithi060.stdout:=========================================================================================================================== 2023-11-27T15:11:59.446 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2023-11-27T15:11:59.446 INFO:teuthology.orchestra.run.smithi060.stdout:=========================================================================================================================== 2023-11-27T15:11:59.446 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2023-11-27T15:11:59.446 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-27T15:11:59.446 INFO:teuthology.orchestra.run.smithi060.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-27T15:11:59.446 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-27T15:11:59.447 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-27T15:11:59.448 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-27T15:11:59.449 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-27T15:11:59.450 INFO:teuthology.orchestra.run.smithi060.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-27T15:11:59.451 INFO:teuthology.orchestra.run.smithi060.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-27T15:11:59.452 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-27T15:11:59.453 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-27T15:11:59.454 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-27T15:11:59.455 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling module streams: 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql 10 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2023-11-27T15:11:59.456 INFO:teuthology.orchestra.run.smithi060.stdout:=========================================================================================================================== 2023-11-27T15:11:59.457 INFO:teuthology.orchestra.run.smithi060.stdout:Install 128 Packages 2023-11-27T15:11:59.457 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 3 Packages 2023-11-27T15:11:59.457 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:11:59.457 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 197 M 2023-11-27T15:11:59.457 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2023-11-27T15:11:59.890 INFO:teuthology.orchestra.run.smithi029.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:11:59.891 INFO:teuthology.orchestra.run.smithi029.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:11:59.906 INFO:teuthology.orchestra.run.smithi029.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-27T15:11:59.907 INFO:teuthology.orchestra.run.smithi029.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-27T15:11:59.908 INFO:teuthology.orchestra.run.smithi029.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-27T15:11:59.910 INFO:teuthology.orchestra.run.smithi029.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-27T15:11:59.912 INFO:teuthology.orchestra.run.smithi029.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-27T15:11:59.915 INFO:teuthology.orchestra.run.smithi029.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-27T15:11:59.916 INFO:teuthology.orchestra.run.smithi029.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-27T15:11:59.949 INFO:teuthology.orchestra.run.smithi060.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 40 kB/s | 6.4 kB 00:00 2023-11-27T15:12:00.063 INFO:teuthology.orchestra.run.smithi029.stdout:Dependencies resolved. 2023-11-27T15:12:00.076 INFO:teuthology.orchestra.run.smithi029.stdout:=========================================================================================================================== 2023-11-27T15:12:00.077 INFO:teuthology.orchestra.run.smithi029.stdout: Package Arch Version Repository Size 2023-11-27T15:12:00.077 INFO:teuthology.orchestra.run.smithi029.stdout:=========================================================================================================================== 2023-11-27T15:12:00.077 INFO:teuthology.orchestra.run.smithi029.stdout:Installing: 2023-11-27T15:12:00.077 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-27T15:12:00.077 INFO:teuthology.orchestra.run.smithi029.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-27T15:12:00.077 INFO:teuthology.orchestra.run.smithi029.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-27T15:12:00.077 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-27T15:12:00.078 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-27T15:12:00.078 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-27T15:12:00.078 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-27T15:12:00.079 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-27T15:12:00.080 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrading: 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout:Installing dependencies: 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-27T15:12:00.081 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-27T15:12:00.082 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-27T15:12:00.083 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-27T15:12:00.084 INFO:teuthology.orchestra.run.smithi029.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-27T15:12:00.084 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-27T15:12:00.084 INFO:teuthology.orchestra.run.smithi029.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-27T15:12:00.084 INFO:teuthology.orchestra.run.smithi029.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-27T15:12:00.084 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-27T15:12:00.084 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-27T15:12:00.084 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-27T15:12:00.084 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-27T15:12:00.085 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-27T15:12:00.085 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-27T15:12:00.085 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-27T15:12:00.085 INFO:teuthology.orchestra.run.smithi029.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-27T15:12:00.085 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-27T15:12:00.085 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-27T15:12:00.085 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-27T15:12:00.085 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-27T15:12:00.085 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:12:00.086 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-27T15:12:00.087 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-27T15:12:00.088 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-27T15:12:00.088 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-27T15:12:00.088 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-27T15:12:00.088 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-27T15:12:00.088 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:12:00.088 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-27T15:12:00.088 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-27T15:12:00.088 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-27T15:12:00.088 INFO:teuthology.orchestra.run.smithi029.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-27T15:12:00.088 INFO:teuthology.orchestra.run.smithi029.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-27T15:12:00.089 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-27T15:12:00.090 INFO:teuthology.orchestra.run.smithi029.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-27T15:12:00.090 INFO:teuthology.orchestra.run.smithi029.stdout:Enabling module streams: 2023-11-27T15:12:00.090 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql 10 2023-11-27T15:12:00.090 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:12:00.090 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction Summary 2023-11-27T15:12:00.090 INFO:teuthology.orchestra.run.smithi029.stdout:=========================================================================================================================== 2023-11-27T15:12:00.090 INFO:teuthology.orchestra.run.smithi029.stdout:Install 128 Packages 2023-11-27T15:12:00.090 INFO:teuthology.orchestra.run.smithi029.stdout:Upgrade 3 Packages 2023-11-27T15:12:00.090 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:12:00.091 INFO:teuthology.orchestra.run.smithi029.stdout:Total download size: 197 M 2023-11-27T15:12:00.091 INFO:teuthology.orchestra.run.smithi029.stdout:Downloading Packages: 2023-11-27T15:12:00.132 INFO:teuthology.orchestra.run.smithi060.stdout:(2/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 4.5 MB/s | 848 kB 00:00 2023-11-27T15:12:00.182 INFO:teuthology.orchestra.run.smithi060.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 3.2 MB/s | 165 kB 00:00 2023-11-27T15:12:00.407 INFO:teuthology.orchestra.run.smithi060.stdout:(4/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 10 MB/s | 2.3 MB 00:00 2023-11-27T15:12:00.429 INFO:teuthology.orchestra.run.smithi029.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 33 kB/s | 6.4 kB 00:00 2023-11-27T15:12:00.642 INFO:teuthology.orchestra.run.smithi060.stdout:(5/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 6.6 MB/s | 1.5 MB 00:00 2023-11-27T15:12:00.933 INFO:teuthology.orchestra.run.smithi060.stdout:(6/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 4.6 MB/s | 5.3 MB 00:01 2023-11-27T15:12:01.213 INFO:teuthology.orchestra.run.smithi029.stdout:(2/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 5.5 MB/s | 5.3 MB 00:00 2023-11-27T15:12:01.258 INFO:teuthology.orchestra.run.smithi060.stdout:(7/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 7.5 MB/s | 4.6 MB 00:00 2023-11-27T15:12:01.272 INFO:teuthology.orchestra.run.smithi029.stdout:(3/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 1.0 MB/s | 848 kB 00:00 2023-11-27T15:12:01.297 INFO:teuthology.orchestra.run.smithi029.stdout:(4/131): ceph-immutable-object-cache-18.0.0-390 1.9 MB/s | 165 kB 00:00 2023-11-27T15:12:01.789 INFO:teuthology.orchestra.run.smithi029.stdout:(5/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 3.1 MB/s | 1.5 MB 00:00 2023-11-27T15:12:01.828 INFO:teuthology.orchestra.run.smithi008.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:12:01.829 INFO:teuthology.orchestra.run.smithi008.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:12:01.845 INFO:teuthology.orchestra.run.smithi008.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-27T15:12:01.846 INFO:teuthology.orchestra.run.smithi008.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-27T15:12:01.847 INFO:teuthology.orchestra.run.smithi008.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-27T15:12:01.849 INFO:teuthology.orchestra.run.smithi008.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-27T15:12:01.851 INFO:teuthology.orchestra.run.smithi008.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-27T15:12:01.855 INFO:teuthology.orchestra.run.smithi008.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-27T15:12:01.856 INFO:teuthology.orchestra.run.smithi008.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-27T15:12:02.007 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2023-11-27T15:12:02.021 INFO:teuthology.orchestra.run.smithi008.stdout:=========================================================================================================================== 2023-11-27T15:12:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2023-11-27T15:12:02.021 INFO:teuthology.orchestra.run.smithi008.stdout:=========================================================================================================================== 2023-11-27T15:12:02.021 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2023-11-27T15:12:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-27T15:12:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-27T15:12:02.021 INFO:teuthology.orchestra.run.smithi008.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-27T15:12:02.022 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-27T15:12:02.023 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout:Upgrading: 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-27T15:12:02.024 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-27T15:12:02.025 INFO:teuthology.orchestra.run.smithi008.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-27T15:12:02.026 INFO:teuthology.orchestra.run.smithi008.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-27T15:12:02.027 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-27T15:12:02.028 INFO:teuthology.orchestra.run.smithi008.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-27T15:12:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-27T15:12:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-27T15:12:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-27T15:12:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-27T15:12:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-27T15:12:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-27T15:12:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-27T15:12:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-27T15:12:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-27T15:12:02.029 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-27T15:12:02.030 INFO:teuthology.orchestra.run.smithi008.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling module streams: 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql 10 2023-11-27T15:12:02.031 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:12:02.032 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2023-11-27T15:12:02.032 INFO:teuthology.orchestra.run.smithi008.stdout:=========================================================================================================================== 2023-11-27T15:12:02.032 INFO:teuthology.orchestra.run.smithi008.stdout:Install 128 Packages 2023-11-27T15:12:02.032 INFO:teuthology.orchestra.run.smithi008.stdout:Upgrade 3 Packages 2023-11-27T15:12:02.032 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:12:02.032 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 197 M 2023-11-27T15:12:02.032 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2023-11-27T15:12:02.827 INFO:teuthology.orchestra.run.smithi008.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2023-11-27T15:12:02.990 INFO:teuthology.orchestra.run.smithi029.stdout:(6/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 3.8 MB/s | 4.6 MB 00:01 2023-11-27T15:12:03.390 INFO:teuthology.orchestra.run.smithi029.stdout:(7/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 1.1 MB/s | 2.3 MB 00:02 2023-11-27T15:12:03.686 INFO:teuthology.orchestra.run.smithi060.stdout:(8/131): ceph-osd-18.0.0-3906.g1c956514.el8.x86 6.4 MB/s | 18 MB 00:02 2023-11-27T15:12:03.819 INFO:teuthology.orchestra.run.smithi008.stdout:(2/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 855 kB/s | 848 kB 00:00 2023-11-27T15:12:03.842 INFO:teuthology.orchestra.run.smithi060.stdout:(9/131): ceph-radosgw-18.0.0-3906.g1c956514.el8 5.0 MB/s | 13 MB 00:02 2023-11-27T15:12:03.867 INFO:teuthology.orchestra.run.smithi060.stdout:(10/131): ceph-selinux-18.0.0-3906.g1c956514.el 134 kB/s | 24 kB 00:00 2023-11-27T15:12:03.978 INFO:teuthology.orchestra.run.smithi008.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 1.0 MB/s | 165 kB 00:00 2023-11-27T15:12:04.043 INFO:teuthology.orchestra.run.smithi060.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 1.4 MB/s | 258 kB 00:00 2023-11-27T15:12:04.143 INFO:teuthology.orchestra.run.smithi060.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 310 kB/s | 31 kB 00:00 2023-11-27T15:12:04.427 INFO:teuthology.orchestra.run.smithi060.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 2.7 MB/s | 771 kB 00:00 2023-11-27T15:12:04.535 INFO:teuthology.orchestra.run.smithi060.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 1.2 MB/s | 137 kB 00:00 2023-11-27T15:12:04.686 INFO:teuthology.orchestra.run.smithi008.stdout:(4/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 2.3 MB/s | 5.3 MB 00:02 2023-11-27T15:12:04.794 INFO:teuthology.orchestra.run.smithi060.stdout:(15/131): ceph-common-18.0.0-3906.g1c956514.el8 4.3 MB/s | 22 MB 00:05 2023-11-27T15:12:04.819 INFO:teuthology.orchestra.run.smithi060.stdout:(16/131): librados-devel-18.0.0-3906.g1c956514. 490 kB/s | 139 kB 00:00 2023-11-27T15:12:05.103 INFO:teuthology.orchestra.run.smithi060.stdout:(17/131): libradosstriper1-18.0.0-3906.g1c95651 1.5 MB/s | 473 kB 00:00 2023-11-27T15:12:05.220 INFO:teuthology.orchestra.run.smithi060.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 385 kB/s | 45 kB 00:00 2023-11-27T15:12:05.236 INFO:teuthology.orchestra.run.smithi008.stdout:(5/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 2.8 MB/s | 1.5 MB 00:00 2023-11-27T15:12:05.362 INFO:teuthology.orchestra.run.smithi060.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 836 kB/s | 119 kB 00:00 2023-11-27T15:12:05.479 INFO:teuthology.orchestra.run.smithi060.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 1.7 MB/s | 197 kB 00:00 2023-11-27T15:12:05.638 INFO:teuthology.orchestra.run.smithi060.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 2.3 MB/s | 371 kB 00:00 2023-11-27T15:12:05.821 INFO:teuthology.orchestra.run.smithi060.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 1.9 MB/s | 364 kB 00:00 2023-11-27T15:12:05.954 INFO:teuthology.orchestra.run.smithi008.stdout:(6/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-11-27T15:12:05.955 INFO:teuthology.orchestra.run.smithi060.stdout:(23/131): python3-rgw-18.0.0-3906.g1c956514.el8 799 kB/s | 107 kB 00:00 2023-11-27T15:12:05.983 INFO:teuthology.orchestra.run.smithi029.stdout:(8/131): ceph-common-18.0.0-3906.g1c956514.el8. 3.8 MB/s | 22 MB 00:05 2023-11-27T15:12:06.064 INFO:teuthology.orchestra.run.smithi060.stdout:(24/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 872 kB/s | 94 kB 00:00 2023-11-27T15:12:06.091 INFO:teuthology.orchestra.run.smithi029.stdout:(9/131): ceph-selinux-18.0.0-3906.g1c956514.el8 225 kB/s | 24 kB 00:00 2023-11-27T15:12:06.390 INFO:teuthology.orchestra.run.smithi060.stdout:(25/131): librgw2-18.0.0-3906.g1c956514.el8.x86 4.2 MB/s | 6.5 MB 00:01 2023-11-27T15:12:06.498 INFO:teuthology.orchestra.run.smithi060.stdout:(26/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 1.6 MB/s | 180 kB 00:00 2023-11-27T15:12:06.598 INFO:teuthology.orchestra.run.smithi060.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3906.g 231 kB/s | 23 kB 00:00 2023-11-27T15:12:06.690 INFO:teuthology.orchestra.run.smithi060.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 1.4 MB/s | 127 kB 00:00 2023-11-27T15:12:06.862 INFO:teuthology.orchestra.run.smithi008.stdout:(7/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 2.8 MB/s | 4.6 MB 00:01 2023-11-27T15:12:06.982 INFO:teuthology.orchestra.run.smithi060.stdout:(29/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 3.8 MB/s | 3.4 MB 00:00 2023-11-27T15:12:07.041 INFO:teuthology.orchestra.run.smithi060.stdout:(30/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 4.9 MB/s | 1.7 MB 00:00 2023-11-27T15:12:07.092 INFO:teuthology.orchestra.run.smithi029.stdout:(10/131): ceph-osd-18.0.0-3906.g1c956514.el8.x8 4.3 MB/s | 18 MB 00:04 2023-11-27T15:12:07.149 INFO:teuthology.orchestra.run.smithi060.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3906.g1c 2.2 MB/s | 240 kB 00:00 2023-11-27T15:12:07.192 INFO:teuthology.orchestra.run.smithi029.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 2.5 MB/s | 258 kB 00:00 2023-11-27T15:12:07.241 INFO:teuthology.orchestra.run.smithi060.stdout:(32/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 517 kB/s | 47 kB 00:00 2023-11-27T15:12:07.269 INFO:teuthology.orchestra.run.smithi029.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 405 kB/s | 31 kB 00:00 2023-11-27T15:12:07.325 INFO:teuthology.orchestra.run.smithi060.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3906.g1 174 kB/s | 15 kB 00:00 2023-11-27T15:12:07.461 INFO:teuthology.orchestra.run.smithi029.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 3.9 MB/s | 771 kB 00:00 2023-11-27T15:12:07.470 INFO:teuthology.orchestra.run.smithi060.stdout:(34/131): ceph-volume-18.0.0-3906.g1c956514.el8 1.8 MB/s | 265 kB 00:00 2023-11-27T15:12:07.520 INFO:teuthology.orchestra.run.smithi029.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 2.3 MB/s | 137 kB 00:00 2023-11-27T15:12:07.551 INFO:teuthology.orchestra.run.smithi060.stdout:(35/131): cephadm-18.0.0-3906.g1c956514.el8.noa 1.0 MB/s | 83 kB 00:00 2023-11-27T15:12:07.587 INFO:teuthology.orchestra.run.smithi029.stdout:(15/131): librados-devel-18.0.0-3906.g1c956514. 2.0 MB/s | 139 kB 00:00 2023-11-27T15:12:07.634 INFO:teuthology.orchestra.run.smithi060.stdout:(36/131): cephfs-top-18.0.0-3906.g1c956514.el8. 216 kB/s | 18 kB 00:00 2023-11-27T15:12:07.704 INFO:teuthology.orchestra.run.smithi029.stdout:(16/131): libradosstriper1-18.0.0-3906.g1c95651 4.0 MB/s | 473 kB 00:00 2023-11-27T15:12:07.985 INFO:teuthology.orchestra.run.smithi060.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 875 kB/s | 306 kB 00:00 2023-11-27T15:12:08.044 INFO:teuthology.orchestra.run.smithi060.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-11-27T15:12:08.102 INFO:teuthology.orchestra.run.smithi060.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-11-27T15:12:08.394 INFO:teuthology.orchestra.run.smithi060.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.6 MB/s | 476 kB 00:00 2023-11-27T15:12:08.445 INFO:teuthology.orchestra.run.smithi060.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 873 kB/s | 44 kB 00:00 2023-11-27T15:12:08.562 INFO:teuthology.orchestra.run.smithi060.stdout:(42/131): ceph-mgr-diskprediction-local-18.0.0- 4.7 MB/s | 7.4 MB 00:01 2023-11-27T15:12:08.620 INFO:teuthology.orchestra.run.smithi060.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2023-11-27T15:12:08.670 INFO:teuthology.orchestra.run.smithi060.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 362 kB/s | 16 kB 00:00 2023-11-27T15:12:08.822 INFO:teuthology.orchestra.run.smithi060.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-11-27T15:12:08.838 INFO:teuthology.orchestra.run.smithi029.stdout:(17/131): librgw2-18.0.0-3906.g1c956514.el8.x86 5.8 MB/s | 6.5 MB 00:01 2023-11-27T15:12:08.846 INFO:teuthology.orchestra.run.smithi060.stdout:(46/131): python3-cachetools-3.1.1-4.el8.noarch 116 kB/s | 33 kB 00:00 2023-11-27T15:12:08.905 INFO:teuthology.orchestra.run.smithi029.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 671 kB/s | 45 kB 00:00 2023-11-27T15:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 810 kB/s | 81 kB 00:00 2023-11-27T15:12:08.972 INFO:teuthology.orchestra.run.smithi029.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 1.7 MB/s | 119 kB 00:00 2023-11-27T15:12:09.005 INFO:teuthology.orchestra.run.smithi060.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 191 kB/s | 11 kB 00:00 2023-11-27T15:12:09.023 INFO:teuthology.orchestra.run.smithi029.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 3.9 MB/s | 197 kB 00:00 2023-11-27T15:12:09.064 INFO:teuthology.orchestra.run.smithi060.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 301 kB/s | 18 kB 00:00 2023-11-27T15:12:09.089 INFO:teuthology.orchestra.run.smithi060.stdout:(50/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.4 MB/s | 384 kB 00:00 2023-11-27T15:12:09.098 INFO:teuthology.orchestra.run.smithi029.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 4.8 MB/s | 371 kB 00:00 2023-11-27T15:12:09.174 INFO:teuthology.orchestra.run.smithi029.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 4.7 MB/s | 364 kB 00:00 2023-11-27T15:12:09.179 INFO:teuthology.orchestra.run.smithi008.stdout:(8/131): ceph-common-18.0.0-3906.g1c956514.el8. 3.2 MB/s | 22 MB 00:06 2023-11-27T15:12:09.189 INFO:teuthology.orchestra.run.smithi060.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 487 kB/s | 49 kB 00:00 2023-11-27T15:12:09.224 INFO:teuthology.orchestra.run.smithi029.stdout:(23/131): python3-rgw-18.0.0-3906.g1c956514.el8 2.1 MB/s | 107 kB 00:00 2023-11-27T15:12:09.246 INFO:teuthology.orchestra.run.smithi008.stdout:(9/131): ceph-selinux-18.0.0-3906.g1c956514.el8 366 kB/s | 24 kB 00:00 2023-11-27T15:12:09.248 INFO:teuthology.orchestra.run.smithi060.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-11-27T15:12:09.275 INFO:teuthology.orchestra.run.smithi029.stdout:(24/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 1.8 MB/s | 94 kB 00:00 2023-11-27T15:12:09.307 INFO:teuthology.orchestra.run.smithi060.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-11-27T15:12:09.524 INFO:teuthology.orchestra.run.smithi060.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-11-27T15:12:09.583 INFO:teuthology.orchestra.run.smithi060.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-11-27T15:12:09.641 INFO:teuthology.orchestra.run.smithi060.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 578 kB/s | 34 kB 00:00 2023-11-27T15:12:09.675 INFO:teuthology.orchestra.run.smithi060.stdout:(57/131): python3-kubernetes-11.0.0-6.el8.noarc 1.9 MB/s | 1.2 MB 00:00 2023-11-27T15:12:09.733 INFO:teuthology.orchestra.run.smithi060.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-11-27T15:12:09.759 INFO:teuthology.orchestra.run.smithi060.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2023-11-27T15:12:09.792 INFO:teuthology.orchestra.run.smithi060.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 320 kB/s | 19 kB 00:00 2023-11-27T15:12:09.834 INFO:teuthology.orchestra.run.smithi029.stdout:(25/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 6.2 MB/s | 3.4 MB 00:00 2023-11-27T15:12:09.852 INFO:teuthology.orchestra.run.smithi060.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 257 kB/s | 24 kB 00:00 2023-11-27T15:12:09.885 INFO:teuthology.orchestra.run.smithi060.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-11-27T15:12:09.893 INFO:teuthology.orchestra.run.smithi029.stdout:(26/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 3.0 MB/s | 180 kB 00:00 2023-11-27T15:12:09.910 INFO:teuthology.orchestra.run.smithi060.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 478 kB/s | 27 kB 00:00 2023-11-27T15:12:09.938 INFO:teuthology.orchestra.run.smithi008.stdout:(10/131): ceph-radosgw-18.0.0-3906.g1c956514.el 4.2 MB/s | 13 MB 00:03 2023-11-27T15:12:09.943 INFO:teuthology.orchestra.run.smithi029.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3906.g 457 kB/s | 23 kB 00:00 2023-11-27T15:12:10.012 INFO:teuthology.orchestra.run.smithi029.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 1.8 MB/s | 127 kB 00:00 2023-11-27T15:12:10.030 INFO:teuthology.orchestra.run.smithi008.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 2.8 MB/s | 258 kB 00:00 2023-11-27T15:12:10.037 INFO:teuthology.orchestra.run.smithi060.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.7 MB/s | 255 kB 00:00 2023-11-27T15:12:10.052 INFO:teuthology.orchestra.run.smithi060.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.7 MB/s | 251 kB 00:00 2023-11-27T15:12:10.094 INFO:teuthology.orchestra.run.smithi060.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-11-27T15:12:10.097 INFO:teuthology.orchestra.run.smithi008.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 466 kB/s | 31 kB 00:00 2023-11-27T15:12:10.153 INFO:teuthology.orchestra.run.smithi060.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 851 kB/s | 85 kB 00:00 2023-11-27T15:12:10.211 INFO:teuthology.orchestra.run.smithi060.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-11-27T15:12:10.236 INFO:teuthology.orchestra.run.smithi060.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-11-27T15:12:10.296 INFO:teuthology.orchestra.run.smithi029.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 6.0 MB/s | 1.7 MB 00:00 2023-11-27T15:12:10.314 INFO:teuthology.orchestra.run.smithi008.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 3.5 MB/s | 771 kB 00:00 2023-11-27T15:12:10.380 INFO:teuthology.orchestra.run.smithi008.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 2.0 MB/s | 137 kB 00:00 2023-11-27T15:12:10.448 INFO:teuthology.orchestra.run.smithi008.stdout:(15/131): librados-devel-18.0.0-3906.g1c956514. 2.0 MB/s | 139 kB 00:00 2023-11-27T15:12:10.615 INFO:teuthology.orchestra.run.smithi008.stdout:(16/131): libradosstriper1-18.0.0-3906.g1c95651 2.8 MB/s | 473 kB 00:00 2023-11-27T15:12:10.754 INFO:teuthology.orchestra.run.smithi060.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-11-27T15:12:10.871 INFO:teuthology.orchestra.run.smithi060.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 946 kB/s | 110 kB 00:00 2023-11-27T15:12:11.038 INFO:teuthology.orchestra.run.smithi060.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 325 kB/s | 54 kB 00:00 2023-11-27T15:12:11.400 INFO:teuthology.orchestra.run.smithi029.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 6.7 MB/s | 7.4 MB 00:01 2023-11-27T15:12:11.464 INFO:teuthology.orchestra.run.smithi029.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3906.g1c 3.7 MB/s | 240 kB 00:00 2023-11-27T15:12:11.513 INFO:teuthology.orchestra.run.smithi060.stdout:(73/131): ceph-test-18.0.0-3906.g1c956514.el8.x 5.3 MB/s | 41 MB 00:07 2023-11-27T15:12:11.555 INFO:teuthology.orchestra.run.smithi060.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 1.2 MB/s | 1.7 MB 00:01 2023-11-27T15:12:11.597 INFO:teuthology.orchestra.run.smithi060.stdout:(75/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 463 kB/s | 259 kB 00:00 2023-11-27T15:12:11.681 INFO:teuthology.orchestra.run.smithi029.stdout:(32/131): ceph-radosgw-18.0.0-3906.g1c956514.el 1.6 MB/s | 13 MB 00:08 2023-11-27T15:12:11.698 INFO:teuthology.orchestra.run.smithi008.stdout:(17/131): librgw2-18.0.0-3906.g1c956514.el8.x86 6.0 MB/s | 6.5 MB 00:01 2023-11-27T15:12:11.706 INFO:teuthology.orchestra.run.smithi029.stdout:(33/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 195 kB/s | 47 kB 00:00 2023-11-27T15:12:11.714 INFO:teuthology.orchestra.run.smithi060.stdout:(76/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2023-11-27T15:12:11.740 INFO:teuthology.orchestra.run.smithi008.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 1.1 MB/s | 45 kB 00:00 2023-11-27T15:12:11.740 INFO:teuthology.orchestra.run.smithi029.stdout:(34/131): ceph-prometheus-alerts-18.0.0-3906.g1 247 kB/s | 15 kB 00:00 2023-11-27T15:12:11.756 INFO:teuthology.orchestra.run.smithi060.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 4.5 MB/s | 688 kB 00:00 2023-11-27T15:12:11.773 INFO:teuthology.orchestra.run.smithi060.stdout:(78/131): python3-markupsafe-0.23-19.el8.x86_64 186 kB/s | 39 kB 00:00 2023-11-27T15:12:11.774 INFO:teuthology.orchestra.run.smithi029.stdout:(35/131): ceph-volume-18.0.0-3906.g1c956514.el8 3.8 MB/s | 265 kB 00:00 2023-11-27T15:12:11.816 INFO:teuthology.orchestra.run.smithi029.stdout:(36/131): cephfs-top-18.0.0-3906.g1c956514.el8. 428 kB/s | 18 kB 00:00 2023-11-27T15:12:11.818 INFO:teuthology.orchestra.run.smithi008.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 1.5 MB/s | 119 kB 00:00 2023-11-27T15:12:11.842 INFO:teuthology.orchestra.run.smithi029.stdout:(37/131): cephadm-18.0.0-3906.g1c956514.el8.noa 824 kB/s | 83 kB 00:00 2023-11-27T15:12:11.865 INFO:teuthology.orchestra.run.smithi060.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.5 MB/s | 538 kB 00:00 2023-11-27T15:12:11.874 INFO:teuthology.orchestra.run.smithi008.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 3.5 MB/s | 197 kB 00:00 2023-11-27T15:12:11.907 INFO:teuthology.orchestra.run.smithi060.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2023-11-27T15:12:11.932 INFO:teuthology.orchestra.run.smithi060.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-11-27T15:12:11.950 INFO:teuthology.orchestra.run.smithi008.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 4.8 MB/s | 371 kB 00:00 2023-11-27T15:12:12.025 INFO:teuthology.orchestra.run.smithi008.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 4.7 MB/s | 364 kB 00:00 2023-11-27T15:12:12.041 INFO:teuthology.orchestra.run.smithi060.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.0 MB/s | 713 kB 00:00 2023-11-27T15:12:12.075 INFO:teuthology.orchestra.run.smithi008.stdout:(23/131): python3-rgw-18.0.0-3906.g1c956514.el8 2.1 MB/s | 107 kB 00:00 2023-11-27T15:12:12.091 INFO:teuthology.orchestra.run.smithi060.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 167 kB/s | 27 kB 00:00 2023-11-27T15:12:12.126 INFO:teuthology.orchestra.run.smithi008.stdout:(24/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 1.8 MB/s | 94 kB 00:00 2023-11-27T15:12:12.126 INFO:teuthology.orchestra.run.smithi029.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 207 kB/s | 59 kB 00:00 2023-11-27T15:12:12.158 INFO:teuthology.orchestra.run.smithi060.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.7 MB/s | 320 kB 00:00 2023-11-27T15:12:12.250 INFO:teuthology.orchestra.run.smithi060.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-11-27T15:12:12.252 INFO:teuthology.orchestra.run.smithi029.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 599 kB/s | 75 kB 00:00 2023-11-27T15:12:12.275 INFO:teuthology.orchestra.run.smithi060.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 239 kB/s | 44 kB 00:00 2023-11-27T15:12:12.277 INFO:teuthology.orchestra.run.smithi029.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 665 kB/s | 306 kB 00:00 2023-11-27T15:12:12.370 INFO:teuthology.orchestra.run.smithi029.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 474 kB/s | 44 kB 00:00 2023-11-27T15:12:12.409 INFO:teuthology.orchestra.run.smithi060.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 15 MB/s | 3.7 MB 00:00 2023-11-27T15:12:12.434 INFO:teuthology.orchestra.run.smithi060.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.4 MB/s | 225 kB 00:00 2023-11-27T15:12:12.460 INFO:teuthology.orchestra.run.smithi060.stdout:(89/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.1 MB/s | 457 kB 00:00 2023-11-27T15:12:12.472 INFO:teuthology.orchestra.run.smithi029.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2023-11-27T15:12:12.505 INFO:teuthology.orchestra.run.smithi029.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-11-27T15:12:12.548 INFO:teuthology.orchestra.run.smithi029.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 437 kB/s | 33 kB 00:00 2023-11-27T15:12:12.560 INFO:teuthology.orchestra.run.smithi060.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 426 kB/s | 64 kB 00:00 2023-11-27T15:12:12.567 INFO:teuthology.orchestra.run.smithi008.stdout:(25/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 7.8 MB/s | 3.4 MB 00:00 2023-11-27T15:12:12.585 INFO:teuthology.orchestra.run.smithi060.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 840 kB/s | 126 kB 00:00 2023-11-27T15:12:12.590 INFO:teuthology.orchestra.run.smithi029.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 194 kB/s | 16 kB 00:00 2023-11-27T15:12:12.611 INFO:teuthology.orchestra.run.smithi060.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 133 kB/s | 20 kB 00:00 2023-11-27T15:12:12.618 INFO:teuthology.orchestra.run.smithi008.stdout:(26/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 3.5 MB/s | 180 kB 00:00 2023-11-27T15:12:12.624 INFO:teuthology.orchestra.run.smithi029.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-11-27T15:12:12.694 INFO:teuthology.orchestra.run.smithi008.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3906.g 303 kB/s | 23 kB 00:00 2023-11-27T15:12:12.708 INFO:teuthology.orchestra.run.smithi029.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 971 kB/s | 81 kB 00:00 2023-11-27T15:12:12.742 INFO:teuthology.orchestra.run.smithi029.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.5 MB/s | 384 kB 00:00 2023-11-27T15:12:12.744 INFO:teuthology.orchestra.run.smithi060.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 563 kB/s | 103 kB 00:00 2023-11-27T15:12:12.751 INFO:teuthology.orchestra.run.smithi008.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 2.2 MB/s | 127 kB 00:00 2023-11-27T15:12:12.786 INFO:teuthology.orchestra.run.smithi060.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 3.5 MB/s | 709 kB 00:00 2023-11-27T15:12:12.792 INFO:teuthology.orchestra.run.smithi029.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 132 kB/s | 11 kB 00:00 2023-11-27T15:12:12.818 INFO:teuthology.orchestra.run.smithi029.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 232 kB/s | 18 kB 00:00 2023-11-27T15:12:12.901 INFO:teuthology.orchestra.run.smithi029.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 583 kB/s | 49 kB 00:00 2023-11-27T15:12:12.952 INFO:teuthology.orchestra.run.smithi029.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 7.3 MB/s | 1.2 MB 00:00 2023-11-27T15:12:12.953 INFO:teuthology.orchestra.run.smithi060.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 14 MB/s | 4.9 MB 00:00 2023-11-27T15:12:12.978 INFO:teuthology.orchestra.run.smithi060.stdout:(96/131): libpq-13.5-1.el8.x86_64.rpm 846 kB/s | 198 kB 00:00 2023-11-27T15:12:12.986 INFO:teuthology.orchestra.run.smithi029.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 697 kB/s | 59 kB 00:00 2023-11-27T15:12:13.028 INFO:teuthology.orchestra.run.smithi029.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 794 kB/s | 60 kB 00:00 2023-11-27T15:12:13.112 INFO:teuthology.orchestra.run.smithi060.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-11-27T15:12:13.121 INFO:teuthology.orchestra.run.smithi029.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-11-27T15:12:13.137 INFO:teuthology.orchestra.run.smithi060.stdout:(98/131): jq-1.6-3.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2023-11-27T15:12:13.139 INFO:teuthology.orchestra.run.smithi029.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 141 kB/s | 16 kB 00:00 2023-11-27T15:12:13.152 INFO:teuthology.orchestra.run.smithi008.stdout:(29/131): ceph-osd-18.0.0-3906.g1c956514.el8.x8 2.5 MB/s | 18 MB 00:07 2023-11-27T15:12:13.171 INFO:teuthology.orchestra.run.smithi060.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.8 MB/s | 354 kB 00:00 2023-11-27T15:12:13.194 INFO:teuthology.orchestra.run.smithi008.stdout:(30/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 3.9 MB/s | 1.7 MB 00:00 2023-11-27T15:12:13.206 INFO:teuthology.orchestra.run.smithi029.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 393 kB/s | 34 kB 00:00 2023-11-27T15:12:13.240 INFO:teuthology.orchestra.run.smithi029.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-11-27T15:12:13.246 INFO:teuthology.orchestra.run.smithi060.stdout:(100/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-11-27T15:12:13.270 INFO:teuthology.orchestra.run.smithi008.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3906.g1c 3.1 MB/s | 240 kB 00:00 2023-11-27T15:12:13.291 INFO:teuthology.orchestra.run.smithi029.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 763 kB/s | 64 kB 00:00 2023-11-27T15:12:13.313 INFO:teuthology.orchestra.run.smithi060.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-11-27T15:12:13.320 INFO:teuthology.orchestra.run.smithi008.stdout:(32/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 946 kB/s | 47 kB 00:00 2023-11-27T15:12:13.325 INFO:teuthology.orchestra.run.smithi029.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 222 kB/s | 19 kB 00:00 2023-11-27T15:12:13.347 INFO:teuthology.orchestra.run.smithi060.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 465 kB/s | 47 kB 00:00 2023-11-27T15:12:13.367 INFO:teuthology.orchestra.run.smithi029.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 316 kB/s | 24 kB 00:00 2023-11-27T15:12:13.370 INFO:teuthology.orchestra.run.smithi008.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3906.g1 292 kB/s | 15 kB 00:00 2023-11-27T15:12:13.409 INFO:teuthology.orchestra.run.smithi029.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 347 kB/s | 29 kB 00:00 2023-11-27T15:12:13.431 INFO:teuthology.orchestra.run.smithi060.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.7 MB/s | 200 kB 00:00 2023-11-27T15:12:13.443 INFO:teuthology.orchestra.run.smithi029.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 362 kB/s | 27 kB 00:00 2023-11-27T15:12:13.446 INFO:teuthology.orchestra.run.smithi008.stdout:(34/131): ceph-volume-18.0.0-3906.g1c956514.el8 3.4 MB/s | 265 kB 00:00 2023-11-27T15:12:13.456 INFO:teuthology.orchestra.run.smithi060.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 393 kB/s | 43 kB 00:00 2023-11-27T15:12:13.502 INFO:teuthology.orchestra.run.smithi029.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-11-27T15:12:13.505 INFO:teuthology.orchestra.run.smithi008.stdout:(35/131): cephadm-18.0.0-3906.g1c956514.el8.noa 1.4 MB/s | 83 kB 00:00 2023-11-27T15:12:13.531 INFO:teuthology.orchestra.run.smithi060.stdout:(105/131): postgresql-server-10.21-2.module+el8 12 MB/s | 5.1 MB 00:00 2023-11-27T15:12:13.544 INFO:teuthology.orchestra.run.smithi029.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-11-27T15:12:13.581 INFO:teuthology.orchestra.run.smithi060.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 384 kB/s | 48 kB 00:00 2023-11-27T15:12:13.586 INFO:teuthology.orchestra.run.smithi029.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 729 kB/s | 61 kB 00:00 2023-11-27T15:12:13.588 INFO:teuthology.orchestra.run.smithi008.stdout:(36/131): cephfs-top-18.0.0-3906.g1c956514.el8. 216 kB/s | 18 kB 00:00 2023-11-27T15:12:13.607 INFO:teuthology.orchestra.run.smithi060.stdout:(107/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2023-11-27T15:12:13.620 INFO:teuthology.orchestra.run.smithi029.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-11-27T15:12:13.663 INFO:teuthology.orchestra.run.smithi029.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 302 kB/s | 23 kB 00:00 2023-11-27T15:12:13.665 INFO:teuthology.orchestra.run.smithi060.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 650 kB/s | 87 kB 00:00 2023-11-27T15:12:13.696 INFO:teuthology.orchestra.run.smithi029.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.0 MB/s | 70 kB 00:00 2023-11-27T15:12:13.766 INFO:teuthology.orchestra.run.smithi060.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 687 kB/s | 109 kB 00:00 2023-11-27T15:12:13.791 INFO:teuthology.orchestra.run.smithi060.stdout:(110/131): python3-requests-oauthlib-1.0.0-1.el 204 kB/s | 43 kB 00:00 2023-11-27T15:12:13.847 INFO:teuthology.orchestra.run.smithi029.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 7.7 MB/s | 1.7 MB 00:00 2023-11-27T15:12:13.891 INFO:teuthology.orchestra.run.smithi060.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.0 MB/s | 229 kB 00:00 2023-11-27T15:12:13.925 INFO:teuthology.orchestra.run.smithi060.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.4 MB/s | 560 kB 00:00 2023-11-27T15:12:13.932 INFO:teuthology.orchestra.run.smithi029.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 199 kB/s | 47 kB 00:00 2023-11-27T15:12:13.983 INFO:teuthology.orchestra.run.smithi029.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.4 MB/s | 200 kB 00:00 2023-11-27T15:12:14.017 INFO:teuthology.orchestra.run.smithi060.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 83 kB/s | 19 kB 00:00 2023-11-27T15:12:14.078 INFO:teuthology.orchestra.run.smithi060.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 831 kB/s | 155 kB 00:00 2023-11-27T15:12:14.101 INFO:teuthology.orchestra.run.smithi029.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 254 kB/s | 43 kB 00:00 2023-11-27T15:12:14.117 INFO:teuthology.orchestra.run.smithi060.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 2.8 MB/s | 559 kB 00:00 2023-11-27T15:12:14.134 INFO:teuthology.orchestra.run.smithi029.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2023-11-27T15:12:14.209 INFO:teuthology.orchestra.run.smithi060.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 93 kB/s | 18 kB 00:00 2023-11-27T15:12:14.243 INFO:teuthology.orchestra.run.smithi060.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.6 MB/s | 605 kB 00:00 2023-11-27T15:12:14.268 INFO:teuthology.orchestra.run.smithi029.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 287 kB/s | 48 kB 00:00 2023-11-27T15:12:14.269 INFO:teuthology.orchestra.run.smithi060.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2023-11-27T15:12:14.302 INFO:teuthology.orchestra.run.smithi029.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 519 kB/s | 87 kB 00:00 2023-11-27T15:12:14.360 INFO:teuthology.orchestra.run.smithi060.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 618 kB/s | 72 kB 00:00 2023-11-27T15:12:14.411 INFO:teuthology.orchestra.run.smithi029.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 300 kB/s | 43 kB 00:00 2023-11-27T15:12:14.416 INFO:teuthology.orchestra.run.smithi060.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 561 kB/s | 83 kB 00:00 2023-11-27T15:12:14.454 INFO:teuthology.orchestra.run.smithi029.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 724 kB/s | 109 kB 00:00 2023-11-27T15:12:14.486 INFO:teuthology.orchestra.run.smithi060.stdout:(121/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 8.4 MB/s | 2.3 MB 00:00 2023-11-27T15:12:14.511 INFO:teuthology.orchestra.run.smithi060.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 397 kB/s | 60 kB 00:00 2023-11-27T15:12:14.537 INFO:teuthology.orchestra.run.smithi060.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 816 kB/s | 98 kB 00:00 2023-11-27T15:12:14.580 INFO:teuthology.orchestra.run.smithi008.stdout:(37/131): ceph-test-18.0.0-3906.g1c956514.el8.x 7.6 MB/s | 41 MB 00:05 2023-11-27T15:12:14.606 INFO:teuthology.orchestra.run.smithi008.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 301 kB/s | 306 kB 00:01 2023-11-27T15:12:14.637 INFO:teuthology.orchestra.run.smithi029.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.0 MB/s | 229 kB 00:00 2023-11-27T15:12:14.680 INFO:teuthology.orchestra.run.smithi029.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.4 MB/s | 560 kB 00:00 2023-11-27T15:12:14.687 INFO:teuthology.orchestra.run.smithi060.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 287 kB/s | 58 kB 00:00 2023-11-27T15:12:14.712 INFO:teuthology.orchestra.run.smithi060.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 192 kB/s | 39 kB 00:00 2023-11-27T15:12:14.722 INFO:teuthology.orchestra.run.smithi008.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 644 kB/s | 75 kB 00:00 2023-11-27T15:12:14.779 INFO:teuthology.orchestra.run.smithi060.stdout:(126/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 2.6 MB/s | 644 kB 00:00 2023-11-27T15:12:14.780 INFO:teuthology.orchestra.run.smithi029.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 131 kB/s | 19 kB 00:00 2023-11-27T15:12:14.856 INFO:teuthology.orchestra.run.smithi008.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 4.3 MB/s | 7.4 MB 00:01 2023-11-27T15:12:14.940 INFO:teuthology.orchestra.run.smithi008.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2023-11-27T15:12:14.964 INFO:teuthology.orchestra.run.smithi029.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 545 kB/s | 155 kB 00:00 2023-11-27T15:12:15.023 INFO:teuthology.orchestra.run.smithi008.stdout:(42/131): liboath-2.6.2-3.el8.x86_64.rpm 133 kB/s | 59 kB 00:00 2023-11-27T15:12:15.024 INFO:teuthology.orchestra.run.smithi060.stdout:(127/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 510 kB/s | 171 kB 00:00 2023-11-27T15:12:15.057 INFO:teuthology.orchestra.run.smithi008.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.6 MB/s | 185 kB 00:00 2023-11-27T15:12:15.073 INFO:teuthology.orchestra.run.smithi029.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 165 kB/s | 18 kB 00:00 2023-11-27T15:12:15.101 INFO:teuthology.orchestra.run.smithi060.stdout:(128/131): NetworkManager-initscripts-updown-1. 355 kB/s | 138 kB 00:00 2023-11-27T15:12:15.149 INFO:teuthology.orchestra.run.smithi029.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 1.5 MB/s | 559 kB 00:00 2023-11-27T15:12:15.149 INFO:teuthology.orchestra.run.smithi008.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 267 kB/s | 33 kB 00:00 2023-11-27T15:12:15.174 INFO:teuthology.orchestra.run.smithi008.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 140 kB/s | 16 kB 00:00 2023-11-27T15:12:15.217 INFO:teuthology.orchestra.run.smithi029.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.1 MB/s | 605 kB 00:00 2023-11-27T15:12:15.232 INFO:teuthology.orchestra.run.smithi060.stdout:(129/131): librados2-18.0.0-3906.g1c956514.el8. 8.1 MB/s | 3.7 MB 00:00 2023-11-27T15:12:15.250 INFO:teuthology.orchestra.run.smithi008.stdout:(46/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 111 kB/s | 44 kB 00:00 2023-11-27T15:12:15.266 INFO:teuthology.orchestra.run.smithi060.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.7 MB/s | 798 kB 00:00 2023-11-27T15:12:15.300 INFO:teuthology.orchestra.run.smithi008.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2023-11-27T15:12:15.335 INFO:teuthology.orchestra.run.smithi029.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2023-11-27T15:12:15.392 INFO:teuthology.orchestra.run.smithi008.stdout:(48/131): python3-cheroot-8.5.2-1.el8.noarch.rp 712 kB/s | 173 kB 00:00 2023-11-27T15:12:15.442 INFO:teuthology.orchestra.run.smithi008.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 78 kB/s | 11 kB 00:00 2023-11-27T15:12:15.444 INFO:teuthology.orchestra.run.smithi029.stdout:(87/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 10 MB/s | 2.3 MB 00:00 2023-11-27T15:12:15.459 INFO:teuthology.orchestra.run.smithi008.stdout:(50/131): python3-google-auth-1.1.1-10.el8.noar 389 kB/s | 81 kB 00:00 2023-11-27T15:12:15.469 INFO:teuthology.orchestra.run.smithi029.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 538 kB/s | 72 kB 00:00 2023-11-27T15:12:15.484 INFO:teuthology.orchestra.run.smithi008.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2023-11-27T15:12:15.585 INFO:teuthology.orchestra.run.smithi008.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 389 kB/s | 49 kB 00:00 2023-11-27T15:12:15.601 INFO:teuthology.orchestra.run.smithi008.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 503 kB/s | 59 kB 00:00 2023-11-27T15:12:15.658 INFO:teuthology.orchestra.run.smithi060.stdout:(131/131): librbd1-18.0.0-3906.g1c956514.el8.x8 5.6 MB/s | 3.6 MB 00:00 2023-11-27T15:12:15.662 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:12:15.662 INFO:teuthology.orchestra.run.smithi060.stdout:Total 12 MB/s | 197 MB 00:16 2023-11-27T15:12:15.702 INFO:teuthology.orchestra.run.smithi008.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 517 kB/s | 60 kB 00:00 2023-11-27T15:12:15.736 INFO:teuthology.orchestra.run.smithi008.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-11-27T15:12:15.786 INFO:teuthology.orchestra.run.smithi008.stdout:(56/131): python3-kubernetes-11.0.0-6.el8.noarc 3.4 MB/s | 1.2 MB 00:00 2023-11-27T15:12:15.811 INFO:teuthology.orchestra.run.smithi008.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2023-11-27T15:12:15.867 INFO:teuthology.orchestra.run.smithi008.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 257 kB/s | 34 kB 00:00 2023-11-27T15:12:15.903 INFO:teuthology.orchestra.run.smithi008.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.6 MB/s | 196 kB 00:00 2023-11-27T15:12:15.937 INFO:teuthology.orchestra.run.smithi029.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 168 kB/s | 83 kB 00:00 2023-11-27T15:12:15.937 INFO:teuthology.orchestra.run.smithi008.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 511 kB/s | 64 kB 00:00 2023-11-27T15:12:15.970 INFO:teuthology.orchestra.run.smithi029.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 119 kB/s | 60 kB 00:00 2023-11-27T15:12:15.971 INFO:teuthology.orchestra.run.smithi008.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 181 kB/s | 19 kB 00:00 2023-11-27T15:12:16.004 INFO:teuthology.orchestra.run.smithi008.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-11-27T15:12:16.046 INFO:teuthology.orchestra.run.smithi008.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-11-27T15:12:16.072 INFO:teuthology.orchestra.run.smithi008.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 274 kB/s | 27 kB 00:00 2023-11-27T15:12:16.088 INFO:teuthology.orchestra.run.smithi029.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 659 kB/s | 98 kB 00:00 2023-11-27T15:12:16.114 INFO:teuthology.orchestra.run.smithi008.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-11-27T15:12:16.122 INFO:teuthology.orchestra.run.smithi029.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 382 kB/s | 58 kB 00:00 2023-11-27T15:12:16.172 INFO:teuthology.orchestra.run.smithi008.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 611 kB/s | 61 kB 00:00 2023-11-27T15:12:16.223 INFO:teuthology.orchestra.run.smithi008.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 787 kB/s | 85 kB 00:00 2023-11-27T15:12:16.256 INFO:teuthology.orchestra.run.smithi029.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 230 kB/s | 39 kB 00:00 2023-11-27T15:12:16.273 INFO:teuthology.orchestra.run.smithi008.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2023-11-27T15:12:16.323 INFO:teuthology.orchestra.run.smithi008.stdout:(69/131): python3-webob-1.8.5-1.el8.1.noarch.rp 908 kB/s | 251 kB 00:00 2023-11-27T15:12:16.339 INFO:teuthology.orchestra.run.smithi029.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 644 kB 00:00 2023-11-27T15:12:16.349 INFO:teuthology.orchestra.run.smithi008.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 925 kB/s | 70 kB 00:00 2023-11-27T15:12:16.365 INFO:teuthology.orchestra.run.smithi029.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.5 MB/s | 171 kB 00:00 2023-11-27T15:12:16.424 INFO:teuthology.orchestra.run.smithi008.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 8.6 MB/s | 1.7 MB 00:00 2023-11-27T15:12:16.482 INFO:teuthology.orchestra.run.smithi029.stdout:(96/131): NetworkManager-initscripts-updown-1.3 965 kB/s | 138 kB 00:00 2023-11-27T15:12:16.545 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2023-11-27T15:12:16.549 INFO:teuthology.orchestra.run.smithi008.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 434 kB/s | 54 kB 00:00 2023-11-27T15:12:16.574 INFO:teuthology.orchestra.run.smithi008.stdout:(73/131): python3-pyasn1-modules-0.3.7-6.el8.no 510 kB/s | 110 kB 00:00 2023-11-27T15:12:16.575 INFO:teuthology.orchestra.run.smithi029.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.2 MB/s | 110 kB 00:00 2023-11-27T15:12:16.677 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2023-11-27T15:12:16.678 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2023-11-27T15:12:16.683 INFO:teuthology.orchestra.run.smithi008.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.9 MB/s | 323 kB 00:00 2023-11-27T15:12:16.777 INFO:teuthology.orchestra.run.smithi008.stdout:(75/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.1 MB/s | 259 kB 00:00 2023-11-27T15:12:16.800 INFO:teuthology.orchestra.run.smithi008.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 332 kB/s | 39 kB 00:00 2023-11-27T15:12:16.892 INFO:teuthology.orchestra.run.smithi008.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 5.8 MB/s | 688 kB 00:00 2023-11-27T15:12:16.919 INFO:teuthology.orchestra.run.smithi029.stdout:(98/131): python3-scipy-1.0.0-21.module+el8.5.0 25 MB/s | 14 MB 00:00 2023-11-27T15:12:16.942 INFO:teuthology.orchestra.run.smithi008.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-11-27T15:12:16.944 INFO:teuthology.orchestra.run.smithi029.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 147 kB/s | 54 kB 00:00 2023-11-27T15:12:16.984 INFO:teuthology.orchestra.run.smithi008.stdout:(79/131): python3-jinja2-2.10.1-3.el8.noarch.rp 2.9 MB/s | 538 kB 00:00 2023-11-27T15:12:17.036 INFO:teuthology.orchestra.run.smithi029.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2023-11-27T15:12:17.103 INFO:teuthology.orchestra.run.smithi029.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 2.0 MB/s | 323 kB 00:00 2023-11-27T15:12:17.179 INFO:teuthology.orchestra.run.smithi029.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 273 kB/s | 39 kB 00:00 2023-11-27T15:12:17.202 INFO:teuthology.orchestra.run.smithi008.stdout:(80/131): python3-scipy-1.0.0-21.module+el8.5.0 16 MB/s | 14 MB 00:00 2023-11-27T15:12:17.252 INFO:teuthology.orchestra.run.smithi008.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 606 kB/s | 187 kB 00:00 2023-11-27T15:12:17.255 INFO:teuthology.orchestra.run.smithi029.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 4.5 MB/s | 688 kB 00:00 2023-11-27T15:12:17.286 INFO:teuthology.orchestra.run.smithi008.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 2.3 MB/s | 713 kB 00:00 2023-11-27T15:12:17.330 INFO:teuthology.orchestra.run.smithi029.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 3.5 MB/s | 538 kB 00:00 2023-11-27T15:12:17.411 INFO:teuthology.orchestra.run.smithi008.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 2.5 MB/s | 320 kB 00:00 2023-11-27T15:12:17.431 INFO:teuthology.orchestra.run.smithi029.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 893 kB/s | 157 kB 00:00 2023-11-27T15:12:17.503 INFO:teuthology.orchestra.run.smithi008.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2023-11-27T15:12:17.528 INFO:teuthology.orchestra.run.smithi008.stdout:(85/131): python3-jsonpatch-1.21-2.el8.noarch.r 96 kB/s | 27 kB 00:00 2023-11-27T15:12:17.554 INFO:teuthology.orchestra.run.smithi008.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 309 kB/s | 44 kB 00:00 2023-11-27T15:12:17.564 INFO:teuthology.orchestra.run.smithi029.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 800 kB/s | 187 kB 00:00 2023-11-27T15:12:17.615 INFO:teuthology.orchestra.run.smithi029.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 3.8 MB/s | 713 kB 00:00 2023-11-27T15:12:17.695 INFO:teuthology.orchestra.run.smithi008.stdout:(87/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.7 MB/s | 457 kB 00:00 2023-11-27T15:12:17.749 INFO:teuthology.orchestra.run.smithi029.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 199 kB/s | 27 kB 00:00 2023-11-27T15:12:17.754 INFO:teuthology.orchestra.run.smithi008.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 15 MB/s | 3.7 MB 00:00 2023-11-27T15:12:17.779 INFO:teuthology.orchestra.run.smithi008.stdout:(89/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 998 kB/s | 225 kB 00:00 2023-11-27T15:12:17.796 INFO:teuthology.orchestra.run.smithi008.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 636 kB/s | 64 kB 00:00 2023-11-27T15:12:17.816 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2023-11-27T15:12:17.820 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2023-11-27T15:12:17.858 INFO:teuthology.orchestra.run.smithi029.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2023-11-27T15:12:17.880 INFO:teuthology.orchestra.run.smithi008.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 199 kB/s | 20 kB 00:00 2023-11-27T15:12:17.892 INFO:teuthology.orchestra.run.smithi029.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2023-11-27T15:12:17.922 INFO:teuthology.orchestra.run.smithi008.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 754 kB/s | 126 kB 00:00 2023-11-27T15:12:17.947 INFO:teuthology.orchestra.run.smithi008.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 686 kB/s | 103 kB 00:00 2023-11-27T15:12:17.989 INFO:teuthology.orchestra.run.smithi008.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2023-11-27T15:12:18.407 INFO:teuthology.orchestra.run.smithi008.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 10 MB/s | 4.9 MB 00:00 2023-11-27T15:12:18.440 INFO:teuthology.orchestra.run.smithi008.stdout:(96/131): libpq-13.5-1.el8.x86_64.rpm 402 kB/s | 198 kB 00:00 2023-11-27T15:12:18.532 INFO:teuthology.orchestra.run.smithi008.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2023-11-27T15:12:18.607 INFO:teuthology.orchestra.run.smithi008.stdout:(98/131): openblas-0.3.15-3.el8.x86_64.rpm 7.7 MB/s | 4.7 MB 00:00 2023-11-27T15:12:18.609 INFO:teuthology.orchestra.run.smithi029.stdout:(111/131): ceph-test-18.0.0-3906.g1c956514.el8. 3.2 MB/s | 41 MB 00:12 2023-11-27T15:12:18.634 INFO:teuthology.orchestra.run.smithi029.stdout:(112/131): python3-prettytable-0.7.2-14.el8.noa 57 kB/s | 44 kB 00:00 2023-11-27T15:12:18.662 INFO:teuthology.orchestra.run.smithi008.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.6 MB/s | 354 kB 00:00 2023-11-27T15:12:18.725 INFO:teuthology.orchestra.run.smithi008.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-11-27T15:12:18.762 INFO:teuthology.orchestra.run.smithi029.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 4.3 MB/s | 3.7 MB 00:00 2023-11-27T15:12:18.787 INFO:teuthology.orchestra.run.smithi029.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2023-11-27T15:12:18.808 INFO:teuthology.orchestra.run.smithi008.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 10 MB/s | 1.5 MB 00:00 2023-11-27T15:12:18.821 INFO:teuthology.orchestra.run.smithi029.stdout:(115/131): python3-werkzeug-0.12.2-4.el8.noarch 2.1 MB/s | 457 kB 00:00 2023-11-27T15:12:18.833 INFO:teuthology.orchestra.run.smithi008.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 431 kB/s | 47 kB 00:00 2023-11-27T15:12:18.880 INFO:teuthology.orchestra.run.smithi029.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 543 kB/s | 64 kB 00:00 2023-11-27T15:12:18.905 INFO:teuthology.orchestra.run.smithi029.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.0 MB/s | 126 kB 00:00 2023-11-27T15:12:18.934 INFO:teuthology.orchestra.run.smithi008.stdout:(103/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-11-27T15:12:18.959 INFO:teuthology.orchestra.run.smithi008.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.3 MB/s | 200 kB 00:00 2023-11-27T15:12:18.981 INFO:teuthology.orchestra.run.smithi029.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 125 kB/s | 20 kB 00:00 2023-11-27T15:12:18.984 INFO:teuthology.orchestra.run.smithi008.stdout:(105/131): python3-jwt-1.6.1-2.el8.noarch.rpm 283 kB/s | 43 kB 00:00 2023-11-27T15:12:19.040 INFO:teuthology.orchestra.run.smithi029.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 648 kB/s | 103 kB 00:00 2023-11-27T15:12:19.060 INFO:teuthology.orchestra.run.smithi008.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-11-27T15:12:19.099 INFO:teuthology.orchestra.run.smithi029.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 3.6 MB/s | 709 kB 00:00 2023-11-27T15:12:19.108 INFO:teuthology.orchestra.run.smithi008.stdout:(107/131): keyutils-libs-devel-1.5.10-9.el8.x86 324 kB/s | 48 kB 00:00 2023-11-27T15:12:19.136 INFO:teuthology.orchestra.run.smithi008.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 576 kB/s | 87 kB 00:00 2023-11-27T15:12:19.166 INFO:teuthology.orchestra.run.smithi029.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.5 MB/s | 198 kB 00:00 2023-11-27T15:12:19.177 INFO:teuthology.orchestra.run.smithi008.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 363 kB/s | 43 kB 00:00 2023-11-27T15:12:19.220 INFO:teuthology.orchestra.run.smithi008.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 979 kB/s | 109 kB 00:00 2023-11-27T15:12:19.253 INFO:teuthology.orchestra.run.smithi008.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.9 MB/s | 229 kB 00:00 2023-11-27T15:12:19.283 INFO:teuthology.orchestra.run.smithi029.stdout:(122/131): openblas-threads-0.3.15-3.el8.x86_64 16 MB/s | 4.9 MB 00:00 2023-11-27T15:12:19.317 INFO:teuthology.orchestra.run.smithi029.stdout:(123/131): jq-1.6-3.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2023-11-27T15:12:19.345 INFO:teuthology.orchestra.run.smithi008.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 149 kB/s | 19 kB 00:00 2023-11-27T15:12:19.370 INFO:teuthology.orchestra.run.smithi008.stdout:(113/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.8 MB/s | 560 kB 00:00 2023-11-27T15:12:19.396 INFO:teuthology.orchestra.run.smithi008.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-11-27T15:12:19.476 INFO:teuthology.orchestra.run.smithi029.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-11-27T15:12:19.488 INFO:teuthology.orchestra.run.smithi008.stdout:(115/131): python3-cryptography-3.2.1-5.el8.x86 3.8 MB/s | 559 kB 00:00 2023-11-27T15:12:19.509 INFO:teuthology.orchestra.run.smithi029.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.5 MB/s | 354 kB 00:00 2023-11-27T15:12:19.553 INFO:teuthology.orchestra.run.smithi008.stdout:(116/131): libverto-devel-0.3.0-5.el8.x86_64.rp 98 kB/s | 18 kB 00:00 2023-11-27T15:12:19.605 INFO:teuthology.orchestra.run.smithi008.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2023-11-27T15:12:19.610 INFO:teuthology.orchestra.run.smithi029.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-11-27T15:12:19.630 INFO:teuthology.orchestra.run.smithi008.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.6 MB/s | 238 kB 00:00 2023-11-27T15:12:19.719 INFO:teuthology.orchestra.run.smithi029.stdout:(127/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-11-27T15:12:19.747 INFO:teuthology.orchestra.run.smithi008.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 509 kB/s | 72 kB 00:00 2023-11-27T15:12:19.773 INFO:teuthology.orchestra.run.smithi008.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 583 kB/s | 83 kB 00:00 2023-11-27T15:12:19.786 INFO:teuthology.orchestra.run.smithi029.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 5.4 MB/s | 1.5 MB 00:00 2023-11-27T15:12:19.831 INFO:teuthology.orchestra.run.smithi008.stdout:(121/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 8.4 MB/s | 2.3 MB 00:00 2023-11-27T15:12:19.865 INFO:teuthology.orchestra.run.smithi008.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 510 kB/s | 60 kB 00:00 2023-11-27T15:12:19.907 INFO:teuthology.orchestra.run.smithi008.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 735 kB/s | 98 kB 00:00 2023-11-27T15:12:19.938 INFO:teuthology.orchestra.run.smithi029.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.1 MB/s | 798 kB 00:00 2023-11-27T15:12:19.940 INFO:teuthology.orchestra.run.smithi008.stdout:(124/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 529 kB/s | 58 kB 00:00 2023-11-27T15:12:20.016 INFO:teuthology.orchestra.run.smithi008.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 256 kB/s | 39 kB 00:00 2023-11-27T15:12:20.041 INFO:teuthology.orchestra.run.smithi008.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.7 MB/s | 171 kB 00:00 2023-11-27T15:12:20.075 INFO:teuthology.orchestra.run.smithi008.stdout:(127/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.8 MB/s | 644 kB 00:00 2023-11-27T15:12:20.180 INFO:teuthology.orchestra.run.smithi029.stdout:(130/131): librbd1-18.0.0-3906.g1c956514.el8.x8 7.7 MB/s | 3.6 MB 00:00 2023-11-27T15:12:20.183 INFO:teuthology.orchestra.run.smithi008.stdout:(128/131): NetworkManager-initscripts-updown-1. 824 kB/s | 138 kB 00:00 2023-11-27T15:12:20.350 INFO:teuthology.orchestra.run.smithi008.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.7 MB/s | 798 kB 00:00 2023-11-27T15:12:20.559 INFO:teuthology.orchestra.run.smithi008.stdout:(130/131): librbd1-18.0.0-3906.g1c956514.el8.x8 7.4 MB/s | 3.6 MB 00:00 2023-11-27T15:12:20.801 INFO:teuthology.orchestra.run.smithi008.stdout:(131/131): librados2-18.0.0-3906.g1c956514.el8. 4.8 MB/s | 3.7 MB 00:00 2023-11-27T15:12:20.805 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:12:20.805 INFO:teuthology.orchestra.run.smithi008.stdout:Total 10 MB/s | 197 MB 00:18 2023-11-27T15:12:20.980 INFO:teuthology.orchestra.run.smithi029.stdout:(131/131): librados2-18.0.0-3906.g1c956514.el8. 2.7 MB/s | 3.7 MB 00:01 2023-11-27T15:12:20.984 INFO:teuthology.orchestra.run.smithi029.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:12:20.984 INFO:teuthology.orchestra.run.smithi029.stdout:Total 9.4 MB/s | 197 MB 00:20 2023-11-27T15:12:21.856 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction check 2023-11-27T15:12:21.987 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction check succeeded. 2023-11-27T15:12:21.988 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction test 2023-11-27T15:12:22.541 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2023-11-27T15:12:22.690 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2023-11-27T15:12:22.690 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2023-11-27T15:12:23.044 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2023-11-27T15:12:23.138 INFO:teuthology.orchestra.run.smithi029.stdout:Transaction test succeeded. 2023-11-27T15:12:23.142 INFO:teuthology.orchestra.run.smithi029.stdout:Running transaction 2023-11-27T15:12:23.242 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-27T15:12:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-27T15:12:23.620 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-27T15:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:12:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:12:24.177 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:12:24.391 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2023-11-27T15:12:24.396 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2023-11-27T15:12:24.614 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:12:24.852 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:12:25.272 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:12:25.427 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:12:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:12:25.743 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:12:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:12:26.053 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:12:26.312 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-27T15:12:26.540 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-27T15:12:27.075 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-27T15:12:27.336 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-27T15:12:27.477 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:12:27.614 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:12:27.743 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:12:27.824 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:12:27.960 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:12:28.102 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:12:28.285 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:12:28.534 INFO:teuthology.orchestra.run.smithi029.stdout: Preparing : 1/1 2023-11-27T15:12:28.729 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-27T15:12:28.942 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:12:28.964 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-27T15:12:29.118 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:12:29.148 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-27T15:12:29.260 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-27T15:12:29.481 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-27T15:12:29.544 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2023-11-27T15:12:29.753 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:12:29.756 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-27T15:12:29.907 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-27T15:12:29.944 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-27T15:12:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:12:30.156 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 5/134 2023-11-27T15:12:30.202 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-27T15:12:30.391 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:12:30.566 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:12:30.590 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 5/134 2023-11-27T15:12:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:12:30.816 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:12:30.839 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:12:30.970 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:12:31.008 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-27T15:12:31.155 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:12:31.210 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-27T15:12:31.241 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:12:31.312 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:12:31.424 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:12:31.447 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-27T15:12:31.565 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:12:31.606 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-27T15:12:31.633 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-27T15:12:31.859 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-27T15:12:31.863 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-27T15:12:32.075 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:12:32.085 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-27T15:12:32.122 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-27T15:12:32.266 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-27T15:12:32.299 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:12:32.397 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-27T15:12:32.454 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:12:32.561 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-27T15:12:32.782 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:12:32.814 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-27T15:12:32.898 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:12:32.972 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:12:33.074 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:12:33.101 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:12:33.239 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:12:33.371 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-27T15:12:33.389 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:12:33.409 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-27T15:12:33.610 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-27T15:12:33.683 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-27T15:12:33.771 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-27T15:12:33.855 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-27T15:12:33.959 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-27T15:12:33.991 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-27T15:12:34.138 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-27T15:12:34.255 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-27T15:12:34.291 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-27T15:12:34.332 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-27T15:12:34.463 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-27T15:12:34.476 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:12:34.616 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-27T15:12:34.782 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:12:34.896 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:12:35.073 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:12:35.115 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-27T15:12:35.163 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:12:35.297 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-27T15:12:35.306 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:12:35.508 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:12:35.690 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:12:35.930 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-27T15:12:36.130 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-27T15:12:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:12:36.443 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:12:36.623 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:12:36.679 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-27T15:12:36.866 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-27T15:12:36.903 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-27T15:12:37.030 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-27T15:12:37.241 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-27T15:12:37.346 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:12:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-27T15:12:37.575 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-27T15:12:37.581 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:12:37.592 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-27T15:12:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-27T15:12:37.789 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-27T15:12:37.945 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-27T15:12:38.042 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-27T15:12:38.081 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-27T15:12:38.205 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-27T15:12:38.320 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-27T15:12:38.324 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-27T15:12:38.469 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:12:38.520 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-27T15:12:38.581 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-27T15:12:38.723 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-27T15:12:38.780 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-27T15:12:38.912 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-27T15:12:39.017 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-27T15:12:39.188 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-27T15:12:39.239 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-27T15:12:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-27T15:12:39.476 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-27T15:12:39.668 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-27T15:12:39.733 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-27T15:12:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-27T15:12:39.998 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-27T15:12:40.222 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-27T15:12:40.255 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-27T15:12:40.379 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-27T15:12:40.459 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-27T15:12:40.660 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-27T15:12:40.716 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-27T15:12:40.850 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-27T15:12:41.040 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-27T15:12:41.205 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-27T15:12:41.490 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-27T15:12:41.541 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-27T15:12:41.821 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-27T15:12:41.833 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-27T15:12:41.854 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-27T15:12:42.100 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-27T15:12:42.145 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-27T15:12:42.185 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-27T15:12:42.299 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-27T15:12:42.402 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-27T15:12:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-27T15:12:42.520 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-27T15:12:42.615 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-27T15:12:42.619 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-27T15:12:42.689 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:12:42.832 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:12:42.886 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-27T15:12:42.906 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:12:43.075 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-27T15:12:43.184 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-27T15:12:43.267 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-27T15:12:43.371 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-27T15:12:43.436 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-27T15:12:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:12:43.596 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-27T15:12:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:12:43.776 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-27T15:12:43.851 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-27T15:12:43.962 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-27T15:12:44.003 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-27T15:12:44.037 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-27T15:12:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-27T15:12:44.272 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-27T15:12:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-27T15:12:44.556 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-27T15:12:44.648 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-27T15:12:44.721 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-27T15:12:44.798 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-27T15:12:44.874 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-27T15:12:44.916 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:12:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-27T15:12:45.040 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-27T15:12:45.170 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-27T15:12:45.209 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-27T15:12:45.325 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-27T15:12:45.363 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-27T15:12:45.372 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-27T15:12:45.570 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-27T15:12:45.581 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-27T15:12:45.598 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-27T15:12:45.747 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-27T15:12:45.765 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-27T15:12:45.805 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-27T15:12:45.934 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-27T15:12:45.944 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-27T15:12:46.047 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-27T15:12:46.064 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-27T15:12:46.148 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-27T15:12:46.269 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-27T15:12:46.309 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-27T15:12:46.382 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-27T15:12:46.457 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-27T15:12:46.518 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-27T15:12:46.627 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-27T15:12:46.689 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-27T15:12:46.776 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-27T15:12:46.858 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-27T15:12:46.979 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-27T15:12:47.013 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-27T15:12:47.090 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-27T15:12:47.166 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-27T15:12:47.271 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-27T15:12:47.341 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-27T15:12:47.367 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-27T15:12:47.516 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-27T15:12:47.576 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-27T15:12:47.604 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-27T15:12:47.811 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-27T15:12:48.005 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-27T15:12:48.205 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-27T15:12:48.339 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-27T15:12:48.421 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-27T15:12:48.530 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-27T15:12:48.654 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-27T15:12:48.768 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-27T15:12:48.772 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-27T15:12:48.914 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-27T15:12:49.034 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-27T15:12:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-27T15:12:49.095 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-27T15:12:49.268 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-27T15:12:49.327 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-27T15:12:49.391 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-27T15:12:49.579 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-27T15:12:49.638 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-27T15:12:49.641 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-27T15:12:49.885 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-27T15:12:49.896 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-27T15:12:49.904 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-27T15:12:50.122 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-27T15:12:50.123 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-27T15:12:50.152 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-27T15:12:50.349 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-27T15:12:50.414 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-27T15:12:50.528 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-27T15:12:50.596 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-27T15:12:50.811 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-27T15:12:50.875 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-27T15:12:50.947 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-27T15:12:51.076 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-27T15:12:51.140 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:12:51.172 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-27T15:12:51.213 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:12:51.303 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-27T15:12:51.399 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-27T15:12:51.474 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-27T15:12:51.517 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-27T15:12:51.683 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-27T15:12:51.764 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-27T15:12:51.901 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-27T15:12:51.956 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-27T15:12:52.110 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-27T15:12:52.192 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-27T15:12:52.292 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-27T15:12:52.404 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:12:52.481 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-27T15:12:52.493 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:12:52.643 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-27T15:12:52.769 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-27T15:12:52.888 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-27T15:12:53.020 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-27T15:12:53.187 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:12:53.200 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-27T15:12:53.289 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:12:53.418 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-27T15:12:53.566 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-27T15:12:53.700 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-27T15:12:53.749 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-27T15:12:53.920 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-27T15:12:53.990 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-27T15:12:54.121 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-27T15:12:54.215 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-27T15:12:54.366 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-27T15:12:54.557 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-27T15:12:54.813 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-27T15:12:54.985 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-27T15:12:55.169 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-27T15:12:55.416 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-27T15:12:55.544 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-27T15:12:55.612 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:12:55.616 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-27T15:12:55.807 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:12:55.834 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-27T15:12:55.840 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-27T15:12:56.065 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-27T15:12:56.105 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-27T15:12:56.246 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-27T15:12:56.421 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:12:56.449 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-27T15:12:56.493 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-27T15:12:56.721 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-27T15:12:56.787 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-27T15:12:56.811 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:12:56.898 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:12:56.970 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-27T15:12:57.024 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-27T15:12:57.073 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:12:57.262 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-27T15:12:57.514 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-27T15:12:57.716 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-27T15:12:57.848 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-27T15:12:57.970 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-27T15:12:58.044 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:12:58.122 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:12:58.361 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-27T15:12:59.383 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-27T15:12:59.720 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-27T15:13:00.055 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-27T15:13:00.451 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-27T15:13:00.765 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-27T15:13:01.021 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-27T15:13:01.897 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-27T15:13:02.103 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:13:02.225 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:13:02.321 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:13:02.420 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-27T15:13:02.552 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:13:03.179 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:13:04.553 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:13:04.643 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:13:04.802 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:13:08.226 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:13:08.478 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:13:09.179 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:13:09.508 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:13:09.645 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:13:10.552 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:13:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:13:16.653 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2023-11-27T15:13:16.653 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2023-11-27T15:13:16.653 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /dev 2023-11-27T15:13:16.653 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /run 2023-11-27T15:13:16.653 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2023-11-27T15:13:16.653 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2023-11-27T15:13:16.653 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2023-11-27T15:13:16.653 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2023-11-27T15:13:16.653 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /dev 2023-11-27T15:13:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:13:17.029 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:13:18.693 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:13:18.850 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:13:19.080 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:13:19.237 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:13:19.719 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:13:19.887 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:13:19.906 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:13:21.744 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:13:21.927 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:13:22.076 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-27T15:13:22.252 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-27T15:13:22.461 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:13:22.684 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:13:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:13:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:13:24.152 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:13:24.249 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:13:24.619 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-27T15:13:24.847 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:13:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:13:25.061 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:13:25.073 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:13:26.257 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:13:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /sys 2023-11-27T15:13:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /proc 2023-11-27T15:13:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /dev 2023-11-27T15:13:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /run 2023-11-27T15:13:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /mnt 2023-11-27T15:13:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /var/tmp 2023-11-27T15:13:26.257 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /home 2023-11-27T15:13:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /tmp 2023-11-27T15:13:26.258 INFO:teuthology.orchestra.run.smithi029.stdout:skipping the directory /dev 2023-11-27T15:13:26.258 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:13:26.368 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-27T15:13:26.594 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:13:26.628 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:13:26.641 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:13:26.802 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:13:26.846 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:13:27.002 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:13:27.368 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:13:27.552 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:13:28.307 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:13:28.507 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:13:28.737 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:13:28.916 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:13:29.398 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:13:29.633 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:13:29.652 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:13:31.513 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:13:31.686 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:13:31.687 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /sys 2023-11-27T15:13:31.687 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /proc 2023-11-27T15:13:31.687 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /dev 2023-11-27T15:13:31.687 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /run 2023-11-27T15:13:31.687 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /mnt 2023-11-27T15:13:31.687 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /var/tmp 2023-11-27T15:13:31.687 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /home 2023-11-27T15:13:31.687 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /tmp 2023-11-27T15:13:31.687 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /dev 2023-11-27T15:13:31.687 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:13:31.701 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:13:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:13:31.813 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-27T15:13:32.025 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-27T15:13:32.032 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-27T15:13:32.142 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:13:32.201 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:13:32.223 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:13:32.409 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:13:32.447 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:13:32.756 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:13:33.358 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-27T15:13:33.705 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-27T15:13:33.781 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:13:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 119/134 2023-11-27T15:13:33.948 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:13:33.957 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:13:34.045 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:13:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:13:34.180 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:13:34.223 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-27T15:13:34.402 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:13:34.426 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-27T15:13:34.461 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:13:34.642 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:13:34.666 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:13:34.679 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:13:34.890 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:13:34.902 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:13:35.224 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:13:35.466 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:13:35.510 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:13:36.224 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-27T15:13:36.436 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:13:36.471 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:13:36.640 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:13:36.683 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:13:36.821 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:13:36.848 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:13:37.101 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:13:37.121 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:13:37.214 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:13:37.285 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-27T15:13:37.389 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:13:37.585 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-27T15:13:37.929 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-27T15:13:39.213 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:13:39.213 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:13:39.213 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-27T15:13:39.213 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-27T15:13:39.213 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:13:39.213 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-27T15:13:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:13:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:13:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-27T15:13:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:13:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:13:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:13:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:13:39.214 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:13:39.321 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:13:39.321 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:13:39.415 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:13:39.583 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:13:39.621 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:13:39.784 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:13:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:13:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-27T15:13:40.317 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-27T15:13:40.318 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-27T15:13:40.319 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-27T15:13:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-27T15:13:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-27T15:13:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-27T15:13:40.320 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-27T15:13:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-27T15:13:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-27T15:13:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-27T15:13:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-27T15:13:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-27T15:13:40.329 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-27T15:13:40.330 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-27T15:13:40.331 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-27T15:13:40.332 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-27T15:13:40.333 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-27T15:13:40.334 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-27T15:13:40.335 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-27T15:13:40.336 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-27T15:13:40.337 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-27T15:13:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:13:40.338 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-27T15:13:40.781 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:13:40.948 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:13:41.086 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:13:41.462 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-27T15:13:41.586 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:13:41.691 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:13:41.760 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:13:41.973 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:13:42.004 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:13:42.589 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-27T15:13:42.925 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-27T15:13:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:13:42.958 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:56 ago on Mon 27 Nov 2023 03:11:46 PM UTC. 2023-11-27T15:13:43.143 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 117/134 2023-11-27T15:13:43.401 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-27T15:13:43.529 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-27T15:13:43.571 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-27T15:13:43.765 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:13:43.803 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-27T15:13:43.830 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:13:44.035 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-27T15:13:44.173 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:13:44.201 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-27T15:13:44.249 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:13:44.356 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:13:44.415 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:13:44.844 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:13:45.123 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:13:46.792 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:13:47.097 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:13:47.115 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:13:47.391 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-27T15:13:47.647 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-27T15:13:47.995 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-27T15:13:48.400 INFO:teuthology.orchestra.run.smithi029.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-27T15:13:48.646 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-27T15:13:48.739 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-27T15:13:48.963 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-27T15:13:48.978 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-27T15:13:49.110 INFO:teuthology.orchestra.run.smithi060.stdout:Installed products updated. 2023-11-27T15:13:49.121 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-27T15:13:49.132 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-27T15:13:49.291 INFO:teuthology.orchestra.run.smithi029.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-27T15:13:49.291 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:13:49.346 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:13:49.346 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2023-11-27T15:13:49.346 INFO:teuthology.orchestra.run.smithi060.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-27T15:13:49.346 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: automake-1.16.1-7.el8.noarch 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: bison-3.0.4-10.el8.x86_64 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.347 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.348 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: flex-2.6.1-9.el8.x86_64 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: jq-1.6-3.el8.x86_64 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-27T15:13:49.349 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: libpq-13.5-1.el8.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-27T15:13:49.350 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-27T15:13:49.351 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.352 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-27T15:13:49.353 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-2.6-1.el8.noarch 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-27T15:13:49.354 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-27T15:13:49.355 INFO:teuthology.orchestra.run.smithi060.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-27T15:13:49.356 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-27T15:13:49.357 INFO:teuthology.orchestra.run.smithi060.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-27T15:13:49.357 INFO:teuthology.orchestra.run.smithi060.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-27T15:13:49.357 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-27T15:13:49.357 INFO:teuthology.orchestra.run.smithi060.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-27T15:13:49.357 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:13:49.357 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2023-11-27T15:13:49.500 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:13:49.500 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:13:49.553 DEBUG:teuthology.parallel:result is None 2023-11-27T15:13:49.659 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:13:49.751 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:13:49.971 INFO:teuthology.orchestra.run.smithi029.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:13:50.020 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-27T15:13:50.238 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:13:50.530 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:13:50.538 INFO:teuthology.orchestra.run.smithi029.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:13:50.538 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-27T15:13:50.539 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-27T15:13:50.539 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-27T15:13:50.539 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-27T15:13:50.539 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-27T15:13:50.539 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:13:50.539 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:13:50.539 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:13:50.539 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:13:50.540 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-27T15:13:50.540 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-27T15:13:50.540 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-27T15:13:50.540 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-27T15:13:50.540 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-27T15:13:50.540 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-27T15:13:50.540 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-27T15:13:50.540 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-27T15:13:50.540 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-27T15:13:50.541 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-27T15:13:50.541 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-27T15:13:50.541 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-27T15:13:50.541 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-27T15:13:50.541 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-27T15:13:50.541 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-27T15:13:50.541 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-27T15:13:50.541 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-27T15:13:50.541 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-27T15:13:50.541 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-27T15:13:50.542 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-27T15:13:50.542 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-27T15:13:50.552 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-27T15:13:50.552 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-27T15:13:50.552 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-27T15:13:50.552 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-27T15:13:50.552 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-27T15:13:50.552 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-27T15:13:50.552 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-27T15:13:50.552 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-27T15:13:50.552 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-27T15:13:50.553 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-27T15:13:50.553 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-27T15:13:50.553 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-27T15:13:50.553 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-27T15:13:50.553 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-27T15:13:50.553 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-27T15:13:50.553 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-27T15:13:50.553 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-27T15:13:50.554 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-27T15:13:50.554 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-27T15:13:50.554 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-27T15:13:50.554 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-27T15:13:50.554 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-27T15:13:50.554 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-27T15:13:50.554 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-27T15:13:50.554 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-27T15:13:50.554 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-27T15:13:50.555 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-27T15:13:50.555 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-27T15:13:50.555 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-27T15:13:50.555 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-27T15:13:50.555 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-27T15:13:50.555 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-27T15:13:50.555 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-27T15:13:50.555 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-27T15:13:50.555 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-27T15:13:50.555 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-27T15:13:50.556 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-27T15:13:50.556 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-27T15:13:50.556 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-27T15:13:50.556 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-27T15:13:50.556 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-27T15:13:50.556 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-27T15:13:50.556 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-27T15:13:50.556 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-27T15:13:50.557 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-27T15:13:50.557 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-27T15:13:50.557 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-27T15:13:50.557 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-27T15:13:50.557 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-27T15:13:50.557 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-27T15:13:50.557 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-27T15:13:50.557 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-27T15:13:50.558 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-27T15:13:50.558 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-27T15:13:50.558 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-27T15:13:50.558 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-27T15:13:50.558 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-27T15:13:50.558 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-27T15:13:50.558 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-27T15:13:50.558 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-27T15:13:50.558 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-27T15:13:50.558 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-27T15:13:50.559 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-27T15:13:50.559 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-27T15:13:50.559 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-27T15:13:50.559 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-27T15:13:50.559 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-27T15:13:50.559 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-27T15:13:50.559 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-27T15:13:50.559 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-27T15:13:50.559 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-27T15:13:50.559 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-27T15:13:50.560 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-27T15:13:50.560 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-27T15:13:50.560 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-27T15:13:50.560 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-27T15:13:50.560 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-27T15:13:50.560 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-27T15:13:50.560 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-27T15:13:50.560 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-27T15:13:50.560 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-27T15:13:50.560 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-27T15:13:50.561 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-27T15:13:50.561 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-27T15:13:50.561 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-27T15:13:50.561 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-27T15:13:50.561 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-27T15:13:50.561 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-27T15:13:50.561 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-27T15:13:50.561 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-27T15:13:50.561 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-27T15:13:50.561 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-27T15:13:50.562 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-27T15:13:50.562 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-27T15:13:50.562 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-27T15:13:50.562 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-27T15:13:50.562 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-27T15:13:50.562 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-27T15:13:50.562 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-27T15:13:50.562 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-27T15:13:50.562 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-27T15:13:50.562 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:13:50.563 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-27T15:13:51.912 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-27T15:13:52.298 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-27T15:13:52.530 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 119/134 2023-11-27T15:13:52.719 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:13:52.919 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-27T15:13:52.985 INFO:teuthology.orchestra.run.smithi029.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:13:52.985 INFO:teuthology.orchestra.run.smithi029.stdout:Last metadata expiration check: 0:02:05 ago on Mon 27 Nov 2023 03:11:47 PM UTC. 2023-11-27T15:13:53.073 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:13:56.580 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:13:56.889 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:13:56.928 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:13:57.173 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-27T15:13:57.431 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-27T15:13:57.864 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-27T15:13:58.137 INFO:teuthology.orchestra.run.smithi029.stdout:Installed products updated. 2023-11-27T15:13:58.190 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-27T15:13:58.370 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:13:58.370 INFO:teuthology.orchestra.run.smithi029.stdout:Upgraded: 2023-11-27T15:13:58.370 INFO:teuthology.orchestra.run.smithi029.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-27T15:13:58.370 INFO:teuthology.orchestra.run.smithi029.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.370 INFO:teuthology.orchestra.run.smithi029.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.370 INFO:teuthology.orchestra.run.smithi029.stdout:Installed: 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: automake-1.16.1-7.el8.noarch 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: bison-3.0.4-10.el8.x86_64 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.371 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:58.372 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: flex-2.6.1-9.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: jq-1.6-3.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.373 INFO:teuthology.orchestra.run.smithi029.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: libpq-13.5-1.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-27T15:13:58.374 INFO:teuthology.orchestra.run.smithi029.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:13:58.375 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-27T15:13:58.376 INFO:teuthology.orchestra.run.smithi029.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-27T15:13:58.377 INFO:teuthology.orchestra.run.smithi029.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-portend-2.6-1.el8.noarch 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-27T15:13:58.378 INFO:teuthology.orchestra.run.smithi029.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-27T15:13:58.379 INFO:teuthology.orchestra.run.smithi029.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-27T15:13:58.380 INFO:teuthology.orchestra.run.smithi029.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-27T15:13:58.381 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:13:58.381 INFO:teuthology.orchestra.run.smithi029.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-27T15:13:58.381 INFO:teuthology.orchestra.run.smithi029.stdout: 2023-11-27T15:13:58.381 INFO:teuthology.orchestra.run.smithi029.stdout:Complete! 2023-11-27T15:13:58.395 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:13:58.575 DEBUG:teuthology.parallel:result is None 2023-11-27T15:13:58.637 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-27T15:13:58.862 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:13:58.995 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:13:59.204 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:13:59.235 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:13:59.235 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:13:59.477 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:13:59.478 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:13:59.737 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:13:59.996 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:14:00.279 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:14:00.279 INFO:teuthology.orchestra.run.smithi008.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-27T15:14:00.279 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:14:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-27T15:14:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-27T15:14:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-27T15:14:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-27T15:14:00.915 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-27T15:14:00.916 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-27T15:14:00.917 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-27T15:14:00.918 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-27T15:14:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-27T15:14:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-27T15:14:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-27T15:14:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-27T15:14:00.926 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-27T15:14:00.927 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-27T15:14:00.928 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-27T15:14:00.929 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-27T15:14:00.930 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-27T15:14:00.931 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-27T15:14:00.932 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-27T15:14:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-27T15:14:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:14:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-27T15:14:03.650 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:14:03.650 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:15 ago on Mon 27 Nov 2023 03:11:48 PM UTC. 2023-11-27T15:14:10.165 INFO:teuthology.orchestra.run.smithi008.stdout:Installed products updated. 2023-11-27T15:14:10.407 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:14:10.407 INFO:teuthology.orchestra.run.smithi008.stdout:Upgraded: 2023-11-27T15:14:10.407 INFO:teuthology.orchestra.run.smithi008.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-27T15:14:10.407 INFO:teuthology.orchestra.run.smithi008.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.407 INFO:teuthology.orchestra.run.smithi008.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.407 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2023-11-27T15:14:10.407 INFO:teuthology.orchestra.run.smithi008.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-27T15:14:10.407 INFO:teuthology.orchestra.run.smithi008.stdout: automake-1.16.1-7.el8.noarch 2023-11-27T15:14:10.407 INFO:teuthology.orchestra.run.smithi008.stdout: bison-3.0.4-10.el8.x86_64 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:14:10.408 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.409 INFO:teuthology.orchestra.run.smithi008.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: flex-2.6.1-9.el8.x86_64 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: jq-1.6-3.el8.x86_64 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.410 INFO:teuthology.orchestra.run.smithi008.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: libpq-13.5-1.el8.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-27T15:14:10.411 INFO:teuthology.orchestra.run.smithi008.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-27T15:14:10.412 INFO:teuthology.orchestra.run.smithi008.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.413 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-27T15:14:10.414 INFO:teuthology.orchestra.run.smithi008.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-portend-2.6-1.el8.noarch 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-27T15:14:10.415 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-27T15:14:10.416 INFO:teuthology.orchestra.run.smithi008.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:14:10.417 INFO:teuthology.orchestra.run.smithi008.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-27T15:14:10.418 INFO:teuthology.orchestra.run.smithi008.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-27T15:14:10.418 INFO:teuthology.orchestra.run.smithi008.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-27T15:14:10.418 INFO:teuthology.orchestra.run.smithi008.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-27T15:14:10.418 INFO:teuthology.orchestra.run.smithi008.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-27T15:14:10.418 INFO:teuthology.orchestra.run.smithi008.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-27T15:14:10.418 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:14:10.418 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2023-11-27T15:14:10.688 DEBUG:teuthology.parallel:result is None 2023-11-27T15:14:10.688 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:14:10.840 DEBUG:teuthology.orchestra.run.smithi008:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-27T15:14:10.894 INFO:teuthology.orchestra.run.smithi008.stdout:18.0.0-3906.g1c956514.el8 2023-11-27T15:14:10.895 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-27T15:14:10.895 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-27T15:14:10.895 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:14:11.102 DEBUG:teuthology.orchestra.run.smithi029:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-27T15:14:11.131 INFO:teuthology.orchestra.run.smithi029.stdout:18.0.0-3906.g1c956514.el8 2023-11-27T15:14:11.131 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-27T15:14:11.132 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-27T15:14:11.132 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:14:11.339 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-27T15:14:11.368 INFO:teuthology.orchestra.run.smithi060.stdout:18.0.0-3906.g1c956514.el8 2023-11-27T15:14:11.368 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-27T15:14:11.368 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-27T15:14:11.369 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-27T15:14:11.369 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:14:11.369 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-27T15:14:11.467 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T15:14:11.467 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-27T15:14:11.517 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T15:14:11.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-27T15:14:11.567 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-27T15:14:11.568 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:14:11.568 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/daemon-helper 2023-11-27T15:14:11.640 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-27T15:14:11.749 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T15:14:11.749 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/daemon-helper 2023-11-27T15:14:11.783 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-27T15:14:11.854 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T15:14:11.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2023-11-27T15:14:11.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-27T15:14:11.959 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-27T15:14:11.959 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:14:11.959 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-27T15:14:12.028 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-27T15:14:12.136 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T15:14:12.137 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-27T15:14:12.169 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-27T15:14:12.241 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T15:14:12.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-27T15:14:12.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-27T15:14:12.346 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-27T15:14:12.347 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:14:12.347 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/stdin-killer 2023-11-27T15:14:12.415 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-27T15:14:12.524 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T15:14:12.524 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/usr/bin/stdin-killer 2023-11-27T15:14:12.557 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-27T15:14:12.628 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T15:14:12.628 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2023-11-27T15:14:12.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-27T15:14:12.732 INFO:teuthology.run_tasks:Running task cephadm... 2023-11-27T15:14:12.800 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-11-27T15:14:12.800 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:14:12.801 INFO:tasks.cephadm:Cluster fsid is 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:14:12.801 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-11-27T15:14:12.801 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.8', 'mon.b': '172.21.15.29', 'mon.c': '172.21.15.60'} 2023-11-27T15:14:12.801 INFO:tasks.cephadm:First mon is mon.a on smithi008 2023-11-27T15:14:12.801 INFO:tasks.cephadm:First mgr is x 2023-11-27T15:14:12.801 INFO:tasks.cephadm:Normalizing hostnames... 2023-11-27T15:14:12.802 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname $(hostname -s) 2023-11-27T15:14:12.873 DEBUG:teuthology.orchestra.run.smithi029:> sudo hostname $(hostname -s) 2023-11-27T15:14:12.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo hostname $(hostname -s) 2023-11-27T15:14:12.939 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-11-27T15:14:12.940 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:14:13.071 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74215/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-3906-g1c956514', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-3906.g1c956514'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-11-16 20:52:31.578811', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-choffman-62793', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-11-27T15:14:13.206 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-choffman-62793, sha1 1c9565146c8647d4e00eb6c7ae626dc192680e7b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:14:13.206 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm 2023-11-27T15:14:13.206 DEBUG:teuthology.orchestra.run.smithi008:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-27T15:14:14.162 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 27 15:14 /home/ubuntu/cephtest/cephadm 2023-11-27T15:14:14.162 DEBUG:teuthology.orchestra.run.smithi029:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-27T15:14:14.816 INFO:teuthology.orchestra.run.smithi029.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 27 15:14 /home/ubuntu/cephtest/cephadm 2023-11-27T15:14:14.816 DEBUG:teuthology.orchestra.run.smithi060:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-27T15:14:15.319 INFO:teuthology.orchestra.run.smithi060.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 27 15:14 /home/ubuntu/cephtest/cephadm 2023-11-27T15:14:15.319 DEBUG:teuthology.orchestra.run.smithi008:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-27T15:14:15.370 DEBUG:teuthology.orchestra.run.smithi029:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-27T15:14:15.393 DEBUG:teuthology.orchestra.run.smithi060:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-27T15:14:15.428 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b on all hosts... 2023-11-27T15:14:15.428 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-27T15:14:15.433 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-27T15:14:15.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-27T15:14:16.994 INFO:teuthology.orchestra.run.smithi060.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:14:18.218 INFO:teuthology.orchestra.run.smithi029.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:14:20.006 INFO:teuthology.orchestra.run.smithi008.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:14:51.277 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:14:51.277 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:14:51.278 INFO:teuthology.orchestra.run.smithi008.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-27T15:14:51.278 INFO:teuthology.orchestra.run.smithi008.stdout: "repo_digests": [ 2023-11-27T15:14:51.278 INFO:teuthology.orchestra.run.smithi008.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-27T15:14:51.278 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2023-11-27T15:14:51.278 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:15:31.721 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-27T15:15:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:15:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-27T15:15:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "repo_digests": [ 2023-11-27T15:15:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-27T15:15:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2023-11-27T15:15:31.722 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-27T15:15:32.346 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2023-11-27T15:15:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:15:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-27T15:15:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: "repo_digests": [ 2023-11-27T15:15:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-27T15:15:32.346 INFO:teuthology.orchestra.run.smithi029.stdout: ] 2023-11-27T15:15:32.346 INFO:teuthology.orchestra.run.smithi029.stdout:} 2023-11-27T15:15:32.366 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /etc/ceph 2023-11-27T15:15:32.448 DEBUG:teuthology.orchestra.run.smithi029:> sudo mkdir -p /etc/ceph 2023-11-27T15:15:32.480 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph 2023-11-27T15:15:32.519 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 777 /etc/ceph 2023-11-27T15:15:32.591 DEBUG:teuthology.orchestra.run.smithi029:> sudo chmod 777 /etc/ceph 2023-11-27T15:15:32.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /etc/ceph 2023-11-27T15:15:32.654 INFO:tasks.cephadm:Writing seed config... 2023-11-27T15:15:32.655 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-11-27T15:15:32.655 INFO:tasks.cephadm: override: [client] debug client = 20 2023-11-27T15:15:32.656 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-11-27T15:15:32.656 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-11-27T15:15:32.656 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-11-27T15:15:32.656 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-11-27T15:15:32.656 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-11-27T15:15:32.656 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-11-27T15:15:32.657 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-11-27T15:15:32.657 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-11-27T15:15:32.657 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-11-27T15:15:32.657 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-11-27T15:15:32.657 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-11-27T15:15:32.657 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-11-27T15:15:32.657 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-11-27T15:15:32.658 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-11-27T15:15:32.658 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-11-27T15:15:32.658 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-11-27T15:15:32.658 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-11-27T15:15:32.658 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-11-27T15:15:32.658 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-11-27T15:15:32.658 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-11-27T15:15:32.659 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-11-27T15:15:32.659 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-11-27T15:15:32.659 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-11-27T15:15:32.659 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-11-27T15:15:32.659 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-11-27T15:15:32.659 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-11-27T15:15:32.659 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-11-27T15:15:32.660 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-11-27T15:15:32.660 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-11-27T15:15:32.660 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-11-27T15:15:32.660 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-11-27T15:15:32.660 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-11-27T15:15:32.660 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-11-27T15:15:32.660 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-11-27T15:15:32.661 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-11-27T15:15:32.661 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-11-27T15:15:32.661 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-11-27T15:15:32.661 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-11-27T15:15:32.661 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-11-27T15:15:32.661 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-11-27T15:15:32.661 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-11-27T15:15:32.662 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:15:32.662 DEBUG:teuthology.orchestra.run.smithi008:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-11-27T15:15:32.707 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9f287ba8-8d37-11ee-95a2-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-11-27T15:15:32.708 DEBUG:teuthology.orchestra.run.smithi008:mon.a> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.a.service 2023-11-27T15:15:32.753 DEBUG:teuthology.orchestra.run.smithi008:mgr.x> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.x.service 2023-11-27T15:15:32.798 INFO:tasks.cephadm:Bootstrapping... 2023-11-27T15:15:32.798 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b -v bootstrap --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.8 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:15:32.823 INFO:journalctl@ceph.mon.a.smithi008.stdout:-- Logs begin at Mon 2023-11-27 14:56:10 UTC. -- 2023-11-27T15:15:32.869 INFO:journalctl@ceph.mgr.x.smithi008.stdout:-- Logs begin at Mon 2023-11-27 14:56:10 UTC. -- 2023-11-27T15:15:33.152 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:15:33.152 INFO:teuthology.orchestra.run.smithi008.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b', '-v', 'bootstrap', '--fsid', '9f287ba8-8d37-11ee-95a2-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.8', '--skip-admin-label'] 2023-11-27T15:15:33.328 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout 4.1.1 2023-11-27T15:15:33.328 INFO:teuthology.orchestra.run.smithi008.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-11-27T15:15:33.329 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying podman|docker is present... 2023-11-27T15:15:33.503 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout 4.1.1 2023-11-27T15:15:33.503 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying lvm2 is present... 2023-11-27T15:15:33.504 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying time synchronization is in place... 2023-11-27T15:15:33.518 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-27T15:15:33.518 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-27T15:15:33.531 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-27T15:15:33.532 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2023-11-27T15:15:33.545 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout enabled 2023-11-27T15:15:33.558 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout active 2023-11-27T15:15:33.558 INFO:teuthology.orchestra.run.smithi008.stdout:Unit chronyd.service is enabled and running 2023-11-27T15:15:33.559 INFO:teuthology.orchestra.run.smithi008.stdout:Repeating the final host check... 2023-11-27T15:15:33.732 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout 4.1.1 2023-11-27T15:15:33.732 INFO:teuthology.orchestra.run.smithi008.stdout:podman (/bin/podman) version 4.1.1 is present 2023-11-27T15:15:33.732 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl is present 2023-11-27T15:15:33.732 INFO:teuthology.orchestra.run.smithi008.stdout:lvcreate is present 2023-11-27T15:15:33.745 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-27T15:15:33.746 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-27T15:15:33.758 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-27T15:15:33.758 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2023-11-27T15:15:33.772 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout enabled 2023-11-27T15:15:33.785 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout active 2023-11-27T15:15:33.786 INFO:teuthology.orchestra.run.smithi008.stdout:Unit chronyd.service is enabled and running 2023-11-27T15:15:33.786 INFO:teuthology.orchestra.run.smithi008.stdout:Host looks OK 2023-11-27T15:15:33.786 INFO:teuthology.orchestra.run.smithi008.stdout:Cluster fsid: 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:15:33.786 INFO:teuthology.orchestra.run.smithi008.stdout:Acquiring lock 139671173212648 on /run/cephadm/9f287ba8-8d37-11ee-95a2-87774f69a715.lock 2023-11-27T15:15:33.787 INFO:teuthology.orchestra.run.smithi008.stdout:Lock 139671173212648 acquired on /run/cephadm/9f287ba8-8d37-11ee-95a2-87774f69a715.lock 2023-11-27T15:15:33.787 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying IP 172.21.15.8 port 3300 ... 2023-11-27T15:15:33.787 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying IP 172.21.15.8 port 6789 ... 2023-11-27T15:15:33.787 INFO:teuthology.orchestra.run.smithi008.stdout:Base mon IP(s) is [172.21.15.8:3300, 172.21.15.8:6789], mon addrv is [v2:172.21.15.8:3300,v1:172.21.15.8:6789] 2023-11-27T15:15:33.795 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-11-27T15:15:33.795 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.8 metric 100 2023-11-27T15:15:33.802 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-11-27T15:15:33.802 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-11-27T15:15:33.802 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1751sec hoplimit 64 pref medium 2023-11-27T15:15:33.809 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-11-27T15:15:33.809 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-11-27T15:15:33.809 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-27T15:15:33.809 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-11-27T15:15:33.809 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25fe/64 scope link 2023-11-27T15:15:33.809 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-27T15:15:33.810 INFO:teuthology.orchestra.run.smithi008.stdout:Mon IP `172.21.15.8` is in CIDR network `172.21.0.0/20` 2023-11-27T15:15:33.810 INFO:teuthology.orchestra.run.smithi008.stdout:Mon IP `172.21.15.8` is in CIDR network `172.21.0.0/20` 2023-11-27T15:15:33.810 INFO:teuthology.orchestra.run.smithi008.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-11-27T15:15:33.811 INFO:teuthology.orchestra.run.smithi008.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-11-27T15:15:33.812 INFO:teuthology.orchestra.run.smithi008.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:15:34.291 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2 2023-11-27T15:15:34.291 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:15:34.291 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Getting image source signatures 2023-11-27T15:15:34.291 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Copying blob sha256:eb1dc24ed1b729f25bfe21673f4e5cc46a126932dc459841b1c31fbebaca605e 2023-11-27T15:15:34.291 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Copying blob sha256:837e66e3ff89804ed1bb42786f80e6c23534ff65fe48526d5007449f4e1f79b0 2023-11-27T15:15:34.291 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Copying config sha256:ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2 2023-11-27T15:15:34.291 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Writing manifest to image destination 2023-11-27T15:15:34.292 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Storing signatures 2023-11-27T15:15:35.373 INFO:teuthology.orchestra.run.smithi008.stdout:ceph: stdout ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-27T15:15:35.373 INFO:teuthology.orchestra.run.smithi008.stdout:Ceph version: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-27T15:15:35.373 INFO:teuthology.orchestra.run.smithi008.stdout:Extracting ceph user uid/gid from container image... 2023-11-27T15:15:36.252 INFO:teuthology.orchestra.run.smithi008.stdout:stat: stdout 167 167 2023-11-27T15:15:36.252 INFO:teuthology.orchestra.run.smithi008.stdout:Creating initial keys... 2023-11-27T15:15:37.128 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph-authtool: stdout AQCYsmRlmmsCKRAAqsRj79RkmmrZh5bUGM069w== 2023-11-27T15:15:38.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph-authtool: stdout AQCZsmRlU5kKIRAAilStKXOQxhWPnRQQ3p9qDQ== 2023-11-27T15:15:38.780 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph-authtool: stdout AQCasmRlB6EMGBAAPDR3j/ZajM7BqoKRYJy9tg== 2023-11-27T15:15:38.781 INFO:teuthology.orchestra.run.smithi008.stdout:Creating initial monmap... 2023-11-27T15:15:39.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-27T15:15:39.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-11-27T15:15:39.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:15:39.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-27T15:15:39.761 INFO:teuthology.orchestra.run.smithi008.stdout:monmaptool for a [v2:172.21.15.8:3300,v1:172.21.15.8:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-27T15:15:39.761 INFO:teuthology.orchestra.run.smithi008.stdout:setting min_mon_release = pacific 2023-11-27T15:15:39.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: set fsid to 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:15:39.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-27T15:15:39.761 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:15:39.761 INFO:teuthology.orchestra.run.smithi008.stdout:Creating mon... 2023-11-27T15:15:40.638 INFO:teuthology.orchestra.run.smithi008.stdout:create mon.a on 2023-11-27T15:15:40.864 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-11-27T15:15:41.075 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-11-27T15:15:41.288 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9f287ba8-8d37-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-9f287ba8-8d37-11ee-95a2-87774f69a715.target. 2023-11-27T15:15:41.288 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9f287ba8-8d37-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-9f287ba8-8d37-11ee-95a2-87774f69a715.target. 2023-11-27T15:15:41.590 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.a 2023-11-27T15:15:41.591 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.a.service: Unit ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.a.service not loaded. 2023-11-27T15:15:41.791 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9f287ba8-8d37-11ee-95a2-87774f69a715.target.wants/ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.a.service → /etc/systemd/system/ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@.service. 2023-11-27T15:15:41.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:41 smithi008 systemd[1]: Starting Ceph mon.a for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:15:42.575 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:42 smithi008 podman[128244]: 2023-11-27T15:15:42.575 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:42 smithi008 bash[128244]: bddd8f35673fedb1a7b8b8e73762083166d3c35036f5fd4761573d2bf605069d 2023-11-27T15:15:42.622 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:15:42.623 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2023-11-27T15:15:42.638 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:15:42.638 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2023-11-27T15:15:42.638 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2023-11-27T15:15:42.638 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to enable service . firewalld.service is not available 2023-11-27T15:15:42.639 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mon to start... 2023-11-27T15:15:42.639 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mon... 2023-11-27T15:15:42.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:42 smithi008 ceph-mon[128284]: mkfs 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:15:42.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:42 smithi008 ceph-mon[128284]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-27T15:15:42.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:42 smithi008 systemd[1]: Started Ceph mon.a for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:15:43.729 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout cluster: 2023-11-27T15:15:43.729 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout id: 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout services: 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.698925s) 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout data: 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-11-27T15:15:43.730 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout pgs: 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.179+0000 7fd3c7339700 1 Processor -- start 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.180+0000 7fd3c7339700 1 -- start start 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.180+0000 7fd3c7339700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.180+0000 7fd3c7339700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0106260 con 0x7fd3c0105910 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.181+0000 7fd3c50d5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.181+0000 7fd3c50d5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50510/0 (socket says 172.21.15.8:50510) 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.181+0000 7fd3c50d5700 1 -- 172.21.15.8:0/2542876186 learned_addr learned my addr 172.21.15.8:0/2542876186 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.181+0000 7fd3c50d5700 1 -- 172.21.15.8:0/2542876186 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c01063a0 con 0x7fd3c0105910 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.182+0000 7fd3c50d5700 1 --2- 172.21.15.8:0/2542876186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0105d20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0098a0 tx=0x7fd3bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0c2f8e61df8064 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:43.731 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.182+0000 7fd3b7fff700 1 -- 172.21.15.8:0/2542876186 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3bc004160 con 0x7fd3c0105910 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.182+0000 7fd3b7fff700 1 -- 172.21.15.8:0/2542876186 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc0042c0 con 0x7fd3c0105910 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.182+0000 7fd3b7fff700 1 -- 172.21.15.8:0/2542876186 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3bc0045a0 con 0x7fd3c0105910 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.182+0000 7fd3c7339700 1 -- 172.21.15.8:0/2542876186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 msgr2=0x7fd3c0105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.182+0000 7fd3c7339700 1 --2- 172.21.15.8:0/2542876186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0105d20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0098a0 tx=0x7fd3bc009bb0 comp rx=0 tx=0).stop 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.183+0000 7fd3c7339700 1 -- 172.21.15.8:0/2542876186 shutdown_connections 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.183+0000 7fd3c7339700 1 --2- 172.21.15.8:0/2542876186 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0105d20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.183+0000 7fd3c7339700 1 -- 172.21.15.8:0/2542876186 >> 172.21.15.8:0/2542876186 conn(0x7fd3c0100fa0 msgr2=0x7fd3c01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.183+0000 7fd3c7339700 1 -- 172.21.15.8:0/2542876186 shutdown_connections 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.183+0000 7fd3c7339700 1 -- 172.21.15.8:0/2542876186 wait complete. 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.184+0000 7fd3c7339700 1 Processor -- start 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.184+0000 7fd3c7339700 1 -- start start 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.184+0000 7fd3c7339700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:43.732 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.184+0000 7fd3c7339700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0199260 con 0x7fd3c0105910 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.185+0000 7fd3c50d5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.185+0000 7fd3c50d5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50526/0 (socket says 172.21.15.8:50526) 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.185+0000 7fd3c50d5700 1 -- 172.21.15.8:0/1101548943 learned_addr learned my addr 172.21.15.8:0/1101548943 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.185+0000 7fd3c50d5700 1 -- 172.21.15.8:0/1101548943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc009550 con 0x7fd3c0105910 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.185+0000 7fd3c50d5700 1 --2- 172.21.15.8:0/1101548943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0198d20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0106640 tx=0x7fd3bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.186+0000 7fd3b67fc700 1 -- 172.21.15.8:0/1101548943 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3bc011630 con 0x7fd3c0105910 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.186+0000 7fd3b67fc700 1 -- 172.21.15.8:0/1101548943 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc011790 con 0x7fd3c0105910 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.186+0000 7fd3c7339700 1 -- 172.21.15.8:0/1101548943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c0106e60 con 0x7fd3c0105910 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.186+0000 7fd3b67fc700 1 -- 172.21.15.8:0/1101548943 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3bc011a30 con 0x7fd3c0105910 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.186+0000 7fd3c7339700 1 -- 172.21.15.8:0/1101548943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c0106cd0 con 0x7fd3c0105910 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.186+0000 7fd3b67fc700 1 -- 172.21.15.8:0/1101548943 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd3bc022070 con 0x7fd3c0105910 2023-11-27T15:15:43.733 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.186+0000 7fd3b67fc700 1 -- 172.21.15.8:0/1101548943 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd3bc01c450 con 0x7fd3c0105910 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.187+0000 7fd3c7339700 1 -- 172.21.15.8:0/1101548943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c0106ff0 con 0x7fd3c0105910 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.190+0000 7fd3b67fc700 1 -- 172.21.15.8:0/1101548943 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fd3bc01b450 con 0x7fd3c0105910 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.266+0000 7fd3c7339700 1 -- 172.21.15.8:0/1101548943 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fd3c002d030 con 0x7fd3c0105910 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.266+0000 7fd3b67fc700 1 -- 172.21.15.8:0/1101548943 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fd3bc01b5f0 con 0x7fd3c0105910 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.267+0000 7fd3c7339700 1 -- 172.21.15.8:0/1101548943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 msgr2=0x7fd3c0198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.267+0000 7fd3c7339700 1 --2- 172.21.15.8:0/1101548943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0198d20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0106640 tx=0x7fd3bc004750 comp rx=0 tx=0).stop 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.267+0000 7fd3c7339700 1 -- 172.21.15.8:0/1101548943 shutdown_connections 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.267+0000 7fd3c7339700 1 --2- 172.21.15.8:0/1101548943 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3c0105910 0x7fd3c0198d20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.267+0000 7fd3c7339700 1 -- 172.21.15.8:0/1101548943 >> 172.21.15.8:0/1101548943 conn(0x7fd3c0100fa0 msgr2=0x7fd3c018d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.267+0000 7fd3c7339700 1 -- 172.21.15.8:0/1101548943 shutdown_connections 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:43.267+0000 7fd3c7339700 1 -- 172.21.15.8:0/1101548943 wait complete. 2023-11-27T15:15:43.734 INFO:teuthology.orchestra.run.smithi008.stdout:mon is available 2023-11-27T15:15:43.735 INFO:teuthology.orchestra.run.smithi008.stdout:Assimilating anything we can from ceph.conf... 2023-11-27T15:15:43.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:43 smithi008 ceph-mon[128284]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-27T15:15:43.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:43 smithi008 ceph-mon[128284]: monmap e1: 1 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0]} removed_ranks: {} 2023-11-27T15:15:43.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:43 smithi008 ceph-mon[128284]: fsmap 2023-11-27T15:15:43.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:43 smithi008 ceph-mon[128284]: osdmap e1: 0 total, 0 up, 0 in 2023-11-27T15:15:43.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:43 smithi008 ceph-mon[128284]: mgrmap e1: no daemons active 2023-11-27T15:15:43.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:43 smithi008 ceph-mon[128284]: from='client.? 172.21.15.8:0/1101548943' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-11-27T15:15:44.710 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:15:44.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [global] 2023-11-27T15:15:44.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout fsid = 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:15:44.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.8:3300,v1:172.21.15.8:6789] 2023-11-27T15:15:44.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-27T15:15:44.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [osd] 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-27T15:15:44.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.270+0000 7f0b87267700 1 Processor -- start 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.270+0000 7f0b87267700 1 -- start start 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.270+0000 7f0b87267700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.270+0000 7f0b87267700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8007a840 con 0x7f0b8007be00 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.271+0000 7f0b85003700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.271+0000 7f0b85003700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50528/0 (socket says 172.21.15.8:50528) 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.271+0000 7f0b85003700 1 -- 172.21.15.8:0/1727657760 learned_addr learned my addr 172.21.15.8:0/1727657760 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.271+0000 7f0b85003700 1 -- 172.21.15.8:0/1727657760 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8007a980 con 0x7f0b8007be00 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.271+0000 7f0b85003700 1 --2- 172.21.15.8:0/1727657760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8007a300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0098a0 tx=0x7f0b7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb30c9b2b380633 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.272+0000 7f0b77fff700 1 -- 172.21.15.8:0/1727657760 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b7c004160 con 0x7f0b8007be00 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.272+0000 7f0b77fff700 1 -- 172.21.15.8:0/1727657760 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c0042c0 con 0x7f0b8007be00 2023-11-27T15:15:44.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.272+0000 7f0b77fff700 1 -- 172.21.15.8:0/1727657760 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b7c0045a0 con 0x7f0b8007be00 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.272+0000 7f0b87267700 1 -- 172.21.15.8:0/1727657760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 msgr2=0x7f0b8007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.272+0000 7f0b87267700 1 --2- 172.21.15.8:0/1727657760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8007a300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0098a0 tx=0x7f0b7c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.272+0000 7f0b87267700 1 -- 172.21.15.8:0/1727657760 shutdown_connections 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.272+0000 7f0b87267700 1 --2- 172.21.15.8:0/1727657760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8007a300 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.272+0000 7f0b87267700 1 -- 172.21.15.8:0/1727657760 >> 172.21.15.8:0/1727657760 conn(0x7f0b80100fa0 msgr2=0x7f0b801033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.272+0000 7f0b87267700 1 -- 172.21.15.8:0/1727657760 shutdown_connections 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.272+0000 7f0b87267700 1 -- 172.21.15.8:0/1727657760 wait complete. 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.273+0000 7f0b87267700 1 Processor -- start 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.274+0000 7f0b87267700 1 -- start start 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.274+0000 7f0b87267700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8019af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:44.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.274+0000 7f0b87267700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8019b490 con 0x7f0b8007be00 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.274+0000 7f0b85003700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8019af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.274+0000 7f0b85003700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8019af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50538/0 (socket says 172.21.15.8:50538) 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.274+0000 7f0b85003700 1 -- 172.21.15.8:0/2406756710 learned_addr learned my addr 172.21.15.8:0/2406756710 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.274+0000 7f0b85003700 1 -- 172.21.15.8:0/2406756710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c009550 con 0x7f0b8007be00 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.275+0000 7f0b85003700 1 --2- 172.21.15.8:0/2406756710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8019af50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c004000 tx=0x7f0b7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.275+0000 7f0b767fc700 1 -- 172.21.15.8:0/2406756710 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b7c011630 con 0x7f0b8007be00 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.275+0000 7f0b767fc700 1 -- 172.21.15.8:0/2406756710 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c011790 con 0x7f0b8007be00 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.275+0000 7f0b767fc700 1 -- 172.21.15.8:0/2406756710 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b7c011a30 con 0x7f0b8007be00 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.275+0000 7f0b87267700 1 -- 172.21.15.8:0/2406756710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8007ae80 con 0x7f0b8007be00 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.275+0000 7f0b87267700 1 -- 172.21.15.8:0/2406756710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8007acf0 con 0x7f0b8007be00 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.275+0000 7f0b767fc700 1 -- 172.21.15.8:0/2406756710 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0b7c022070 con 0x7f0b8007be00 2023-11-27T15:15:44.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.275+0000 7f0b767fc700 1 -- 172.21.15.8:0/2406756710 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0b7c01c450 con 0x7f0b8007be00 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.276+0000 7f0b87267700 1 -- 172.21.15.8:0/2406756710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b80194e80 con 0x7f0b8007be00 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.279+0000 7f0b767fc700 1 -- 172.21.15.8:0/2406756710 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f0b7c01b450 con 0x7f0b8007be00 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.352+0000 7f0b87267700 1 -- 172.21.15.8:0/2406756710 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0b80062120 con 0x7f0b8007be00 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.356+0000 7f0b767fc700 1 -- 172.21.15.8:0/2406756710 <== mon.0 v2:172.21.15.8:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b7c01b5f0 con 0x7f0b8007be00 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.356+0000 7f0b767fc700 1 -- 172.21.15.8:0/2406756710 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7f0b80062120 con 0x7f0b8007be00 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.357+0000 7f0b87267700 1 -- 172.21.15.8:0/2406756710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 msgr2=0x7f0b8019af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.357+0000 7f0b87267700 1 --2- 172.21.15.8:0/2406756710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8019af50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c004000 tx=0x7f0b7c004750 comp rx=0 tx=0).stop 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.357+0000 7f0b87267700 1 -- 172.21.15.8:0/2406756710 shutdown_connections 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.357+0000 7f0b87267700 1 --2- 172.21.15.8:0/2406756710 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b8007be00 0x7f0b8019af50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.357+0000 7f0b87267700 1 -- 172.21.15.8:0/2406756710 >> 172.21.15.8:0/2406756710 conn(0x7f0b80100fa0 msgr2=0x7f0b801027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.357+0000 7f0b87267700 1 -- 172.21.15.8:0/2406756710 shutdown_connections 2023-11-27T15:15:44.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:44.357+0000 7f0b87267700 1 -- 172.21.15.8:0/2406756710 wait complete. 2023-11-27T15:15:44.725 INFO:teuthology.orchestra.run.smithi008.stdout:Generating new minimal ceph.conf... 2023-11-27T15:15:44.785 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:44 smithi008 ceph-mon[128284]: from='client.? 172.21.15.8:0/2406756710' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-27T15:15:44.786 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:44 smithi008 ceph-mon[128284]: from='client.? 172.21.15.8:0/2406756710' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-11-27T15:15:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:45 smithi008 ceph-mon[128284]: from='client.? 172.21.15.8:0/1602544359' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:45.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.236+0000 7ff7ba102700 1 Processor -- start 2023-11-27T15:15:45.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.236+0000 7ff7ba102700 1 -- start start 2023-11-27T15:15:45.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.236+0000 7ff7ba102700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:45.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.236+0000 7ff7ba102700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b407a820 con 0x7ff7b407bde0 2023-11-27T15:15:45.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.236+0000 7ff7b37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:45.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.236+0000 7ff7b37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50546/0 (socket says 172.21.15.8:50546) 2023-11-27T15:15:45.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.237+0000 7ff7b37fe700 1 -- 172.21.15.8:0/2786629157 learned_addr learned my addr 172.21.15.8:0/2786629157 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.237+0000 7ff7b37fe700 1 -- 172.21.15.8:0/2786629157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b407a960 con 0x7ff7b407bde0 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.237+0000 7ff7b37fe700 1 --2- 172.21.15.8:0/2786629157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b407a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40098a0 tx=0x7ff7a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c0d1700aea277f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.237+0000 7ff7b27fc700 1 -- 172.21.15.8:0/2786629157 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7a4004160 con 0x7ff7b407bde0 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.237+0000 7ff7b27fc700 1 -- 172.21.15.8:0/2786629157 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff7a40042c0 con 0x7ff7b407bde0 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.237+0000 7ff7b27fc700 1 -- 172.21.15.8:0/2786629157 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7a4011a00 con 0x7ff7b407bde0 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.238+0000 7ff7ba102700 1 -- 172.21.15.8:0/2786629157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 msgr2=0x7ff7b407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.238+0000 7ff7ba102700 1 --2- 172.21.15.8:0/2786629157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b407a2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40098a0 tx=0x7ff7a4009bb0 comp rx=0 tx=0).stop 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.238+0000 7ff7ba102700 1 -- 172.21.15.8:0/2786629157 shutdown_connections 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.238+0000 7ff7ba102700 1 --2- 172.21.15.8:0/2786629157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b407a2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.238+0000 7ff7ba102700 1 -- 172.21.15.8:0/2786629157 >> 172.21.15.8:0/2786629157 conn(0x7ff7b4100fa0 msgr2=0x7ff7b41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.238+0000 7ff7ba102700 1 -- 172.21.15.8:0/2786629157 shutdown_connections 2023-11-27T15:15:45.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.238+0000 7ff7ba102700 1 -- 172.21.15.8:0/2786629157 wait complete. 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.239+0000 7ff7ba102700 1 Processor -- start 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.239+0000 7ff7ba102700 1 -- start start 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.240+0000 7ff7ba102700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.240+0000 7ff7ba102700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b419bcf0 con 0x7ff7b407bde0 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.240+0000 7ff7b37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.240+0000 7ff7b37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50554/0 (socket says 172.21.15.8:50554) 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.240+0000 7ff7b37fe700 1 -- 172.21.15.8:0/1602544359 learned_addr learned my addr 172.21.15.8:0/1602544359 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.240+0000 7ff7b37fe700 1 -- 172.21.15.8:0/1602544359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4009550 con 0x7ff7b407bde0 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.241+0000 7ff7b37fe700 1 --2- 172.21.15.8:0/1602544359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b419b7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006af0 tx=0x7ff7a4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.241+0000 7ff7b0ff9700 1 -- 172.21.15.8:0/1602544359 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7a4003a40 con 0x7ff7b407bde0 2023-11-27T15:15:45.805 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.241+0000 7ff7b0ff9700 1 -- 172.21.15.8:0/1602544359 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff7a4003ba0 con 0x7ff7b407bde0 2023-11-27T15:15:45.806 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.241+0000 7ff7ba102700 1 -- 172.21.15.8:0/1602544359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b410be00 con 0x7ff7b407bde0 2023-11-27T15:15:45.806 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.241+0000 7ff7b0ff9700 1 -- 172.21.15.8:0/1602544359 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7a401a700 con 0x7ff7b407bde0 2023-11-27T15:15:45.806 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.241+0000 7ff7ba102700 1 -- 172.21.15.8:0/1602544359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b410bc70 con 0x7ff7b407bde0 2023-11-27T15:15:45.806 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.241+0000 7ff7b0ff9700 1 -- 172.21.15.8:0/1602544359 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff7a401ac30 con 0x7ff7b407bde0 2023-11-27T15:15:45.806 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.241+0000 7ff7b0ff9700 1 -- 172.21.15.8:0/1602544359 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff7a4019700 con 0x7ff7b407bde0 2023-11-27T15:15:45.806 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.242+0000 7ff7ba102700 1 -- 172.21.15.8:0/1602544359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b4062120 con 0x7ff7b407bde0 2023-11-27T15:15:45.806 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.245+0000 7ff7b0ff9700 1 -- 172.21.15.8:0/1602544359 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7ff7a402de40 con 0x7ff7b407bde0 2023-11-27T15:15:45.806 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.319+0000 7ff7ba102700 1 -- 172.21.15.8:0/1602544359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff7b419c230 con 0x7ff7b407bde0 2023-11-27T15:15:45.806 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.319+0000 7ff7b0ff9700 1 -- 172.21.15.8:0/1602544359 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7ff7a4024ad0 con 0x7ff7b407bde0 2023-11-27T15:15:45.806 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.320+0000 7ff7ba102700 1 -- 172.21.15.8:0/1602544359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 msgr2=0x7ff7b419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:45.807 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.320+0000 7ff7ba102700 1 --2- 172.21.15.8:0/1602544359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b419b7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006af0 tx=0x7ff7a4011ad0 comp rx=0 tx=0).stop 2023-11-27T15:15:45.807 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.320+0000 7ff7ba102700 1 -- 172.21.15.8:0/1602544359 shutdown_connections 2023-11-27T15:15:45.807 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.320+0000 7ff7ba102700 1 --2- 172.21.15.8:0/1602544359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff7b407bde0 0x7ff7b419b7b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:45.807 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.320+0000 7ff7ba102700 1 -- 172.21.15.8:0/1602544359 >> 172.21.15.8:0/1602544359 conn(0x7ff7b4100fa0 msgr2=0x7ff7b4101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:45.807 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.321+0000 7ff7ba102700 1 -- 172.21.15.8:0/1602544359 shutdown_connections 2023-11-27T15:15:45.807 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:45.321+0000 7ff7ba102700 1 -- 172.21.15.8:0/1602544359 wait complete. 2023-11-27T15:15:45.807 INFO:teuthology.orchestra.run.smithi008.stdout:Restarting the monitor... 2023-11-27T15:15:46.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:45 smithi008 systemd[1]: Stopping Ceph mon.a for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:15:46.602 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:46 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-a[128260]: 2023-11-27T15:15:46.269+0000 7f0d9ba6e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:15:46.602 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:46 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-a[128260]: 2023-11-27T15:15:46.269+0000 7f0d9ba6e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-27T15:15:46.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:46 smithi008 bash[128625]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-a 2023-11-27T15:15:47.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:46 smithi008 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.a.service: Succeeded. 2023-11-27T15:15:47.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:46 smithi008 systemd[1]: Stopped Ceph mon.a for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:15:47.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:46 smithi008 systemd[1]: Starting Ceph mon.a for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:15:47.672 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 podman[128748]: 2023-11-27T15:15:47.672 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 bash[128748]: a152f348fbabe883a5901fab582e0e6b37ed4131ed288b93b14adf55a0b0e296 2023-11-27T15:15:47.672 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: set uid:gid to 167:167 (ceph:ceph) 2023-11-27T15:15:47.693 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-27T15:15:47.693 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: pidfile_write: ignore empty --pid-file 2023-11-27T15:15:47.693 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: load: jerasure load: lrc 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: RocksDB version: 7.9.2 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Git sha 0 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: DB SUMMARY 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: DB Session ID: 62XFZCOROEQU8JR34JZF 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: CURRENT file: CURRENT 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: IDENTITY file: IDENTITY 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 97690 ; 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.error_if_exists: 0 2023-11-27T15:15:47.694 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.create_if_missing: 0 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.paranoid_checks: 1 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.env: 0x55b88f1ae720 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.fs: PosixFileSystem 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.info_log: 0x55b890dbf3c0 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_file_opening_threads: 16 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.statistics: (nil) 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.use_fsync: 0 2023-11-27T15:15:47.695 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_log_file_size: 0 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.keep_log_file_num: 1000 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.recycle_log_file_num: 0 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.allow_fallocate: 1 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.allow_mmap_reads: 0 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.allow_mmap_writes: 0 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.use_direct_reads: 0 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.create_missing_column_families: 0 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.db_log_dir: 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.wal_dir: 2023-11-27T15:15:47.696 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.advise_random_on_open: 1 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.db_write_buffer_size: 0 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.write_buffer_manager: 0x55b8900303c0 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-27T15:15:47.697 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.rate_limiter: (nil) 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.wal_recovery_mode: 2 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.enable_thread_tracking: 0 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.enable_pipelined_write: 0 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.unordered_write: 0 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.row_cache: None 2023-11-27T15:15:47.698 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.wal_filter: None 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.allow_ingest_behind: 0 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.two_write_queues: 0 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.manual_wal_flush: 0 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.wal_compression: 0 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.atomic_flush: 0 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.log_readahead_size: 0 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-27T15:15:47.699 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.best_efforts_recovery: 0 2023-11-27T15:15:47.700 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-27T15:15:47.700 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-27T15:15:47.700 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.allow_data_in_errors: 0 2023-11-27T15:15:47.700 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.db_host_id: __hostname__ 2023-11-27T15:15:47.700 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-27T15:15:47.701 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_background_jobs: 2 2023-11-27T15:15:47.701 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_background_compactions: -1 2023-11-27T15:15:47.701 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_subcompactions: 1 2023-11-27T15:15:47.701 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-27T15:15:47.701 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-27T15:15:47.701 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-27T15:15:47.701 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_total_wal_size: 0 2023-11-27T15:15:47.701 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-27T15:15:47.701 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-27T15:15:47.701 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_open_files: -1 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bytes_per_sync: 0 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_readahead_size: 0 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_background_flushes: -1 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Compression algorithms supported: 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: kZSTD supported: 0 2023-11-27T15:15:47.702 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: kXpressCompression supported: 0 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: kLZ4HCCompression supported: 1 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: kZlibCompression supported: 1 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: kSnappyCompression supported: 1 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: kLZ4Compression supported: 1 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: kBZip2Compression supported: 0 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.merge_operator: 2023-11-27T15:15:47.703 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_filter: None 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_filter_factory: None 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.sst_partitioner_factory: None 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b890dbf4e0) 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout: cache_index_and_filter_blocks: 1 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout: pin_top_level_index_and_filter: 1 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout: index_type: 0 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout: data_block_index_type: 0 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout: index_shortening: 1 2023-11-27T15:15:47.704 INFO:journalctl@ceph.mon.a.smithi008.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: checksum: 4 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: no_block_cache: 0 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_cache: 0x55b8900d11f0 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_cache_name: BinnedLRUCache 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_cache_options: 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: capacity : 536870912 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: num_shard_bits : 4 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: strict_capacity_limit : 0 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: high_pri_pool_ratio: 0.000 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_cache_compressed: (nil) 2023-11-27T15:15:47.705 INFO:journalctl@ceph.mon.a.smithi008.stdout: persistent_cache: (nil) 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_size: 4096 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_size_deviation: 10 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_restart_interval: 16 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: index_block_restart_interval: 1 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: metadata_block_size: 4096 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: partition_filters: 0 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: use_delta_encoding: 1 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: filter_policy: bloomfilter 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: whole_key_filtering: 1 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: verify_compression: 0 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: read_amp_bytes_per_bit: 0 2023-11-27T15:15:47.706 INFO:journalctl@ceph.mon.a.smithi008.stdout: format_version: 5 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout: enable_index_compression: 1 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout: block_align: 0 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout: max_auto_readahead_size: 262144 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout: prepopulate_block_cache: 0 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout: initial_auto_readahead_size: 8192 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout: num_file_reads_for_auto_readahead: 2 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.write_buffer_size: 33554432 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_write_buffer_number: 2 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compression: NoCompression 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bottommost_compression: Disabled 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.prefix_extractor: nullptr 2023-11-27T15:15:47.707 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.num_levels: 7 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-27T15:15:47.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compression_opts.level: 32767 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compression_opts.strategy: 0 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compression_opts.enabled: false 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-27T15:15:47.709 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.target_file_size_base: 67108864 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-27T15:15:47.710 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.arena_block_size: 1048576 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.disable_auto_compactions: 0 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-27T15:15:47.711 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.table_properties_collectors: 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.inplace_update_support: 0 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-27T15:15:47.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-27T15:15:47.713 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.bloom_locality: 0 2023-11-27T15:15:47.713 INFO:teuthology.orchestra.run.smithi008.stdout:Setting mon public_network to 172.21.0.0/20 2023-11-27T15:15:47.965 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.max_successive_merges: 0 2023-11-27T15:15:47.965 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-27T15:15:47.965 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.paranoid_file_checks: 0 2023-11-27T15:15:47.965 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.force_consistency_checks: 1 2023-11-27T15:15:47.965 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.report_bg_io_stats: 0 2023-11-27T15:15:47.965 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.ttl: 2592000 2023-11-27T15:15:47.965 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-27T15:15:47.965 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-27T15:15:47.965 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-27T15:15:47.965 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.enable_blob_files: false 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.min_blob_size: 0 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.blob_file_size: 268435456 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.blob_file_starting_level: 0 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-11-27T15:15:47.966 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7cc4721b-8e17-4f15-9ce2-c1854016ddaa 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098147677429, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098147679095, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92998, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 91020, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13416, "raw_average_key_size": 49, "raw_value_size": 83712, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 269, "num_filter_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701098147, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7cc4721b-8e17-4f15-9ce2-c1854016ddaa", "db_session_id": "62XFZCOROEQU8JR34JZF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098147679227, "job": 1, "event": "recovery_finished"} 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b89016e000 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: DB pointer 0x55b89015a000 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-27T15:15:47.967 INFO:journalctl@ceph.mon.a.smithi008.stdout: ** DB Stats ** 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: ** Compaction Stats [default] ** 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:15:47.968 INFO:journalctl@ceph.mon.a.smithi008.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: L0 2/0 92.65 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: Sum 2/0 92.65 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: ** Compaction Stats [default] ** 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 82.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2023-11-27T15:15:47.969 INFO:journalctl@ceph.mon.a.smithi008.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: Cumulative compaction: 0.00 GB write, 14.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: Interval compaction: 0.00 GB write, 14.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: Block cache BinnedLRUCache@0x55b8900d11f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-11-27T15:15:47.970 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: starting mon.a rank 0 at public addrs [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] at bind addrs [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: mon.a@-1(???) e1 preinit fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: mon.a@-1(???).mds e1 new map 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: mon.a@-1(???).mds e1 print_map 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout: e1 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout: legacy client fscid: -1 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout: 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout: No filesystems configured 2023-11-27T15:15:47.971 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-27T15:15:47.972 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:15:47.972 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:15:47.972 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:15:47.972 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-11-27T15:15:47.972 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 systemd[1]: Started Ceph mon.a for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:15:47.972 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-27T15:15:47.972 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: monmap e1: 1 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0]} removed_ranks: {} 2023-11-27T15:15:47.972 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: fsmap 2023-11-27T15:15:47.972 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: osdmap e1: 0 total, 0 up, 0 in 2023-11-27T15:15:47.972 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:47 smithi008 ceph-mon[128788]: mgrmap e1: no daemons active 2023-11-27T15:15:48.796 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.233+0000 7f0da1ebb700 1 Processor -- start 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.233+0000 7f0da1ebb700 1 -- start start 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.233+0000 7f0da1ebb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.233+0000 7f0da1ebb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c07a840 con 0x7f0d9c07be00 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.234+0000 7f0d9b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.234+0000 7f0d9b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50566/0 (socket says 172.21.15.8:50566) 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.234+0000 7f0d9b7fe700 1 -- 172.21.15.8:0/2826347960 learned_addr learned my addr 172.21.15.8:0/2826347960 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.234+0000 7f0d9b7fe700 1 -- 172.21.15.8:0/2826347960 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c07a980 con 0x7f0d9c07be00 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.235+0000 7f0d9b7fe700 1 --2- 172.21.15.8:0/2826347960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c07a300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0098a0 tx=0x7f0d8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883af1438a819f4b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.235+0000 7f0d9a7fc700 1 -- 172.21.15.8:0/2826347960 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c004160 con 0x7f0d9c07be00 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.235+0000 7f0d9a7fc700 1 -- 172.21.15.8:0/2826347960 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0d8c0042c0 con 0x7f0d9c07be00 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.235+0000 7f0d9a7fc700 1 -- 172.21.15.8:0/2826347960 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c011a00 con 0x7f0d9c07be00 2023-11-27T15:15:48.801 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.236+0000 7f0da1ebb700 1 -- 172.21.15.8:0/2826347960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 msgr2=0x7f0d9c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.236+0000 7f0da1ebb700 1 --2- 172.21.15.8:0/2826347960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c07a300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0098a0 tx=0x7f0d8c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.236+0000 7f0da1ebb700 1 -- 172.21.15.8:0/2826347960 shutdown_connections 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.236+0000 7f0da1ebb700 1 --2- 172.21.15.8:0/2826347960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c07a300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.236+0000 7f0da1ebb700 1 -- 172.21.15.8:0/2826347960 >> 172.21.15.8:0/2826347960 conn(0x7f0d9c100fa0 msgr2=0x7f0d9c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.236+0000 7f0da1ebb700 1 -- 172.21.15.8:0/2826347960 shutdown_connections 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.236+0000 7f0da1ebb700 1 -- 172.21.15.8:0/2826347960 wait complete. 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.237+0000 7f0da1ebb700 1 Processor -- start 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.237+0000 7f0da1ebb700 1 -- start start 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.237+0000 7f0da1ebb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.237+0000 7f0da1ebb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c19bce0 con 0x7f0d9c07be00 2023-11-27T15:15:48.802 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.238+0000 7f0d9b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.238+0000 7f0d9b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c19b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50568/0 (socket says 172.21.15.8:50568) 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.238+0000 7f0d9b7fe700 1 -- 172.21.15.8:0/1421285498 learned_addr learned my addr 172.21.15.8:0/1421285498 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.238+0000 7f0d9b7fe700 1 -- 172.21.15.8:0/1421285498 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c009550 con 0x7f0d9c07be00 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.238+0000 7f0d9b7fe700 1 --2- 172.21.15.8:0/1421285498 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c19b7a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c006b20 tx=0x7f0d8c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.238+0000 7f0d98ff9700 1 -- 172.21.15.8:0/1421285498 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c003b70 con 0x7f0d9c07be00 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.238+0000 7f0d98ff9700 1 -- 172.21.15.8:0/1421285498 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0d8c003cd0 con 0x7f0d9c07be00 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.239+0000 7f0da1ebb700 1 -- 172.21.15.8:0/1421285498 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9c10bde0 con 0x7f0d9c07be00 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.239+0000 7f0d98ff9700 1 -- 172.21.15.8:0/1421285498 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c0247c0 con 0x7f0d9c07be00 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.239+0000 7f0da1ebb700 1 -- 172.21.15.8:0/1421285498 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9c10bc50 con 0x7f0d9c07be00 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.239+0000 7f0d98ff9700 1 -- 172.21.15.8:0/1421285498 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0d8c024d30 con 0x7f0d9c07be00 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.239+0000 7f0d98ff9700 1 -- 172.21.15.8:0/1421285498 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0d8c0193c0 con 0x7f0d9c07be00 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.240+0000 7f0da1ebb700 1 -- 172.21.15.8:0/1421285498 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9c062120 con 0x7f0d9c07be00 2023-11-27T15:15:48.803 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.243+0000 7f0d98ff9700 1 -- 172.21.15.8:0/1421285498 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f0d8c01ab40 con 0x7f0d9c07be00 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.315+0000 7f0da1ebb700 1 -- 172.21.15.8:0/1421285498 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f0d9c19c220 con 0x7f0d9c07be00 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.318+0000 7f0d98ff9700 1 -- 172.21.15.8:0/1421285498 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0d8c01ace0 con 0x7f0d9c07be00 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.319+0000 7f0da1ebb700 1 -- 172.21.15.8:0/1421285498 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 msgr2=0x7f0d9c19b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.319+0000 7f0da1ebb700 1 --2- 172.21.15.8:0/1421285498 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c19b7a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c006b20 tx=0x7f0d8c011ad0 comp rx=0 tx=0).stop 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.319+0000 7f0da1ebb700 1 -- 172.21.15.8:0/1421285498 shutdown_connections 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.319+0000 7f0da1ebb700 1 --2- 172.21.15.8:0/1421285498 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d9c07be00 0x7f0d9c19b7a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.319+0000 7f0da1ebb700 1 -- 172.21.15.8:0/1421285498 >> 172.21.15.8:0/1421285498 conn(0x7f0d9c100fa0 msgr2=0x7f0d9c101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.319+0000 7f0da1ebb700 1 -- 172.21.15.8:0/1421285498 shutdown_connections 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:48.319+0000 7f0da1ebb700 1 -- 172.21.15.8:0/1421285498 wait complete. 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:Wrote config to /etc/ceph/ceph.conf 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:15:48.804 INFO:teuthology.orchestra.run.smithi008.stdout:Creating mgr... 2023-11-27T15:15:48.805 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying port 9283 ... 2023-11-27T15:15:48.805 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying port 8765 ... 2023-11-27T15:15:49.098 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.x 2023-11-27T15:15:49.099 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.x.service: Unit ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.x.service not loaded. 2023-11-27T15:15:49.305 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9f287ba8-8d37-11ee-95a2-87774f69a715.target.wants/ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@.service. 2023-11-27T15:15:49.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:49 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1421285498' entity='client.admin' 2023-11-27T15:15:49.364 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:49 smithi008 systemd[1]: Starting Ceph mgr.x for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:15:50.109 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:49 smithi008 podman[129021]: 2023-11-27T15:15:50.109 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:50 smithi008 bash[129021]: 33e28ad2124ef01816a8819af979a0b2775028e7723a7f106030b3acba6fd21f 2023-11-27T15:15:50.133 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:15:50.133 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2023-11-27T15:15:50.150 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:15:50.150 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2023-11-27T15:15:50.150 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2023-11-27T15:15:50.150 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to enable service . firewalld.service is not available 2023-11-27T15:15:50.168 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:15:50.168 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2023-11-27T15:15:50.184 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:15:50.185 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2023-11-27T15:15:50.185 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2023-11-27T15:15:50.185 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-11-27T15:15:50.185 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr to start... 2023-11-27T15:15:50.185 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr... 2023-11-27T15:15:50.364 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:50 smithi008 systemd[1]: Started Ceph mgr.x for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:15:50.364 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:50 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:50.231+0000 7f8387010200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:15:50.364 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:50 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:50.348+0000 7f8387010200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:15:50.848 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:50 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:50.472+0000 7f8387010200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:15:50.848 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:50 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:50.604+0000 7f8387010200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:15:50.848 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:50 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:50.843+0000 7f8387010200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:15:51.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:50 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2643895806' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:15:51.269 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:15:51.272 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:15:51.272 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:15:51.272 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:15:51.272 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:15:51.272 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:15:51.272 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:15:51.272 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:51.272 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:15:51.272 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:15:51.273 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:51.274 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:15:51.274 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:15:51.281 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:15:51.281 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:15:51.281 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:15:51.281 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:15:51.281 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:51.281 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:15:51.281 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:15:51.282 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:51.283 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:15:42.569758+0000", 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.763+0000 7f39291ae700 1 Processor -- start 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.763+0000 7f39291ae700 1 -- start start 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.764+0000 7f39291ae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.764+0000 7f39291ae700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392407a800 con 0x7f392407bdc0 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.764+0000 7f3922d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.764+0000 7f3922d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49408/0 (socket says 172.21.15.8:49408) 2023-11-27T15:15:51.284 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.764+0000 7f3922d9d700 1 -- 172.21.15.8:0/3435072834 learned_addr learned my addr 172.21.15.8:0/3435072834 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.765+0000 7f3922d9d700 1 -- 172.21.15.8:0/3435072834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392407a940 con 0x7f392407bdc0 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.765+0000 7f3922d9d700 1 --2- 172.21.15.8:0/3435072834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392407a2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f39140098a0 tx=0x7f3914009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5ff53a9a3c2be4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.765+0000 7f3921d9b700 1 -- 172.21.15.8:0/3435072834 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3914004160 con 0x7f392407bdc0 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.765+0000 7f3921d9b700 1 -- 172.21.15.8:0/3435072834 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f39140042c0 con 0x7f392407bdc0 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.765+0000 7f3921d9b700 1 -- 172.21.15.8:0/3435072834 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3914011a00 con 0x7f392407bdc0 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.766+0000 7f39291ae700 1 -- 172.21.15.8:0/3435072834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 msgr2=0x7f392407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.766+0000 7f39291ae700 1 --2- 172.21.15.8:0/3435072834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392407a2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f39140098a0 tx=0x7f3914009bb0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.766+0000 7f39291ae700 1 -- 172.21.15.8:0/3435072834 shutdown_connections 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.766+0000 7f39291ae700 1 --2- 172.21.15.8:0/3435072834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392407a2c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.766+0000 7f39291ae700 1 -- 172.21.15.8:0/3435072834 >> 172.21.15.8:0/3435072834 conn(0x7f3924100fa0 msgr2=0x7f39241033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:51.285 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.766+0000 7f39291ae700 1 -- 172.21.15.8:0/3435072834 shutdown_connections 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.766+0000 7f39291ae700 1 -- 172.21.15.8:0/3435072834 wait complete. 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.767+0000 7f39291ae700 1 Processor -- start 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.767+0000 7f39291ae700 1 -- start start 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.768+0000 7f39291ae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.768+0000 7f39291ae700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392419bcf0 con 0x7f392407bdc0 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.768+0000 7f3922d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.768+0000 7f3922d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49410/0 (socket says 172.21.15.8:49410) 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.768+0000 7f3922d9d700 1 -- 172.21.15.8:0/2643895806 learned_addr learned my addr 172.21.15.8:0/2643895806 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.768+0000 7f3922d9d700 1 -- 172.21.15.8:0/2643895806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3914009550 con 0x7f392407bdc0 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.768+0000 7f3922d9d700 1 --2- 172.21.15.8:0/2643895806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392419b7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3914006af0 tx=0x7f3914011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.769+0000 7f390bfff700 1 -- 172.21.15.8:0/2643895806 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3914003a40 con 0x7f392407bdc0 2023-11-27T15:15:51.286 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.769+0000 7f390bfff700 1 -- 172.21.15.8:0/2643895806 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3914003ba0 con 0x7f392407bdc0 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.769+0000 7f39291ae700 1 -- 172.21.15.8:0/2643895806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392410bdc0 con 0x7f392407bdc0 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.769+0000 7f390bfff700 1 -- 172.21.15.8:0/2643895806 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f391401a700 con 0x7f392407bdc0 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.769+0000 7f39291ae700 1 -- 172.21.15.8:0/2643895806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392410bc30 con 0x7f392407bdc0 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.769+0000 7f390bfff700 1 -- 172.21.15.8:0/2643895806 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f391401ac30 con 0x7f392407bdc0 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.769+0000 7f390bfff700 1 -- 172.21.15.8:0/2643895806 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3914019700 con 0x7f392407bdc0 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.770+0000 7f39291ae700 1 -- 172.21.15.8:0/2643895806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392410bf50 con 0x7f392407bdc0 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.773+0000 7f390bfff700 1 -- 172.21.15.8:0/2643895806 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f391402de40 con 0x7f392407bdc0 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.850+0000 7f39291ae700 1 -- 172.21.15.8:0/2643895806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f392419c2f0 con 0x7f392407bdc0 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.850+0000 7f390bfff700 1 -- 172.21.15.8:0/2643895806 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f3914024ad0 con 0x7f392407bdc0 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.851+0000 7f39291ae700 1 -- 172.21.15.8:0/2643895806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 msgr2=0x7f392419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.851+0000 7f39291ae700 1 --2- 172.21.15.8:0/2643895806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392419b7b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3914006af0 tx=0x7f3914011ad0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.851+0000 7f39291ae700 1 -- 172.21.15.8:0/2643895806 shutdown_connections 2023-11-27T15:15:51.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.851+0000 7f39291ae700 1 --2- 172.21.15.8:0/2643895806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f392407bdc0 0x7f392419b7b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:51.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.851+0000 7f39291ae700 1 -- 172.21.15.8:0/2643895806 >> 172.21.15.8:0/2643895806 conn(0x7f3924100fa0 msgr2=0x7f3924101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:51.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.851+0000 7f39291ae700 1 -- 172.21.15.8:0/2643895806 shutdown_connections 2023-11-27T15:15:51.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:50.851+0000 7f39291ae700 1 -- 172.21.15.8:0/2643895806 wait complete. 2023-11-27T15:15:51.288 INFO:teuthology.orchestra.run.smithi008.stdout:mgr not available, waiting (1/15)... 2023-11-27T15:15:51.596 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:51 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:51.295+0000 7f8387010200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:15:51.596 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:51 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:51.416+0000 7f8387010200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:15:51.863 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:51 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:51.591+0000 7f8387010200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:15:51.864 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:51 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:51.721+0000 7f8387010200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:15:52.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:52 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:52.306+0000 7f8387010200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:15:53.568 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:53 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:53.138+0000 7f8387010200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:15:53.569 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:53 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:53.356+0000 7f8387010200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:15:54.352 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:53 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/898290834' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:15:54.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:15:54.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:15:54.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:15:54.364 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:15:54.365 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-11-27T15:15:54.366 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:15:54.366 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:54.366 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:15:54.366 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:15:54.366 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:54.366 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:15:54.366 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:54.366 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:15:54.366 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:15:54.374 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:15:54.374 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:15:54.374 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:15:54.374 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:15:54.374 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:54.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:15:54.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:15:42.569758+0000", 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.834+0000 7faf6632f700 1 Processor -- start 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.834+0000 7faf6632f700 1 -- start start 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.835+0000 7faf6632f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.835+0000 7faf6632f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6007a820 con 0x7faf6007bde0 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.835+0000 7faf5ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.835+0000 7faf5ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49424/0 (socket says 172.21.15.8:49424) 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.835+0000 7faf5ffff700 1 -- 172.21.15.8:0/4232743028 learned_addr learned my addr 172.21.15.8:0/4232743028 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.836+0000 7faf5ffff700 1 -- 172.21.15.8:0/4232743028 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6007a960 con 0x7faf6007bde0 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.836+0000 7faf5ffff700 1 --2- 172.21.15.8:0/4232743028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6007a2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf500098a0 tx=0x7faf50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19be8c9a73081038 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.836+0000 7faf5effd700 1 -- 172.21.15.8:0/4232743028 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf50004160 con 0x7faf6007bde0 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.836+0000 7faf5effd700 1 -- 172.21.15.8:0/4232743028 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf500042c0 con 0x7faf6007bde0 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.836+0000 7faf5effd700 1 -- 172.21.15.8:0/4232743028 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf50011a00 con 0x7faf6007bde0 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.837+0000 7faf6632f700 1 -- 172.21.15.8:0/4232743028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 msgr2=0x7faf6007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.837+0000 7faf6632f700 1 --2- 172.21.15.8:0/4232743028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6007a2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf500098a0 tx=0x7faf50009bb0 comp rx=0 tx=0).stop 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.837+0000 7faf6632f700 1 -- 172.21.15.8:0/4232743028 shutdown_connections 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.837+0000 7faf6632f700 1 --2- 172.21.15.8:0/4232743028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6007a2e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:54.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.837+0000 7faf6632f700 1 -- 172.21.15.8:0/4232743028 >> 172.21.15.8:0/4232743028 conn(0x7faf60100fa0 msgr2=0x7faf601033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.837+0000 7faf6632f700 1 -- 172.21.15.8:0/4232743028 shutdown_connections 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.837+0000 7faf6632f700 1 -- 172.21.15.8:0/4232743028 wait complete. 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.838+0000 7faf6632f700 1 Processor -- start 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.838+0000 7faf6632f700 1 -- start start 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.839+0000 7faf6632f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.839+0000 7faf6632f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6019bcf0 con 0x7faf6007bde0 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.839+0000 7faf5ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.839+0000 7faf5ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6019b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49436/0 (socket says 172.21.15.8:49436) 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.839+0000 7faf5ffff700 1 -- 172.21.15.8:0/898290834 learned_addr learned my addr 172.21.15.8:0/898290834 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.839+0000 7faf5ffff700 1 -- 172.21.15.8:0/898290834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf50009550 con 0x7faf6007bde0 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.840+0000 7faf5ffff700 1 --2- 172.21.15.8:0/898290834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6019b7b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf50006af0 tx=0x7faf50011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:54.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.840+0000 7faf5d7fa700 1 -- 172.21.15.8:0/898290834 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf50003a40 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.840+0000 7faf5d7fa700 1 -- 172.21.15.8:0/898290834 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf50003ba0 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.840+0000 7faf6632f700 1 -- 172.21.15.8:0/898290834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6010be00 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.840+0000 7faf5d7fa700 1 -- 172.21.15.8:0/898290834 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf5001a700 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.840+0000 7faf6632f700 1 -- 172.21.15.8:0/898290834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6010bc70 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.840+0000 7faf5d7fa700 1 -- 172.21.15.8:0/898290834 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faf5001ac30 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.840+0000 7faf5d7fa700 1 -- 172.21.15.8:0/898290834 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faf50019700 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.841+0000 7faf6632f700 1 -- 172.21.15.8:0/898290834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6010bf90 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.844+0000 7faf5d7fa700 1 -- 172.21.15.8:0/898290834 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7faf50024ad0 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.922+0000 7faf6632f700 1 -- 172.21.15.8:0/898290834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7faf6019c2f0 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.922+0000 7faf5d7fa700 1 -- 172.21.15.8:0/898290834 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7faf50024c70 con 0x7faf6007bde0 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.923+0000 7faf6632f700 1 -- 172.21.15.8:0/898290834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 msgr2=0x7faf6019b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:54.380 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.923+0000 7faf6632f700 1 --2- 172.21.15.8:0/898290834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6019b7b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf50006af0 tx=0x7faf50011ad0 comp rx=0 tx=0).stop 2023-11-27T15:15:54.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.923+0000 7faf6632f700 1 -- 172.21.15.8:0/898290834 shutdown_connections 2023-11-27T15:15:54.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.923+0000 7faf6632f700 1 --2- 172.21.15.8:0/898290834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf6007bde0 0x7faf6019b7b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:54.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.923+0000 7faf6632f700 1 -- 172.21.15.8:0/898290834 >> 172.21.15.8:0/898290834 conn(0x7faf60100fa0 msgr2=0x7faf60101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:54.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.923+0000 7faf6632f700 1 -- 172.21.15.8:0/898290834 shutdown_connections 2023-11-27T15:15:54.381 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:53.923+0000 7faf6632f700 1 -- 172.21.15.8:0/898290834 wait complete. 2023-11-27T15:15:54.381 INFO:teuthology.orchestra.run.smithi008.stdout:mgr not available, waiting (2/15)... 2023-11-27T15:15:54.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:54 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:54.405+0000 7f8387010200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:15:54.614 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:54 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:54.531+0000 7f8387010200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:15:55.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:54 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:54.765+0000 7f8387010200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:15:55.434 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:55 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:55.145+0000 7f8387010200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:15:55.863 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:55 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:55.430+0000 7f8387010200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:15:55.863 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:55 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:55.632+0000 7f8387010200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:15:56.363 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:55 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:55.994+0000 7f8387010200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:15:57.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:57 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3164406692' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:15:57.364 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:57 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:57.019+0000 7f8387010200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:15:57.364 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:57 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:57.224+0000 7f8387010200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:15:57.453 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:15:57.453 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:15:57.453 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:15:57.453 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:15:57.453 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:15:57.454 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:15:57.454 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:15:57.454 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:57.454 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:15:57.454 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:15:57.454 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:15:57.454 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:15:57.454 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:15:57.454 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:15:57.455 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:15:57.455 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-11-27T15:15:57.455 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:15:57.455 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:57.455 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:15:57.455 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:15:57.455 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:57.455 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:15:57.455 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:57.455 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:15:57.462 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:15:57.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:15:57.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:15:57.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:15:57.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:15:57.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:57.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:15:57.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:15:57.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:15:57.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:15:57.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:15:57.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:15:57.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:15:57.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:15:57.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:15:57.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:57.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:15:57.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:57.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:15:57.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:15:57.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:57.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:15:57.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:15:42.569758+0000", 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.935+0000 7fd829d36700 1 Processor -- start 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.935+0000 7fd829d36700 1 -- start start 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.936+0000 7fd829d36700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.936+0000 7fd829d36700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82407a820 con 0x7fd82407bde0 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.936+0000 7fd8237fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.936+0000 7fd8237fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49438/0 (socket says 172.21.15.8:49438) 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.936+0000 7fd8237fe700 1 -- 172.21.15.8:0/2840719271 learned_addr learned my addr 172.21.15.8:0/2840719271 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.937+0000 7fd8237fe700 1 -- 172.21.15.8:0/2840719271 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82407a960 con 0x7fd82407bde0 2023-11-27T15:15:57.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.937+0000 7fd8237fe700 1 --2- 172.21.15.8:0/2840719271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82407a2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8140098a0 tx=0x7fd814009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0417679b28fd1ba server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.937+0000 7fd8227fc700 1 -- 172.21.15.8:0/2840719271 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd814004160 con 0x7fd82407bde0 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.937+0000 7fd8227fc700 1 -- 172.21.15.8:0/2840719271 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd8140042c0 con 0x7fd82407bde0 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.937+0000 7fd8227fc700 1 -- 172.21.15.8:0/2840719271 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd814011a00 con 0x7fd82407bde0 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.938+0000 7fd829d36700 1 -- 172.21.15.8:0/2840719271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 msgr2=0x7fd82407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.938+0000 7fd829d36700 1 --2- 172.21.15.8:0/2840719271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82407a2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8140098a0 tx=0x7fd814009bb0 comp rx=0 tx=0).stop 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.938+0000 7fd829d36700 1 -- 172.21.15.8:0/2840719271 shutdown_connections 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.938+0000 7fd829d36700 1 --2- 172.21.15.8:0/2840719271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82407a2e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.938+0000 7fd829d36700 1 -- 172.21.15.8:0/2840719271 >> 172.21.15.8:0/2840719271 conn(0x7fd824100fa0 msgr2=0x7fd8241033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.938+0000 7fd829d36700 1 -- 172.21.15.8:0/2840719271 shutdown_connections 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.938+0000 7fd829d36700 1 -- 172.21.15.8:0/2840719271 wait complete. 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.939+0000 7fd829d36700 1 Processor -- start 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.939+0000 7fd829d36700 1 -- start start 2023-11-27T15:15:57.468 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.940+0000 7fd829d36700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.940+0000 7fd829d36700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82419bcf0 con 0x7fd82407bde0 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.940+0000 7fd8237fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.940+0000 7fd8237fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49440/0 (socket says 172.21.15.8:49440) 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.940+0000 7fd8237fe700 1 -- 172.21.15.8:0/3164406692 learned_addr learned my addr 172.21.15.8:0/3164406692 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.940+0000 7fd8237fe700 1 -- 172.21.15.8:0/3164406692 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd814009550 con 0x7fd82407bde0 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.940+0000 7fd8237fe700 1 --2- 172.21.15.8:0/3164406692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82419b7b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd814006af0 tx=0x7fd814011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.941+0000 7fd820ff9700 1 -- 172.21.15.8:0/3164406692 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd814003a40 con 0x7fd82407bde0 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.941+0000 7fd820ff9700 1 -- 172.21.15.8:0/3164406692 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd814003ba0 con 0x7fd82407bde0 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.941+0000 7fd820ff9700 1 -- 172.21.15.8:0/3164406692 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd81401a700 con 0x7fd82407bde0 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.941+0000 7fd829d36700 1 -- 172.21.15.8:0/3164406692 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82410be00 con 0x7fd82407bde0 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.941+0000 7fd829d36700 1 -- 172.21.15.8:0/3164406692 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82410bc70 con 0x7fd82407bde0 2023-11-27T15:15:57.469 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.941+0000 7fd820ff9700 1 -- 172.21.15.8:0/3164406692 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd81401ac30 con 0x7fd82407bde0 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.941+0000 7fd820ff9700 1 -- 172.21.15.8:0/3164406692 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd814019700 con 0x7fd82407bde0 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.942+0000 7fd829d36700 1 -- 172.21.15.8:0/3164406692 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd824062120 con 0x7fd82407bde0 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:56.946+0000 7fd820ff9700 1 -- 172.21.15.8:0/3164406692 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fd81402de40 con 0x7fd82407bde0 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:57.023+0000 7fd829d36700 1 -- 172.21.15.8:0/3164406692 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd82419c230 con 0x7fd82407bde0 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:57.024+0000 7fd820ff9700 1 -- 172.21.15.8:0/3164406692 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fd814024ad0 con 0x7fd82407bde0 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:57.025+0000 7fd829d36700 1 -- 172.21.15.8:0/3164406692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 msgr2=0x7fd82419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:57.025+0000 7fd829d36700 1 --2- 172.21.15.8:0/3164406692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82419b7b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd814006af0 tx=0x7fd814011ad0 comp rx=0 tx=0).stop 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:57.025+0000 7fd829d36700 1 -- 172.21.15.8:0/3164406692 shutdown_connections 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:57.025+0000 7fd829d36700 1 --2- 172.21.15.8:0/3164406692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd82407bde0 0x7fd82419b7b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:57.025+0000 7fd829d36700 1 -- 172.21.15.8:0/3164406692 >> 172.21.15.8:0/3164406692 conn(0x7fd824100fa0 msgr2=0x7fd824101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:57.025+0000 7fd829d36700 1 -- 172.21.15.8:0/3164406692 shutdown_connections 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:57.025+0000 7fd829d36700 1 -- 172.21.15.8:0/3164406692 wait complete. 2023-11-27T15:15:57.470 INFO:teuthology.orchestra.run.smithi008.stdout:mgr not available, waiting (3/15)... 2023-11-27T15:15:57.863 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:57 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:57.488+0000 7f8387010200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:15:57.864 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:57 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:57.645+0000 7f8387010200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:15:58.363 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:57 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:57.934+0000 7f8387010200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:15:59.363 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:15:59.050+0000 7f8387010200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: Activating manager daemon x 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: mgrmap e2: x(active, starting, since 0.00450618s) 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: from='mgr.14100 172.21.15.8:0/2802333281' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: from='mgr.14100 172.21.15.8:0/2802333281' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: from='mgr.14100 172.21.15.8:0/2802333281' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: from='mgr.14100 172.21.15.8:0/2802333281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: from='mgr.14100 172.21.15.8:0/2802333281' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: Manager daemon x is now available 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: from='mgr.14100 172.21.15.8:0/2802333281' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: from='mgr.14100 172.21.15.8:0/2802333281' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:15:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: from='mgr.14100 172.21.15.8:0/2802333281' entity='mgr.x' 2023-11-27T15:15:59.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: from='mgr.14100 172.21.15.8:0/2802333281' entity='mgr.x' 2023-11-27T15:15:59.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:15:59 smithi008 ceph-mon[128788]: from='mgr.14100 172.21.15.8:0/2802333281' entity='mgr.x' 2023-11-27T15:16:00.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:16:00.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:16:00.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:16:00.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:16:00.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:16:00.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-11-27T15:16:00.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:16:00.543 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:16:00.543 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:16:00.543 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:16:00.543 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:16:00.543 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:16:00.543 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:16:00.543 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:16:00.543 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:16:00.543 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:16:00.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:16:00.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:16:00.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:16:00.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:16:00.552 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:16:00.553 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:15:42.569758+0000", 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.973+0000 7f8a5fb80700 1 Processor -- start 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.973+0000 7f8a5fb80700 1 -- start start 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.974+0000 7f8a5fb80700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.974+0000 7f8a5fb80700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5807a840 con 0x7f8a5807be00 2023-11-27T15:16:00.554 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.974+0000 7f8a5d91c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.974+0000 7f8a5d91c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40372/0 (socket says 172.21.15.8:40372) 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.974+0000 7f8a5d91c700 1 -- 172.21.15.8:0/276056663 learned_addr learned my addr 172.21.15.8:0/276056663 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.975+0000 7f8a5d91c700 1 -- 172.21.15.8:0/276056663 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5807a980 con 0x7f8a5807be00 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.975+0000 7f8a5d91c700 1 --2- 172.21.15.8:0/276056663 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5807a300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a540098a0 tx=0x7f8a54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948a7adeccea055f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.975+0000 7f8a5c91a700 1 -- 172.21.15.8:0/276056663 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a54004160 con 0x7f8a5807be00 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.975+0000 7f8a5c91a700 1 -- 172.21.15.8:0/276056663 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a540042c0 con 0x7f8a5807be00 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.975+0000 7f8a5c91a700 1 -- 172.21.15.8:0/276056663 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a54011a00 con 0x7f8a5807be00 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.976+0000 7f8a5fb80700 1 -- 172.21.15.8:0/276056663 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 msgr2=0x7f8a5807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.976+0000 7f8a5fb80700 1 --2- 172.21.15.8:0/276056663 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5807a300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a540098a0 tx=0x7f8a54009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.976+0000 7f8a5fb80700 1 -- 172.21.15.8:0/276056663 shutdown_connections 2023-11-27T15:16:00.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.976+0000 7f8a5fb80700 1 --2- 172.21.15.8:0/276056663 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5807a300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.976+0000 7f8a5fb80700 1 -- 172.21.15.8:0/276056663 >> 172.21.15.8:0/276056663 conn(0x7f8a58100fa0 msgr2=0x7f8a581033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.976+0000 7f8a5fb80700 1 -- 172.21.15.8:0/276056663 shutdown_connections 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.976+0000 7f8a5fb80700 1 -- 172.21.15.8:0/276056663 wait complete. 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.977+0000 7f8a5fb80700 1 Processor -- start 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.977+0000 7f8a5fb80700 1 -- start start 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.977+0000 7f8a5fb80700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.977+0000 7f8a5fb80700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5819bd00 con 0x7f8a5807be00 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.978+0000 7f8a5d91c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.978+0000 7f8a5d91c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5819b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40384/0 (socket says 172.21.15.8:40384) 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.978+0000 7f8a5d91c700 1 -- 172.21.15.8:0/599126609 learned_addr learned my addr 172.21.15.8:0/599126609 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:00.556 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.978+0000 7f8a5d91c700 1 -- 172.21.15.8:0/599126609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54009550 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.978+0000 7f8a5d91c700 1 --2- 172.21.15.8:0/599126609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5819b7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a54006b20 tx=0x7f8a54011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.978+0000 7f8a4effd700 1 -- 172.21.15.8:0/599126609 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a54003b70 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.978+0000 7f8a4effd700 1 -- 172.21.15.8:0/599126609 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a54003cd0 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.978+0000 7f8a4effd700 1 -- 172.21.15.8:0/599126609 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a540247c0 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.978+0000 7f8a5fb80700 1 -- 172.21.15.8:0/599126609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5810bde0 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.979+0000 7f8a5fb80700 1 -- 172.21.15.8:0/599126609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5810bc50 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.979+0000 7f8a4effd700 1 -- 172.21.15.8:0/599126609 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 2) v1 ==== 45239+0+0 (secure 0 0 0) 0x7f8a5401ab40 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.980+0000 7f8a5fb80700 1 -- 172.21.15.8:0/599126609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5810bf70 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.980+0000 7f8a4effd700 1 -- 172.21.15.8:0/599126609 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8a5404c160 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:15:59.982+0000 7f8a4effd700 1 -- 172.21.15.8:0/599126609 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f8a54024e00 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.059+0000 7f8a4effd700 1 -- 172.21.15.8:0/599126609 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7f8a5401a7b0 con 0x7f8a5807be00 2023-11-27T15:16:00.557 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.059+0000 7f8a4effd700 1 --2- 172.21.15.8:0/599126609 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f8a440393b0 0x7f8a4403b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.059+0000 7f8a5d11b700 1 --2- 172.21.15.8:0/599126609 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f8a440393b0 0x7f8a4403b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.061+0000 7f8a5d11b700 1 --2- 172.21.15.8:0/599126609 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f8a440393b0 0x7f8a4403b790 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8a48006fd0 tx=0x7f8a48006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.061+0000 7f8a5fb80700 1 -- 172.21.15.8:0/599126609 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8a5819f490 con 0x7f8a5807be00 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.063+0000 7f8a4effd700 1 -- 172.21.15.8:0/599126609 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f8a54018350 con 0x7f8a5807be00 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.064+0000 7f8a5fb80700 1 -- 172.21.15.8:0/599126609 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f8a440393b0 msgr2=0x7f8a4403b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.064+0000 7f8a5fb80700 1 --2- 172.21.15.8:0/599126609 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f8a440393b0 0x7f8a4403b790 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8a48006fd0 tx=0x7f8a48006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.064+0000 7f8a5fb80700 1 -- 172.21.15.8:0/599126609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 msgr2=0x7f8a5819b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.064+0000 7f8a5fb80700 1 --2- 172.21.15.8:0/599126609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5819b7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a54006b20 tx=0x7f8a54011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.064+0000 7f8a5fb80700 1 -- 172.21.15.8:0/599126609 shutdown_connections 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.064+0000 7f8a5fb80700 1 --2- 172.21.15.8:0/599126609 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f8a440393b0 0x7f8a4403b790 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.558 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.064+0000 7f8a5fb80700 1 --2- 172.21.15.8:0/599126609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a5807be00 0x7f8a5819b7c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:00.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.064+0000 7f8a5fb80700 1 -- 172.21.15.8:0/599126609 >> 172.21.15.8:0/599126609 conn(0x7f8a58100fa0 msgr2=0x7f8a58101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:00.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.064+0000 7f8a5fb80700 1 -- 172.21.15.8:0/599126609 shutdown_connections 2023-11-27T15:16:00.559 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:00.064+0000 7f8a5fb80700 1 -- 172.21.15.8:0/599126609 wait complete. 2023-11-27T15:16:00.559 INFO:teuthology.orchestra.run.smithi008.stdout:mgr is available 2023-11-27T15:16:01.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:01 smithi008 ceph-mon[128788]: mgrmap e3: x(active, since 1.00995s) 2023-11-27T15:16:01.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:01 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/599126609' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:16:01.632 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:16:01.632 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [global] 2023-11-27T15:16:01.632 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout fsid = 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:16:01.632 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-27T15:16:01.632 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [osd] 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-27T15:16:01.633 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.050+0000 7fe4c17cb700 1 Processor -- start 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.050+0000 7fe4c17cb700 1 -- start start 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.050+0000 7fe4c17cb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc079d00 0x7fe4bc1084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.050+0000 7fe4c17cb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc1089f0 con 0x7fe4bc079d00 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.050+0000 7fe4baffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc079d00 0x7fe4bc1084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.050+0000 7fe4baffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc079d00 0x7fe4bc1084b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40392/0 (socket says 172.21.15.8:40392) 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.050+0000 7fe4baffd700 1 -- 172.21.15.8:0/2290785138 learned_addr learned my addr 172.21.15.8:0/2290785138 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.051+0000 7fe4baffd700 1 -- 172.21.15.8:0/2290785138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc108b30 con 0x7fe4bc079d00 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.051+0000 7fe4baffd700 1 --2- 172.21.15.8:0/2290785138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc079d00 0x7fe4bc1084b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0098a0 tx=0x7fe4ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ee86f05a032ef1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.051+0000 7fe4b9ffb700 1 -- 172.21.15.8:0/2290785138 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac004160 con 0x7fe4bc079d00 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.051+0000 7fe4b9ffb700 1 -- 172.21.15.8:0/2290785138 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe4ac0042c0 con 0x7fe4bc079d00 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.051+0000 7fe4b9ffb700 1 -- 172.21.15.8:0/2290785138 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac011a00 con 0x7fe4bc079d00 2023-11-27T15:16:01.634 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.052+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2290785138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc079d00 msgr2=0x7fe4bc1084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.052+0000 7fe4c17cb700 1 --2- 172.21.15.8:0/2290785138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc079d00 0x7fe4bc1084b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0098a0 tx=0x7fe4ac009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.052+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2290785138 shutdown_connections 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.052+0000 7fe4c17cb700 1 --2- 172.21.15.8:0/2290785138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc079d00 0x7fe4bc1084b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.052+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2290785138 >> 172.21.15.8:0/2290785138 conn(0x7fe4bc0ffe80 msgr2=0x7fe4bc102290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.052+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2290785138 shutdown_connections 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.052+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2290785138 wait complete. 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.053+0000 7fe4c17cb700 1 Processor -- start 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.054+0000 7fe4c17cb700 1 -- start start 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.054+0000 7fe4c17cb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc19b770 0x7fe4bc19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.054+0000 7fe4c17cb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc19c0c0 con 0x7fe4bc19b770 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.054+0000 7fe4baffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc19b770 0x7fe4bc19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.054+0000 7fe4baffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc19b770 0x7fe4bc19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40398/0 (socket says 172.21.15.8:40398) 2023-11-27T15:16:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.054+0000 7fe4baffd700 1 -- 172.21.15.8:0/2123531486 learned_addr learned my addr 172.21.15.8:0/2123531486 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.054+0000 7fe4baffd700 1 -- 172.21.15.8:0/2123531486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac009550 con 0x7fe4bc19b770 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.055+0000 7fe4baffd700 1 --2- 172.21.15.8:0/2123531486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc19b770 0x7fe4bc19bb80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac006af0 tx=0x7fe4ac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.055+0000 7fe4a3fff700 1 -- 172.21.15.8:0/2123531486 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac003a40 con 0x7fe4bc19b770 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.055+0000 7fe4a3fff700 1 -- 172.21.15.8:0/2123531486 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe4ac003ba0 con 0x7fe4bc19b770 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.055+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2123531486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4bc109b80 con 0x7fe4bc19b770 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.055+0000 7fe4a3fff700 1 -- 172.21.15.8:0/2123531486 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4ac01a700 con 0x7fe4bc19b770 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.055+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2123531486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4bc1099f0 con 0x7fe4bc19b770 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.056+0000 7fe4a3fff700 1 -- 172.21.15.8:0/2123531486 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7fe4ac024430 con 0x7fe4bc19b770 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.056+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2123531486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4bc109d10 con 0x7fe4bc19b770 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.057+0000 7fe4a3fff700 1 --2- 172.21.15.8:0/2123531486 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fe4a4038650 0x7fe4a403ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:01.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.057+0000 7fe4a3fff700 1 -- 172.21.15.8:0/2123531486 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe4ac04c4d0 con 0x7fe4bc19b770 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.057+0000 7fe4ba7fc700 1 --2- 172.21.15.8:0/2123531486 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fe4a4038650 0x7fe4a403ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.058+0000 7fe4ba7fc700 1 --2- 172.21.15.8:0/2123531486 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fe4a4038650 0x7fe4a403ab00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0006fd0 tx=0x7fe4b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.063+0000 7fe4a3fff700 1 -- 172.21.15.8:0/2123531486 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe4ac0189a0 con 0x7fe4bc19b770 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.065+0000 7fe4a3fff700 1 -- 172.21.15.8:0/2123531486 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fe4ac0246e0 con 0x7fe4bc19b770 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.268+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2123531486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe4bc1a0560 con 0x7fe4bc19b770 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.269+0000 7fe4a3fff700 1 -- 172.21.15.8:0/2123531486 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fe4ac018d90 con 0x7fe4bc19b770 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.271+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2123531486 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fe4a4038650 msgr2=0x7fe4a403ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.271+0000 7fe4c17cb700 1 --2- 172.21.15.8:0/2123531486 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fe4a4038650 0x7fe4a403ab00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0006fd0 tx=0x7fe4b0006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.271+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2123531486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc19b770 msgr2=0x7fe4bc19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.271+0000 7fe4c17cb700 1 --2- 172.21.15.8:0/2123531486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc19b770 0x7fe4bc19bb80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac006af0 tx=0x7fe4ac011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.271+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2123531486 shutdown_connections 2023-11-27T15:16:01.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.271+0000 7fe4c17cb700 1 --2- 172.21.15.8:0/2123531486 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fe4a4038650 0x7fe4a403ab00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:01.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.271+0000 7fe4c17cb700 1 --2- 172.21.15.8:0/2123531486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4bc19b770 0x7fe4bc19bb80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:01.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.271+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2123531486 >> 172.21.15.8:0/2123531486 conn(0x7fe4bc0ffe80 msgr2=0x7fe4bc100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:01.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.271+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2123531486 shutdown_connections 2023-11-27T15:16:01.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:01.271+0000 7fe4c17cb700 1 -- 172.21.15.8:0/2123531486 wait complete. 2023-11-27T15:16:01.638 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling cephadm module... 2023-11-27T15:16:02.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:02 smithi008 ceph-mon[128788]: mgrmap e4: x(active, since 2s) 2023-11-27T15:16:02.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:02 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2123531486' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-27T15:16:03.348 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:03 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/228903478' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-11-27T15:16:03.348 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:03 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: ignoring --setuser ceph since I am not root 2023-11-27T15:16:03.348 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:03 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: ignoring --setgroup ceph since I am not root 2023-11-27T15:16:03.348 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:03 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:03.343+0000 7f164fdd5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:16:03.614 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.161+0000 7fbf40dd2700 1 Processor -- start 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.161+0000 7fbf40dd2700 1 -- start start 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.162+0000 7fbf40dd2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.162+0000 7fbf40dd2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c106260 con 0x7fbf3c105910 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.162+0000 7fbf3a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.162+0000 7fbf3a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40400/0 (socket says 172.21.15.8:40400) 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.162+0000 7fbf3a59c700 1 -- 172.21.15.8:0/3896673173 learned_addr learned my addr 172.21.15.8:0/3896673173 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.162+0000 7fbf3a59c700 1 -- 172.21.15.8:0/3896673173 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c1063a0 con 0x7fbf3c105910 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.163+0000 7fbf3a59c700 1 --2- 172.21.15.8:0/3896673173 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c105d20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0098a0 tx=0x7fbf2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e72ef195d93ed3a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.163+0000 7fbf3959a700 1 -- 172.21.15.8:0/3896673173 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf2c004160 con 0x7fbf3c105910 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.163+0000 7fbf3959a700 1 -- 172.21.15.8:0/3896673173 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf2c0042c0 con 0x7fbf3c105910 2023-11-27T15:16:03.615 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.163+0000 7fbf3959a700 1 -- 172.21.15.8:0/3896673173 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf2c011a00 con 0x7fbf3c105910 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.163+0000 7fbf40dd2700 1 -- 172.21.15.8:0/3896673173 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 msgr2=0x7fbf3c105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.163+0000 7fbf40dd2700 1 --2- 172.21.15.8:0/3896673173 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c105d20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0098a0 tx=0x7fbf2c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.164+0000 7fbf40dd2700 1 -- 172.21.15.8:0/3896673173 shutdown_connections 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.164+0000 7fbf40dd2700 1 --2- 172.21.15.8:0/3896673173 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c105d20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.164+0000 7fbf40dd2700 1 -- 172.21.15.8:0/3896673173 >> 172.21.15.8:0/3896673173 conn(0x7fbf3c100fa0 msgr2=0x7fbf3c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.164+0000 7fbf40dd2700 1 -- 172.21.15.8:0/3896673173 shutdown_connections 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.164+0000 7fbf40dd2700 1 -- 172.21.15.8:0/3896673173 wait complete. 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.165+0000 7fbf40dd2700 1 Processor -- start 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.165+0000 7fbf40dd2700 1 -- start start 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.166+0000 7fbf40dd2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c07bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.166+0000 7fbf40dd2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c07c190 con 0x7fbf3c105910 2023-11-27T15:16:03.616 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.166+0000 7fbf3a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c07bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.166+0000 7fbf3a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c07bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40414/0 (socket says 172.21.15.8:40414) 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.166+0000 7fbf3a59c700 1 -- 172.21.15.8:0/228903478 learned_addr learned my addr 172.21.15.8:0/228903478 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.166+0000 7fbf3a59c700 1 -- 172.21.15.8:0/228903478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c009550 con 0x7fbf3c105910 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.167+0000 7fbf3a59c700 1 --2- 172.21.15.8:0/228903478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c07bc50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006af0 tx=0x7fbf2c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.167+0000 7fbf2b7fe700 1 -- 172.21.15.8:0/228903478 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf2c003a40 con 0x7fbf3c105910 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.167+0000 7fbf2b7fe700 1 -- 172.21.15.8:0/228903478 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf2c003ba0 con 0x7fbf3c105910 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.167+0000 7fbf2b7fe700 1 -- 172.21.15.8:0/228903478 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf2c01a6e0 con 0x7fbf3c105910 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.167+0000 7fbf40dd2700 1 -- 172.21.15.8:0/228903478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c107960 con 0x7fbf3c105910 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.167+0000 7fbf40dd2700 1 -- 172.21.15.8:0/228903478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c1077d0 con 0x7fbf3c105910 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.168+0000 7fbf2b7fe700 1 -- 172.21.15.8:0/228903478 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fbf2c024430 con 0x7fbf3c105910 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.168+0000 7fbf40dd2700 1 -- 172.21.15.8:0/228903478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c062120 con 0x7fbf3c105910 2023-11-27T15:16:03.617 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.169+0000 7fbf2b7fe700 1 --2- 172.21.15.8:0/228903478 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fbf240387d0 0x7fbf2403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.169+0000 7fbf2b7fe700 1 -- 172.21.15.8:0/228903478 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbf2c04c660 con 0x7fbf3c105910 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.169+0000 7fbf39d9b700 1 --2- 172.21.15.8:0/228903478 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fbf240387d0 0x7fbf2403ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.170+0000 7fbf39d9b700 1 --2- 172.21.15.8:0/228903478 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fbf240387d0 0x7fbf2403ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf30006fd0 tx=0x7fbf30006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.174+0000 7fbf2b7fe700 1 -- 172.21.15.8:0/228903478 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbf2c0246e0 con 0x7fbf3c105910 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:02.439+0000 7fbf40dd2700 1 -- 172.21.15.8:0/228903478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fbf3c07b170 con 0x7fbf3c105910 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.073+0000 7fbf2b7fe700 1 -- 172.21.15.8:0/228903478 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fbf2c02d8c0 con 0x7fbf3c105910 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.075+0000 7fbf2b7fe700 1 -- 172.21.15.8:0/228903478 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fbf2c024b10 con 0x7fbf3c105910 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.079+0000 7fbf40dd2700 1 -- 172.21.15.8:0/228903478 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fbf240387d0 msgr2=0x7fbf2403ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.079+0000 7fbf40dd2700 1 --2- 172.21.15.8:0/228903478 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fbf240387d0 0x7fbf2403ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf30006fd0 tx=0x7fbf30006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.079+0000 7fbf40dd2700 1 -- 172.21.15.8:0/228903478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 msgr2=0x7fbf3c07bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:03.618 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.079+0000 7fbf40dd2700 1 --2- 172.21.15.8:0/228903478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c07bc50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c006af0 tx=0x7fbf2c011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.619 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.079+0000 7fbf40dd2700 1 -- 172.21.15.8:0/228903478 shutdown_connections 2023-11-27T15:16:03.619 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.079+0000 7fbf40dd2700 1 --2- 172.21.15.8:0/228903478 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7fbf240387d0 0x7fbf2403ac80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.619 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.079+0000 7fbf40dd2700 1 --2- 172.21.15.8:0/228903478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbf3c105910 0x7fbf3c07bc50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:03.619 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.079+0000 7fbf40dd2700 1 -- 172.21.15.8:0/228903478 >> 172.21.15.8:0/228903478 conn(0x7fbf3c100fa0 msgr2=0x7fbf3c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:03.619 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.079+0000 7fbf40dd2700 1 -- 172.21.15.8:0/228903478 shutdown_connections 2023-11-27T15:16:03.619 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:03.079+0000 7fbf40dd2700 1 -- 172.21.15.8:0/228903478 wait complete. 2023-11-27T15:16:03.619 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:03 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:03.451+0000 7f164fdd5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:16:03.619 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:03 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:03.568+0000 7f164fdd5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:16:03.909 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:03 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:03.707+0000 7f164fdd5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:16:04.356 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:04 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/228903478' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-11-27T15:16:04.357 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:04 smithi008 ceph-mon[128788]: mgrmap e5: x(active, since 4s) 2023-11-27T15:16:04.357 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:03 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:03.941+0000 7f164fdd5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:16:04.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:04 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:04.352+0000 7f164fdd5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:16:04.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:04 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:04.473+0000 7f164fdd5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:16:04.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:16:04.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-11-27T15:16:04.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-27T15:16:04.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-27T15:16:04.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-27T15:16:04.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:16:04.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.178+0000 7f513b875700 1 Processor -- start 2023-11-27T15:16:04.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.178+0000 7f513b875700 1 -- start start 2023-11-27T15:16:04.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.178+0000 7f513b875700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:04.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.178+0000 7f513b875700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513407a800 con 0x7f513407bdc0 2023-11-27T15:16:04.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.179+0000 7f5139611700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.179+0000 7f5139611700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40452/0 (socket says 172.21.15.8:40452) 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.179+0000 7f5139611700 1 -- 172.21.15.8:0/2780399934 learned_addr learned my addr 172.21.15.8:0/2780399934 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.179+0000 7f5139611700 1 -- 172.21.15.8:0/2780399934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513407a940 con 0x7f513407bdc0 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.179+0000 7f5139611700 1 --2- 172.21.15.8:0/2780399934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513407a2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51300098a0 tx=0x7f5130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9239de2de1a2b6d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.180+0000 7f512bfff700 1 -- 172.21.15.8:0/2780399934 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5130004160 con 0x7f513407bdc0 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.180+0000 7f512bfff700 1 -- 172.21.15.8:0/2780399934 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f51300042c0 con 0x7f513407bdc0 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.180+0000 7f512bfff700 1 -- 172.21.15.8:0/2780399934 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5130011a00 con 0x7f513407bdc0 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.180+0000 7f513b875700 1 -- 172.21.15.8:0/2780399934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 msgr2=0x7f513407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.180+0000 7f513b875700 1 --2- 172.21.15.8:0/2780399934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513407a2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51300098a0 tx=0x7f5130009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.180+0000 7f513b875700 1 -- 172.21.15.8:0/2780399934 shutdown_connections 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.180+0000 7f513b875700 1 --2- 172.21.15.8:0/2780399934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513407a2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:04.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.180+0000 7f513b875700 1 -- 172.21.15.8:0/2780399934 >> 172.21.15.8:0/2780399934 conn(0x7f5134100fa0 msgr2=0x7f51341033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.180+0000 7f513b875700 1 -- 172.21.15.8:0/2780399934 shutdown_connections 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.181+0000 7f513b875700 1 -- 172.21.15.8:0/2780399934 wait complete. 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.181+0000 7f513b875700 1 Processor -- start 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.182+0000 7f513b875700 1 -- start start 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.182+0000 7f513b875700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.182+0000 7f513b875700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513419bcf0 con 0x7f513407bdc0 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.182+0000 7f5139611700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.182+0000 7f5139611700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40466/0 (socket says 172.21.15.8:40466) 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.182+0000 7f5139611700 1 -- 172.21.15.8:0/3638488323 learned_addr learned my addr 172.21.15.8:0/3638488323 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.183+0000 7f5139611700 1 -- 172.21.15.8:0/3638488323 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130009550 con 0x7f513407bdc0 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.183+0000 7f5139611700 1 --2- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513419b7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5130006af0 tx=0x7f5130011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:04.914 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.183+0000 7f512a7fc700 1 -- 172.21.15.8:0/3638488323 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5130003a40 con 0x7f513407bdc0 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.183+0000 7f512a7fc700 1 -- 172.21.15.8:0/3638488323 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5130003ba0 con 0x7f513407bdc0 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.183+0000 7f512a7fc700 1 -- 172.21.15.8:0/3638488323 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f513001a700 con 0x7f513407bdc0 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.183+0000 7f513b875700 1 -- 172.21.15.8:0/3638488323 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513410bdc0 con 0x7f513407bdc0 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.183+0000 7f513b875700 1 -- 172.21.15.8:0/3638488323 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513410bc30 con 0x7f513407bdc0 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.184+0000 7f512a7fc700 1 -- 172.21.15.8:0/3638488323 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f5130024ad0 con 0x7f513407bdc0 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.185+0000 7f513b875700 1 -- 172.21.15.8:0/3638488323 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5134062120 con 0x7f513407bdc0 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.185+0000 7f512a7fc700 1 --2- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f51200387d0 0x7f512003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.185+0000 7f512a7fc700 1 -- 172.21.15.8:0/3638488323 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f513004c840 con 0x7f513407bdc0 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.185+0000 7f5138e10700 1 -- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f51200387d0 msgr2=0x7f512003ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/560372725 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.185+0000 7f5138e10700 1 --2- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f51200387d0 0x7f512003ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.190+0000 7f512a7fc700 1 -- 172.21.15.8:0/3638488323 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5130018b80 con 0x7f513407bdc0 2023-11-27T15:16:04.915 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.386+0000 7f5138e10700 1 -- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f51200387d0 msgr2=0x7f512003ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/560372725 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.386+0000 7f5138e10700 1 --2- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f51200387d0 0x7f512003ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.457+0000 7f513b875700 1 -- 172.21.15.8:0/3638488323 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f513419c230 con 0x7f513407bdc0 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.457+0000 7f512a7fc700 1 -- 172.21.15.8:0/3638488323 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f5130018350 con 0x7f513407bdc0 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.459+0000 7f513b875700 1 -- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f51200387d0 msgr2=0x7f512003ac80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.459+0000 7f513b875700 1 --2- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f51200387d0 0x7f512003ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.459+0000 7f513b875700 1 -- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 msgr2=0x7f513419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.459+0000 7f513b875700 1 --2- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513419b7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5130006af0 tx=0x7f5130011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.459+0000 7f513b875700 1 -- 172.21.15.8:0/3638488323 shutdown_connections 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.459+0000 7f513b875700 1 --2- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7f51200387d0 0x7f512003ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.459+0000 7f513b875700 1 --2- 172.21.15.8:0/3638488323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f513407bdc0 0x7f513419b7b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.460+0000 7f513b875700 1 -- 172.21.15.8:0/3638488323 >> 172.21.15.8:0/3638488323 conn(0x7f5134100fa0 msgr2=0x7f5134101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.460+0000 7f513b875700 1 -- 172.21.15.8:0/3638488323 shutdown_connections 2023-11-27T15:16:04.916 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:04.460+0000 7f513b875700 1 -- 172.21.15.8:0/3638488323 wait complete. 2023-11-27T15:16:04.917 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for the mgr to restart... 2023-11-27T15:16:04.917 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr epoch 5... 2023-11-27T15:16:05.077 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:04 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:04.650+0000 7f164fdd5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:16:05.077 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:04 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:04.779+0000 7f164fdd5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:16:05.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:05 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3638488323' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-27T15:16:05.863 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:05 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:05.360+0000 7f164fdd5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:16:06.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:06 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:06.137+0000 7f164fdd5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:16:06.614 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:06 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:06.351+0000 7f164fdd5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:16:07.614 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:07 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:07.331+0000 7f164fdd5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:16:07.614 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:07 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:07.455+0000 7f164fdd5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:16:08.064 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:07 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:07.684+0000 7f164fdd5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:16:08.348 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:08 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:08.060+0000 7f164fdd5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:16:08.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:08 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:08.343+0000 7f164fdd5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:16:08.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:08 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:08.543+0000 7f164fdd5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:16:09.363 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:08 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:08.899+0000 7f164fdd5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:16:10.304 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:09 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:09.865+0000 7f164fdd5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:16:10.305 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:10.043+0000 7f164fdd5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:16:10.305 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:10.300+0000 7f164fdd5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:16:10.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:10.452+0000 7f164fdd5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:16:11.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:10.739+0000 7f164fdd5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:16:12.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:11.841+0000 7f164fdd5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:16:12.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: Active manager daemon x restarted 2023-11-27T15:16:12.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: Activating manager daemon x 2023-11-27T15:16:12.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: osdmap e2: 0 total, 0 up, 0 in 2023-11-27T15:16:12.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: mgrmap e6: x(active, starting, since 0.00404741s) 2023-11-27T15:16:12.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:12.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:16:12.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:16:12.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:16:12.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:16:12.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: Manager daemon x is now available 2023-11-27T15:16:12.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:12.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:12.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:12.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:11 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:13.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:12 smithi008 ceph-mon[128788]: Found migration_current of "None". Setting to last migration. 2023-11-27T15:16:13.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:12 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:13.284 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:12 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:16:13.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:12 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:16:13.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:12 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:13.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:12 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:13.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:12 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:13.285 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:12 smithi008 ceph-mon[128788]: mgrmap e7: x(active, since 1.00706s) 2023-11-27T15:16:13.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:16:13.313 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.439+0000 7ff44ab1e700 1 Processor -- start 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.439+0000 7ff44ab1e700 1 -- start start 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.439+0000 7ff44ab1e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.439+0000 7ff44ab1e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44407a820 con 0x7ff44407bde0 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.440+0000 7ff4488ba700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.440+0000 7ff4488ba700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40478/0 (socket says 172.21.15.8:40478) 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.440+0000 7ff4488ba700 1 -- 172.21.15.8:0/2867995475 learned_addr learned my addr 172.21.15.8:0/2867995475 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.440+0000 7ff4488ba700 1 -- 172.21.15.8:0/2867995475 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44407a960 con 0x7ff44407bde0 2023-11-27T15:16:13.314 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.441+0000 7ff4488ba700 1 --2- 172.21.15.8:0/2867995475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44407a2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4380098a0 tx=0x7ff438009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e997049bcd0625f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.441+0000 7ff4437fe700 1 -- 172.21.15.8:0/2867995475 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff438004160 con 0x7ff44407bde0 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.441+0000 7ff4437fe700 1 -- 172.21.15.8:0/2867995475 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff4380042c0 con 0x7ff44407bde0 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.441+0000 7ff4437fe700 1 -- 172.21.15.8:0/2867995475 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff438011a00 con 0x7ff44407bde0 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.442+0000 7ff44ab1e700 1 -- 172.21.15.8:0/2867995475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 msgr2=0x7ff44407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.442+0000 7ff44ab1e700 1 --2- 172.21.15.8:0/2867995475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44407a2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff4380098a0 tx=0x7ff438009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.442+0000 7ff44ab1e700 1 -- 172.21.15.8:0/2867995475 shutdown_connections 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.442+0000 7ff44ab1e700 1 --2- 172.21.15.8:0/2867995475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44407a2e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.442+0000 7ff44ab1e700 1 -- 172.21.15.8:0/2867995475 >> 172.21.15.8:0/2867995475 conn(0x7ff444100fa0 msgr2=0x7ff4441033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.442+0000 7ff44ab1e700 1 -- 172.21.15.8:0/2867995475 shutdown_connections 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.442+0000 7ff44ab1e700 1 -- 172.21.15.8:0/2867995475 wait complete. 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.443+0000 7ff44ab1e700 1 Processor -- start 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.443+0000 7ff44ab1e700 1 -- start start 2023-11-27T15:16:13.315 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.443+0000 7ff44ab1e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.443+0000 7ff44ab1e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44419bcf0 con 0x7ff44407bde0 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.444+0000 7ff4488ba700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.444+0000 7ff4488ba700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40480/0 (socket says 172.21.15.8:40480) 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.444+0000 7ff4488ba700 1 -- 172.21.15.8:0/289574167 learned_addr learned my addr 172.21.15.8:0/289574167 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.444+0000 7ff4488ba700 1 -- 172.21.15.8:0/289574167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff438009550 con 0x7ff44407bde0 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.444+0000 7ff4488ba700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44419b7b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff438006af0 tx=0x7ff438011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.445+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff438003a40 con 0x7ff44407bde0 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.445+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff438003ba0 con 0x7ff44407bde0 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.445+0000 7ff44ab1e700 1 -- 172.21.15.8:0/289574167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44410be00 con 0x7ff44407bde0 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.445+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff43801a700 con 0x7ff44407bde0 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.445+0000 7ff44ab1e700 1 -- 172.21.15.8:0/289574167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44410bc70 con 0x7ff44407bde0 2023-11-27T15:16:13.316 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.446+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff438024430 con 0x7ff44407bde0 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.446+0000 7ff441ffb700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 0x7ff43403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.446+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 --> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff43403b3e0 con 0x7ff434038820 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.447+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff43804c640 con 0x7ff44407bde0 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.447+0000 7ff443fff700 1 -- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 msgr2=0x7ff43403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/560372725 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.447+0000 7ff443fff700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 0x7ff43403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.648+0000 7ff443fff700 1 -- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 msgr2=0x7ff43403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/560372725 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:05.648+0000 7ff443fff700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 0x7ff43403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:06.049+0000 7ff443fff700 1 -- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 msgr2=0x7ff43403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/560372725 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:06.049+0000 7ff443fff700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 0x7ff43403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:06.850+0000 7ff443fff700 1 -- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 msgr2=0x7ff43403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/560372725 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:06.850+0000 7ff443fff700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 0x7ff43403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-27T15:16:13.317 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:08.451+0000 7ff443fff700 1 -- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 msgr2=0x7ff43403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/560372725 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:08.452+0000 7ff443fff700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 0x7ff43403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:11.655+0000 7ff443fff700 1 -- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 msgr2=0x7ff43403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/560372725 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:11.655+0000 7ff443fff700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 0x7ff43403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:11.846+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7ff4380246e0 con 0x7ff44407bde0 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:11.846+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 msgr2=0x7ff43403acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:11.846+0000 7ff441ffb700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 0x7ff43403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.848+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7ff43804e200 con 0x7ff44407bde0 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.848+0000 7ff441ffb700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff43403bfb0 0x7ff43403e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.849+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 --> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff43403b3e0 con 0x7ff43403bfb0 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.849+0000 7ff443fff700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff43403bfb0 0x7ff43403e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:13.318 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.850+0000 7ff443fff700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff43403bfb0 0x7ff43403e390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff430003a10 tx=0x7ff4300092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.851+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 <== mgr.14120 v2:172.21.15.8:6800/2324276308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7ff43403b3e0 con 0x7ff43403bfb0 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.858+0000 7ff44ab1e700 1 -- 172.21.15.8:0/289574167 --> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff44419caf0 con 0x7ff43403bfb0 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.858+0000 7ff441ffb700 1 -- 172.21.15.8:0/289574167 <== mgr.14120 v2:172.21.15.8:6800/2324276308 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff44419caf0 con 0x7ff43403bfb0 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 -- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff43403bfb0 msgr2=0x7ff43403e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff43403bfb0 0x7ff43403e390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff430003a10 tx=0x7ff4300092b0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 -- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 msgr2=0x7ff44419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44419b7b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff438006af0 tx=0x7ff438011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 -- 172.21.15.8:0/289574167 shutdown_connections 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff44407bde0 0x7ff44419b7b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff43403bfb0 0x7ff43403e390 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 --2- 172.21.15.8:0/289574167 >> [v2:172.21.15.8:6800/560372725,v1:172.21.15.8:6801/560372725] conn(0x7ff434038820 0x7ff43403acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:13.319 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 -- 172.21.15.8:0/289574167 >> 172.21.15.8:0/289574167 conn(0x7ff444100fa0 msgr2=0x7ff444101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:13.320 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 -- 172.21.15.8:0/289574167 shutdown_connections 2023-11-27T15:16:13.320 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:12.859+0000 7ff44ab1e700 1 -- 172.21.15.8:0/289574167 wait complete. 2023-11-27T15:16:13.320 INFO:teuthology.orchestra.run.smithi008.stdout:mgr epoch 5 is available 2023-11-27T15:16:13.320 INFO:teuthology.orchestra.run.smithi008.stdout:Setting orchestrator backend to cephadm... 2023-11-27T15:16:14.250 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:13 smithi008 ceph-mon[128788]: [27/Nov/2023:15:16:12] ENGINE Bus STARTING 2023-11-27T15:16:14.250 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:13 smithi008 ceph-mon[128788]: [27/Nov/2023:15:16:12] ENGINE Serving on https://172.21.15.8:7150 2023-11-27T15:16:14.250 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:13 smithi008 ceph-mon[128788]: [27/Nov/2023:15:16:12] ENGINE Serving on http://172.21.15.8:8765 2023-11-27T15:16:14.250 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:13 smithi008 ceph-mon[128788]: [27/Nov/2023:15:16:12] ENGINE Bus STARTED 2023-11-27T15:16:14.250 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:13 smithi008 ceph-mon[128788]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-27T15:16:14.251 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:13 smithi008 ceph-mon[128788]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-27T15:16:14.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.866+0000 7fa330ce2700 1 Processor -- start 2023-11-27T15:16:14.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.866+0000 7fa330ce2700 1 -- start start 2023-11-27T15:16:14.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.867+0000 7fa330ce2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c07a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:14.494 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.867+0000 7fa330ce2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c07a860 con 0x7fa32c07be20 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.867+0000 7fa32a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c07a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.867+0000 7fa32a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c07a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50810/0 (socket says 172.21.15.8:50810) 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.867+0000 7fa32a59c700 1 -- 172.21.15.8:0/1686236232 learned_addr learned my addr 172.21.15.8:0/1686236232 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.867+0000 7fa32a59c700 1 -- 172.21.15.8:0/1686236232 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c07a9a0 con 0x7fa32c07be20 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.868+0000 7fa32a59c700 1 --2- 172.21.15.8:0/1686236232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c07a320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0098a0 tx=0x7fa31c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a47cabcf580004d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.868+0000 7fa32959a700 1 -- 172.21.15.8:0/1686236232 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa31c004160 con 0x7fa32c07be20 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.868+0000 7fa32959a700 1 -- 172.21.15.8:0/1686236232 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa31c0042c0 con 0x7fa32c07be20 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.868+0000 7fa32959a700 1 -- 172.21.15.8:0/1686236232 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa31c011a00 con 0x7fa32c07be20 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.868+0000 7fa330ce2700 1 -- 172.21.15.8:0/1686236232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 msgr2=0x7fa32c07a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.868+0000 7fa330ce2700 1 --2- 172.21.15.8:0/1686236232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c07a320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0098a0 tx=0x7fa31c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.495 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.869+0000 7fa330ce2700 1 -- 172.21.15.8:0/1686236232 shutdown_connections 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.869+0000 7fa330ce2700 1 --2- 172.21.15.8:0/1686236232 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c07a320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.869+0000 7fa330ce2700 1 -- 172.21.15.8:0/1686236232 >> 172.21.15.8:0/1686236232 conn(0x7fa32c100fa0 msgr2=0x7fa32c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.869+0000 7fa330ce2700 1 -- 172.21.15.8:0/1686236232 shutdown_connections 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.869+0000 7fa330ce2700 1 -- 172.21.15.8:0/1686236232 wait complete. 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.870+0000 7fa330ce2700 1 Processor -- start 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.870+0000 7fa330ce2700 1 -- start start 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.870+0000 7fa330ce2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.870+0000 7fa330ce2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c19bcf0 con 0x7fa32c07be20 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.870+0000 7fa32a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.870+0000 7fa32a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c19b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50824/0 (socket says 172.21.15.8:50824) 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.870+0000 7fa32a59c700 1 -- 172.21.15.8:0/189110732 learned_addr learned my addr 172.21.15.8:0/189110732 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:14.496 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.871+0000 7fa32a59c700 1 -- 172.21.15.8:0/189110732 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c009550 con 0x7fa32c07be20 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.871+0000 7fa32a59c700 1 --2- 172.21.15.8:0/189110732 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c19b7b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006af0 tx=0x7fa31c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.871+0000 7fa31b7fe700 1 -- 172.21.15.8:0/189110732 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa31c003a40 con 0x7fa32c07be20 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.871+0000 7fa31b7fe700 1 -- 172.21.15.8:0/189110732 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa31c003ba0 con 0x7fa32c07be20 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.871+0000 7fa31b7fe700 1 -- 172.21.15.8:0/189110732 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa31c01a700 con 0x7fa32c07be20 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.871+0000 7fa330ce2700 1 -- 172.21.15.8:0/189110732 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c10bdc0 con 0x7fa32c07be20 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.871+0000 7fa330ce2700 1 -- 172.21.15.8:0/189110732 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c10bc30 con 0x7fa32c07be20 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.872+0000 7fa31b7fe700 1 -- 172.21.15.8:0/189110732 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fa31c024ad0 con 0x7fa32c07be20 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.872+0000 7fa330ce2700 1 -- 172.21.15.8:0/189110732 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c062120 con 0x7fa32c07be20 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.873+0000 7fa31b7fe700 1 --2- 172.21.15.8:0/189110732 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa3140386f0 0x7fa31403aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.873+0000 7fa31b7fe700 1 -- 172.21.15.8:0/189110732 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa31c04c500 con 0x7fa32c07be20 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.873+0000 7fa329d9b700 1 --2- 172.21.15.8:0/189110732 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa3140386f0 0x7fa31403aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:14.497 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.874+0000 7fa329d9b700 1 --2- 172.21.15.8:0/189110732 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa3140386f0 0x7fa31403aba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa320006fd0 tx=0x7fa320006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.878+0000 7fa31b7fe700 1 -- 172.21.15.8:0/189110732 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa31c018b80 con 0x7fa32c07be20 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:13.901+0000 7fa31b7fe700 1 -- 172.21.15.8:0/189110732 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fa31c02d910 con 0x7fa32c07be20 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.099+0000 7fa330ce2700 1 -- 172.21.15.8:0/189110732 --> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fa32c102390 con 0x7fa3140386f0 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.109+0000 7fa31b7fe700 1 -- 172.21.15.8:0/189110732 <== mgr.14120 v2:172.21.15.8:6800/2324276308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa32c102390 con 0x7fa3140386f0 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.112+0000 7fa330ce2700 1 -- 172.21.15.8:0/189110732 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa3140386f0 msgr2=0x7fa31403aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.112+0000 7fa330ce2700 1 --2- 172.21.15.8:0/189110732 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa3140386f0 0x7fa31403aba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa320006fd0 tx=0x7fa320006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.112+0000 7fa330ce2700 1 -- 172.21.15.8:0/189110732 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 msgr2=0x7fa32c19b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.112+0000 7fa330ce2700 1 --2- 172.21.15.8:0/189110732 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c19b7b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006af0 tx=0x7fa31c011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.112+0000 7fa330ce2700 1 -- 172.21.15.8:0/189110732 shutdown_connections 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.112+0000 7fa330ce2700 1 --2- 172.21.15.8:0/189110732 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa3140386f0 0x7fa31403aba0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.498 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.112+0000 7fa330ce2700 1 --2- 172.21.15.8:0/189110732 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa32c07be20 0x7fa32c19b7b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:14.499 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.112+0000 7fa330ce2700 1 -- 172.21.15.8:0/189110732 >> 172.21.15.8:0/189110732 conn(0x7fa32c100fa0 msgr2=0x7fa32c101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:14.499 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.112+0000 7fa330ce2700 1 -- 172.21.15.8:0/189110732 shutdown_connections 2023-11-27T15:16:14.499 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:14.112+0000 7fa330ce2700 1 -- 172.21.15.8:0/189110732 wait complete. 2023-11-27T15:16:15.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:14 smithi008 ceph-mon[128788]: mgrmap e8: x(active, since 2s) 2023-11-27T15:16:15.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:14 smithi008 ceph-mon[128788]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:15.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:14 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:15.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:14 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:15.673 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout value unchanged 2023-11-27T15:16:15.673 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.024+0000 7ff874159700 1 Processor -- start 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.024+0000 7ff874159700 1 -- start start 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.025+0000 7ff874159700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.025+0000 7ff874159700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c106260 con 0x7ff86c105910 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.025+0000 7ff871ef5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.025+0000 7ff871ef5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50832/0 (socket says 172.21.15.8:50832) 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.025+0000 7ff871ef5700 1 -- 172.21.15.8:0/474146493 learned_addr learned my addr 172.21.15.8:0/474146493 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.026+0000 7ff871ef5700 1 -- 172.21.15.8:0/474146493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c1063a0 con 0x7ff86c105910 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.026+0000 7ff871ef5700 1 --2- 172.21.15.8:0/474146493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c105d20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff8680098a0 tx=0x7ff868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0172f80a41dcc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.026+0000 7ff870ef3700 1 -- 172.21.15.8:0/474146493 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff868004160 con 0x7ff86c105910 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.026+0000 7ff870ef3700 1 -- 172.21.15.8:0/474146493 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff8680042c0 con 0x7ff86c105910 2023-11-27T15:16:15.674 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.026+0000 7ff870ef3700 1 -- 172.21.15.8:0/474146493 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff868011a00 con 0x7ff86c105910 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.027+0000 7ff874159700 1 -- 172.21.15.8:0/474146493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 msgr2=0x7ff86c105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.027+0000 7ff874159700 1 --2- 172.21.15.8:0/474146493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c105d20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff8680098a0 tx=0x7ff868009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.027+0000 7ff874159700 1 -- 172.21.15.8:0/474146493 shutdown_connections 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.027+0000 7ff874159700 1 --2- 172.21.15.8:0/474146493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c105d20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.027+0000 7ff874159700 1 -- 172.21.15.8:0/474146493 >> 172.21.15.8:0/474146493 conn(0x7ff86c100fa0 msgr2=0x7ff86c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.027+0000 7ff874159700 1 -- 172.21.15.8:0/474146493 shutdown_connections 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.027+0000 7ff874159700 1 -- 172.21.15.8:0/474146493 wait complete. 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.028+0000 7ff874159700 1 Processor -- start 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.028+0000 7ff874159700 1 -- start start 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.028+0000 7ff874159700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.028+0000 7ff874159700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c19bcb0 con 0x7ff86c105910 2023-11-27T15:16:15.675 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.029+0000 7ff871ef5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.029+0000 7ff871ef5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50842/0 (socket says 172.21.15.8:50842) 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.029+0000 7ff871ef5700 1 -- 172.21.15.8:0/3748512404 learned_addr learned my addr 172.21.15.8:0/3748512404 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.029+0000 7ff871ef5700 1 -- 172.21.15.8:0/3748512404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff868009550 con 0x7ff86c105910 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.029+0000 7ff871ef5700 1 --2- 172.21.15.8:0/3748512404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c19b770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff868006af0 tx=0x7ff868011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.029+0000 7ff862ffd700 1 -- 172.21.15.8:0/3748512404 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff868003a40 con 0x7ff86c105910 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.029+0000 7ff862ffd700 1 -- 172.21.15.8:0/3748512404 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff868003ba0 con 0x7ff86c105910 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.029+0000 7ff862ffd700 1 -- 172.21.15.8:0/3748512404 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff86801a6e0 con 0x7ff86c105910 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.029+0000 7ff874159700 1 -- 172.21.15.8:0/3748512404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86c107960 con 0x7ff86c105910 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.030+0000 7ff874159700 1 -- 172.21.15.8:0/3748512404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86c1077d0 con 0x7ff86c105910 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.031+0000 7ff862ffd700 1 -- 172.21.15.8:0/3748512404 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff868024a60 con 0x7ff86c105910 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.031+0000 7ff874159700 1 -- 172.21.15.8:0/3748512404 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86c109de0 con 0x7ff86c105910 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.031+0000 7ff862ffd700 1 --2- 172.21.15.8:0/3748512404 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff858038870 0x7ff85803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:15.676 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.031+0000 7ff862ffd700 1 -- 172.21.15.8:0/3748512404 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff86804c7e0 con 0x7ff86c105910 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.031+0000 7ff8716f4700 1 --2- 172.21.15.8:0/3748512404 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff858038870 0x7ff85803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.032+0000 7ff8716f4700 1 --2- 172.21.15.8:0/3748512404 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff858038870 0x7ff85803ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff85c006fd0 tx=0x7ff85c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.036+0000 7ff862ffd700 1 -- 172.21.15.8:0/3748512404 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff868024430 con 0x7ff86c105910 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.244+0000 7ff874159700 1 -- 172.21.15.8:0/3748512404 --> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7ff86c102320 con 0x7ff858038870 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.245+0000 7ff862ffd700 1 -- 172.21.15.8:0/3748512404 <== mgr.14120 v2:172.21.15.8:6800/2324276308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7ff86c102320 con 0x7ff858038870 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.247+0000 7ff874159700 1 -- 172.21.15.8:0/3748512404 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff858038870 msgr2=0x7ff85803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.247+0000 7ff874159700 1 --2- 172.21.15.8:0/3748512404 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff858038870 0x7ff85803ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff85c006fd0 tx=0x7ff85c006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.247+0000 7ff874159700 1 -- 172.21.15.8:0/3748512404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 msgr2=0x7ff86c19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.247+0000 7ff874159700 1 --2- 172.21.15.8:0/3748512404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c19b770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff868006af0 tx=0x7ff868011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.247+0000 7ff874159700 1 -- 172.21.15.8:0/3748512404 shutdown_connections 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.247+0000 7ff874159700 1 --2- 172.21.15.8:0/3748512404 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7ff858038870 0x7ff85803ad20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.677 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.247+0000 7ff874159700 1 --2- 172.21.15.8:0/3748512404 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff86c105910 0x7ff86c19b770 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.678 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.247+0000 7ff874159700 1 -- 172.21.15.8:0/3748512404 >> 172.21.15.8:0/3748512404 conn(0x7ff86c100fa0 msgr2=0x7ff86c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:15.678 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.247+0000 7ff874159700 1 -- 172.21.15.8:0/3748512404 shutdown_connections 2023-11-27T15:16:15.678 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:15.248+0000 7ff874159700 1 -- 172.21.15.8:0/3748512404 wait complete. 2023-11-27T15:16:15.678 INFO:teuthology.orchestra.run.smithi008.stdout:Generating ssh key... 2023-11-27T15:16:16.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-mon[128788]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:17.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: Generating public/private rsa key pair. 2023-11-27T15:16:17.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: Your identification has been saved in /tmp/tmpgj15mdj9/key. 2023-11-27T15:16:17.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: Your public key has been saved in /tmp/tmpgj15mdj9/key.pub. 2023-11-27T15:16:17.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: The key fingerprint is: 2023-11-27T15:16:17.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: SHA256:wJMbMY9ype8E9qkmCAXYOUQwnTvJg7+78A+aQ2tDW/8 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:16:17.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: The key's randomart image is: 2023-11-27T15:16:17.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: +---[RSA 3072]----+ 2023-11-27T15:16:17.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: |=*oo o . | 2023-11-27T15:16:17.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: |.o* . O | 2023-11-27T15:16:17.115 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: | o.+. % . | 2023-11-27T15:16:17.115 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: |..* + O . | 2023-11-27T15:16:17.115 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: |.. o . S | 2023-11-27T15:16:17.115 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: | +.o + | 2023-11-27T15:16:17.115 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: |+ *.o o . | 2023-11-27T15:16:17.115 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: |.X.. + | 2023-11-27T15:16:17.115 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: |+.=+. .E | 2023-11-27T15:16:17.115 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: +----[SHA256]-----+ 2023-11-27T15:16:17.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.206+0000 7f9e622e5700 1 Processor -- start 2023-11-27T15:16:17.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.206+0000 7f9e622e5700 1 -- start start 2023-11-27T15:16:17.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.207+0000 7f9e622e5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.207+0000 7f9e622e5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c07a820 con 0x7f9e5c07bde0 2023-11-27T15:16:17.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.207+0000 7f9e5bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.207+0000 7f9e5bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50846/0 (socket says 172.21.15.8:50846) 2023-11-27T15:16:17.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.207+0000 7f9e5bfff700 1 -- 172.21.15.8:0/3560861783 learned_addr learned my addr 172.21.15.8:0/3560861783 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:17.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.207+0000 7f9e5bfff700 1 -- 172.21.15.8:0/3560861783 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c07a960 con 0x7f9e5c07bde0 2023-11-27T15:16:17.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.208+0000 7f9e5bfff700 1 --2- 172.21.15.8:0/3560861783 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c07a2e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0098a0 tx=0x7f9e4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8545b150afe3c704 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.208+0000 7f9e5affd700 1 -- 172.21.15.8:0/3560861783 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e4c004160 con 0x7f9e5c07bde0 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.208+0000 7f9e5affd700 1 -- 172.21.15.8:0/3560861783 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e4c0042c0 con 0x7f9e5c07bde0 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.208+0000 7f9e5affd700 1 -- 172.21.15.8:0/3560861783 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e4c011a00 con 0x7f9e5c07bde0 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.208+0000 7f9e622e5700 1 -- 172.21.15.8:0/3560861783 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 msgr2=0x7f9e5c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.208+0000 7f9e622e5700 1 --2- 172.21.15.8:0/3560861783 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c07a2e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0098a0 tx=0x7f9e4c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.209+0000 7f9e622e5700 1 -- 172.21.15.8:0/3560861783 shutdown_connections 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.209+0000 7f9e622e5700 1 --2- 172.21.15.8:0/3560861783 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c07a2e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.209+0000 7f9e622e5700 1 -- 172.21.15.8:0/3560861783 >> 172.21.15.8:0/3560861783 conn(0x7f9e5c100fa0 msgr2=0x7f9e5c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.209+0000 7f9e622e5700 1 -- 172.21.15.8:0/3560861783 shutdown_connections 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.209+0000 7f9e622e5700 1 -- 172.21.15.8:0/3560861783 wait complete. 2023-11-27T15:16:17.159 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.210+0000 7f9e622e5700 1 Processor -- start 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.210+0000 7f9e622e5700 1 -- start start 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.211+0000 7f9e622e5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.211+0000 7f9e622e5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c19bcf0 con 0x7f9e5c07bde0 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.211+0000 7f9e5bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.211+0000 7f9e5bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c19b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50862/0 (socket says 172.21.15.8:50862) 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.211+0000 7f9e5bfff700 1 -- 172.21.15.8:0/3874945800 learned_addr learned my addr 172.21.15.8:0/3874945800 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.211+0000 7f9e5bfff700 1 -- 172.21.15.8:0/3874945800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c009550 con 0x7f9e5c07bde0 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.212+0000 7f9e5bfff700 1 --2- 172.21.15.8:0/3874945800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c19b7b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c006af0 tx=0x7f9e4c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.212+0000 7f9e597fa700 1 -- 172.21.15.8:0/3874945800 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e4c003a40 con 0x7f9e5c07bde0 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.212+0000 7f9e597fa700 1 -- 172.21.15.8:0/3874945800 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9e4c003ba0 con 0x7f9e5c07bde0 2023-11-27T15:16:17.160 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.212+0000 7f9e597fa700 1 -- 172.21.15.8:0/3874945800 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e4c01a700 con 0x7f9e5c07bde0 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.212+0000 7f9e622e5700 1 -- 172.21.15.8:0/3874945800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c10be00 con 0x7f9e5c07bde0 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.212+0000 7f9e622e5700 1 -- 172.21.15.8:0/3874945800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c10bc70 con 0x7f9e5c07bde0 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.213+0000 7f9e597fa700 1 -- 172.21.15.8:0/3874945800 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f9e4c024430 con 0x7f9e5c07bde0 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.213+0000 7f9e622e5700 1 -- 172.21.15.8:0/3874945800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c062120 con 0x7f9e5c07bde0 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.214+0000 7f9e597fa700 1 --2- 172.21.15.8:0/3874945800 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f9e44038820 0x7f9e4403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.214+0000 7f9e597fa700 1 -- 172.21.15.8:0/3874945800 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9e4c04c750 con 0x7f9e5c07bde0 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.214+0000 7f9e5b7fe700 1 --2- 172.21.15.8:0/3874945800 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f9e44038820 0x7f9e4403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.214+0000 7f9e5b7fe700 1 --2- 172.21.15.8:0/3874945800 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f9e44038820 0x7f9e4403acd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e50006fd0 tx=0x7f9e50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.219+0000 7f9e597fa700 1 -- 172.21.15.8:0/3874945800 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9e4c0246e0 con 0x7f9e5c07bde0 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.427+0000 7f9e622e5700 1 -- 172.21.15.8:0/3874945800 --> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e5c1023a0 con 0x7f9e44038820 2023-11-27T15:16:17.161 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.686+0000 7f9e597fa700 1 -- 172.21.15.8:0/3874945800 <== mgr.14120 v2:172.21.15.8:6800/2324276308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9e5c1023a0 con 0x7f9e44038820 2023-11-27T15:16:17.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.688+0000 7f9e622e5700 1 -- 172.21.15.8:0/3874945800 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f9e44038820 msgr2=0x7f9e4403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.688+0000 7f9e622e5700 1 --2- 172.21.15.8:0/3874945800 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f9e44038820 0x7f9e4403acd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9e50006fd0 tx=0x7f9e50006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:17.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.688+0000 7f9e622e5700 1 -- 172.21.15.8:0/3874945800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 msgr2=0x7f9e5c19b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:17.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.688+0000 7f9e622e5700 1 --2- 172.21.15.8:0/3874945800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c19b7b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c006af0 tx=0x7f9e4c011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.688+0000 7f9e622e5700 1 -- 172.21.15.8:0/3874945800 shutdown_connections 2023-11-27T15:16:17.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.689+0000 7f9e622e5700 1 --2- 172.21.15.8:0/3874945800 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f9e44038820 0x7f9e4403acd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.689+0000 7f9e622e5700 1 --2- 172.21.15.8:0/3874945800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e5c07bde0 0x7f9e5c19b7b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:17.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.689+0000 7f9e622e5700 1 -- 172.21.15.8:0/3874945800 >> 172.21.15.8:0/3874945800 conn(0x7f9e5c100fa0 msgr2=0x7f9e5c101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:17.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.689+0000 7f9e622e5700 1 -- 172.21.15.8:0/3874945800 shutdown_connections 2023-11-27T15:16:17.162 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:16.689+0000 7f9e622e5700 1 -- 172.21.15.8:0/3874945800 wait complete. 2023-11-27T15:16:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:17 smithi008 ceph-mon[128788]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:17.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:17 smithi008 ceph-mon[128788]: Generating ssh key... 2023-11-27T15:16:17.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:17 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:17.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:17 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbveisYWttW783R4QIwCRUPLXfL4xU3wXqMnf1aTMwH6zYaeqHj/veOzOWpYvGO2v/j8AOF2kEHh8ce8fKho320KndoLsEMk1TsSM5tgORvy1MElVufSDL8HsRXtWuYZWB0oKRNpypZQ9pLkvnQ6j2tmR3BPsusIlf63pFORmP8GhBVyta8EQQ5sZn1K3qM6lGOUzgcit4PishJFWKW0iQY8RZ7LeoexO2vXxWyj+WkcIWj3x4I0U5w/PGagCdkjWT6VpVRGaLoao3ZlcUoQbm7WCgxEmR21g70jHHXMbef5xjp25GgNVCWmWo9gXQbDzT5g05LNbnMYIsSuTQlAHi4gHUBHfWmuuD7mqEXhn2kH0/fr3Babz3V9QanqbpR7kdnD103TgfxVu2jKmjv91zidjGZr1mRU4dR6jvk3DrQp/3G0WGcqHWOuuhpO6hf3MpoUE+vBFsv1gn32FUTqWt3B8M0gjwwI2V61cgasgUe5Vjv4uPKPI9Dl7YUVUlJU0= ceph-9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.732+0000 7fdbd9e13700 1 Processor -- start 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.733+0000 7fdbd9e13700 1 -- start start 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.733+0000 7fdbd9e13700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.733+0000 7fdbd9e13700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd407a820 con 0x7fdbd407bde0 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.733+0000 7fdbd37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.733+0000 7fdbd37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50878/0 (socket says 172.21.15.8:50878) 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.733+0000 7fdbd37fe700 1 -- 172.21.15.8:0/3849649610 learned_addr learned my addr 172.21.15.8:0/3849649610 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.734+0000 7fdbd37fe700 1 -- 172.21.15.8:0/3849649610 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd407a960 con 0x7fdbd407bde0 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.734+0000 7fdbd37fe700 1 --2- 172.21.15.8:0/3849649610 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd407a2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40098a0 tx=0x7fdbc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7178cb1e1cdccd4f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.734+0000 7fdbd27fc700 1 -- 172.21.15.8:0/3849649610 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbc4004160 con 0x7fdbd407bde0 2023-11-27T15:16:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.734+0000 7fdbd27fc700 1 -- 172.21.15.8:0/3849649610 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdbc40042c0 con 0x7fdbd407bde0 2023-11-27T15:16:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.734+0000 7fdbd27fc700 1 -- 172.21.15.8:0/3849649610 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbc4011a00 con 0x7fdbd407bde0 2023-11-27T15:16:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.735+0000 7fdbd9e13700 1 -- 172.21.15.8:0/3849649610 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 msgr2=0x7fdbd407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.735+0000 7fdbd9e13700 1 --2- 172.21.15.8:0/3849649610 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd407a2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40098a0 tx=0x7fdbc4009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.735+0000 7fdbd9e13700 1 -- 172.21.15.8:0/3849649610 shutdown_connections 2023-11-27T15:16:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.735+0000 7fdbd9e13700 1 --2- 172.21.15.8:0/3849649610 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd407a2e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.735+0000 7fdbd9e13700 1 -- 172.21.15.8:0/3849649610 >> 172.21.15.8:0/3849649610 conn(0x7fdbd4100fa0 msgr2=0x7fdbd41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.735+0000 7fdbd9e13700 1 -- 172.21.15.8:0/3849649610 shutdown_connections 2023-11-27T15:16:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.735+0000 7fdbd9e13700 1 -- 172.21.15.8:0/3849649610 wait complete. 2023-11-27T15:16:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.736+0000 7fdbd9e13700 1 Processor -- start 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.736+0000 7fdbd9e13700 1 -- start start 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.736+0000 7fdbd9e13700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.736+0000 7fdbd9e13700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd419bcf0 con 0x7fdbd407bde0 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.736+0000 7fdbd37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.737+0000 7fdbd37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50884/0 (socket says 172.21.15.8:50884) 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.737+0000 7fdbd37fe700 1 -- 172.21.15.8:0/1364120368 learned_addr learned my addr 172.21.15.8:0/1364120368 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.737+0000 7fdbd37fe700 1 -- 172.21.15.8:0/1364120368 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc4009550 con 0x7fdbd407bde0 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.737+0000 7fdbd37fe700 1 --2- 172.21.15.8:0/1364120368 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd419b7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4006af0 tx=0x7fdbc4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.737+0000 7fdbd0ff9700 1 -- 172.21.15.8:0/1364120368 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbc4003a40 con 0x7fdbd407bde0 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.737+0000 7fdbd0ff9700 1 -- 172.21.15.8:0/1364120368 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdbc4003ba0 con 0x7fdbd407bde0 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.737+0000 7fdbd0ff9700 1 -- 172.21.15.8:0/1364120368 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbc401a700 con 0x7fdbd407bde0 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.737+0000 7fdbd9e13700 1 -- 172.21.15.8:0/1364120368 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd410be00 con 0x7fdbd407bde0 2023-11-27T15:16:18.339 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.737+0000 7fdbd9e13700 1 -- 172.21.15.8:0/1364120368 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd410bc70 con 0x7fdbd407bde0 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.738+0000 7fdbd9e13700 1 -- 172.21.15.8:0/1364120368 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd410bf90 con 0x7fdbd407bde0 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.739+0000 7fdbd0ff9700 1 -- 172.21.15.8:0/1364120368 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fdbc4024ad0 con 0x7fdbd407bde0 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.739+0000 7fdbd0ff9700 1 --2- 172.21.15.8:0/1364120368 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fdbbc038820 0x7fdbbc03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.739+0000 7fdbd0ff9700 1 -- 172.21.15.8:0/1364120368 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdbc404c6b0 con 0x7fdbd407bde0 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.739+0000 7fdbd2ffd700 1 --2- 172.21.15.8:0/1364120368 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fdbbc038820 0x7fdbbc03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.740+0000 7fdbd2ffd700 1 --2- 172.21.15.8:0/1364120368 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fdbbc038820 0x7fdbbc03acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8006fd0 tx=0x7fdbc8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.744+0000 7fdbd0ff9700 1 -- 172.21.15.8:0/1364120368 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdbc4018b80 con 0x7fdbd407bde0 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.954+0000 7fdbd9e13700 1 -- 172.21.15.8:0/1364120368 --> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbd41023a0 con 0x7fdbbc038820 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.955+0000 7fdbd0ff9700 1 -- 172.21.15.8:0/1364120368 <== mgr.14120 v2:172.21.15.8:6800/2324276308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fdbd41023a0 con 0x7fdbbc038820 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.957+0000 7fdbd9e13700 1 -- 172.21.15.8:0/1364120368 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fdbbc038820 msgr2=0x7fdbbc03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.957+0000 7fdbd9e13700 1 --2- 172.21.15.8:0/1364120368 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fdbbc038820 0x7fdbbc03acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8006fd0 tx=0x7fdbc8006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:18.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.957+0000 7fdbd9e13700 1 -- 172.21.15.8:0/1364120368 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 msgr2=0x7fdbd419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:18.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.957+0000 7fdbd9e13700 1 --2- 172.21.15.8:0/1364120368 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd419b7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4006af0 tx=0x7fdbc4011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:18.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.957+0000 7fdbd9e13700 1 -- 172.21.15.8:0/1364120368 shutdown_connections 2023-11-27T15:16:18.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.957+0000 7fdbd9e13700 1 --2- 172.21.15.8:0/1364120368 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fdbbc038820 0x7fdbbc03acd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:18.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.957+0000 7fdbd9e13700 1 --2- 172.21.15.8:0/1364120368 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdbd407bde0 0x7fdbd419b7b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:18.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.957+0000 7fdbd9e13700 1 -- 172.21.15.8:0/1364120368 >> 172.21.15.8:0/1364120368 conn(0x7fdbd4100fa0 msgr2=0x7fdbd4101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:18.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.957+0000 7fdbd9e13700 1 -- 172.21.15.8:0/1364120368 shutdown_connections 2023-11-27T15:16:18.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:17.957+0000 7fdbd9e13700 1 -- 172.21.15.8:0/1364120368 wait complete. 2023-11-27T15:16:18.342 INFO:teuthology.orchestra.run.smithi008.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:16:18.342 INFO:teuthology.orchestra.run.smithi008.stdout:Adding key to root@localhost authorized_keys... 2023-11-27T15:16:18.342 INFO:teuthology.orchestra.run.smithi008.stdout:Adding host smithi008... 2023-11-27T15:16:19.109 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:18 smithi008 ceph-mon[128788]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:19.793 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:19 smithi008 ceph-mon[128788]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi008", "addr": "172.21.15.8", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:21.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:20 smithi008 ceph-mon[128788]: Deploying cephadm binary to smithi008 2023-11-27T15:16:22.535 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Added host 'smithi008' with addr '172.21.15.8' 2023-11-27T15:16:22.535 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.851+0000 7fd64d42a700 1 Processor -- start 2023-11-27T15:16:22.535 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.852+0000 7fd64d42a700 1 -- start start 2023-11-27T15:16:22.535 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.852+0000 7fd64d42a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.852+0000 7fd64d42a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64807a820 con 0x7fd64807bde0 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.852+0000 7fd646ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.852+0000 7fd646ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50900/0 (socket says 172.21.15.8:50900) 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.852+0000 7fd646ffd700 1 -- 172.21.15.8:0/960840010 learned_addr learned my addr 172.21.15.8:0/960840010 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.853+0000 7fd646ffd700 1 -- 172.21.15.8:0/960840010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64807a960 con 0x7fd64807bde0 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.853+0000 7fd646ffd700 1 --2- 172.21.15.8:0/960840010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64807a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6380098a0 tx=0x7fd638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bfd5070178890b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.853+0000 7fd645ffb700 1 -- 172.21.15.8:0/960840010 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd638004160 con 0x7fd64807bde0 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.853+0000 7fd645ffb700 1 -- 172.21.15.8:0/960840010 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd6380042c0 con 0x7fd64807bde0 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.853+0000 7fd645ffb700 1 -- 172.21.15.8:0/960840010 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd638011a00 con 0x7fd64807bde0 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.854+0000 7fd64d42a700 1 -- 172.21.15.8:0/960840010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 msgr2=0x7fd64807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.854+0000 7fd64d42a700 1 --2- 172.21.15.8:0/960840010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64807a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd6380098a0 tx=0x7fd638009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.854+0000 7fd64d42a700 1 -- 172.21.15.8:0/960840010 shutdown_connections 2023-11-27T15:16:22.536 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.854+0000 7fd64d42a700 1 --2- 172.21.15.8:0/960840010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64807a2e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.854+0000 7fd64d42a700 1 -- 172.21.15.8:0/960840010 >> 172.21.15.8:0/960840010 conn(0x7fd648100fa0 msgr2=0x7fd6481033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.854+0000 7fd64d42a700 1 -- 172.21.15.8:0/960840010 shutdown_connections 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.854+0000 7fd64d42a700 1 -- 172.21.15.8:0/960840010 wait complete. 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.855+0000 7fd64d42a700 1 Processor -- start 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.856+0000 7fd64d42a700 1 -- start start 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.856+0000 7fd64d42a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.856+0000 7fd64d42a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64819bcc0 con 0x7fd64807bde0 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.856+0000 7fd646ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.856+0000 7fd646ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50904/0 (socket says 172.21.15.8:50904) 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.856+0000 7fd646ffd700 1 -- 172.21.15.8:0/1622326056 learned_addr learned my addr 172.21.15.8:0/1622326056 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.857+0000 7fd646ffd700 1 -- 172.21.15.8:0/1622326056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd638009550 con 0x7fd64807bde0 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.857+0000 7fd646ffd700 1 --2- 172.21.15.8:0/1622326056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64819b780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd638006af0 tx=0x7fd638011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:22.537 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.857+0000 7fd62ffff700 1 -- 172.21.15.8:0/1622326056 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd638003a40 con 0x7fd64807bde0 2023-11-27T15:16:22.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.857+0000 7fd62ffff700 1 -- 172.21.15.8:0/1622326056 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd638003ba0 con 0x7fd64807bde0 2023-11-27T15:16:22.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.857+0000 7fd62ffff700 1 -- 172.21.15.8:0/1622326056 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd63801a700 con 0x7fd64807bde0 2023-11-27T15:16:22.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.857+0000 7fd64d42a700 1 -- 172.21.15.8:0/1622326056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64810be00 con 0x7fd64807bde0 2023-11-27T15:16:22.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.857+0000 7fd64d42a700 1 -- 172.21.15.8:0/1622326056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64810bc70 con 0x7fd64807bde0 2023-11-27T15:16:22.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.858+0000 7fd64d42a700 1 -- 172.21.15.8:0/1622326056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd648062120 con 0x7fd64807bde0 2023-11-27T15:16:22.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.859+0000 7fd62ffff700 1 -- 172.21.15.8:0/1622326056 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fd638024ad0 con 0x7fd64807bde0 2023-11-27T15:16:22.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.859+0000 7fd62ffff700 1 --2- 172.21.15.8:0/1622326056 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fd630038820 0x7fd63003acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:22.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.859+0000 7fd62ffff700 1 -- 172.21.15.8:0/1622326056 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd63804c780 con 0x7fd64807bde0 2023-11-27T15:16:22.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.859+0000 7fd6467fc700 1 --2- 172.21.15.8:0/1622326056 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fd630038820 0x7fd63003acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:22.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.860+0000 7fd6467fc700 1 --2- 172.21.15.8:0/1622326056 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fd630038820 0x7fd63003acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd63c006fd0 tx=0x7fd63c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:18.864+0000 7fd62ffff700 1 -- 172.21.15.8:0/1622326056 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd638018b80 con 0x7fd64807bde0 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:19.075+0000 7fd64d42a700 1 -- 172.21.15.8:0/1622326056 --> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi008", "addr": "172.21.15.8", "target": ["mon-mgr", ""]}) v1 -- 0x7fd648102390 con 0x7fd630038820 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.158+0000 7fd62ffff700 1 -- 172.21.15.8:0/1622326056 <== mgr.14120 v2:172.21.15.8:6800/2324276308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fd648102390 con 0x7fd630038820 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.161+0000 7fd64d42a700 1 -- 172.21.15.8:0/1622326056 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fd630038820 msgr2=0x7fd63003acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.161+0000 7fd64d42a700 1 --2- 172.21.15.8:0/1622326056 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fd630038820 0x7fd63003acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd63c006fd0 tx=0x7fd63c006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.161+0000 7fd64d42a700 1 -- 172.21.15.8:0/1622326056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 msgr2=0x7fd64819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.161+0000 7fd64d42a700 1 --2- 172.21.15.8:0/1622326056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64819b780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd638006af0 tx=0x7fd638011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.161+0000 7fd64d42a700 1 -- 172.21.15.8:0/1622326056 shutdown_connections 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.161+0000 7fd64d42a700 1 --2- 172.21.15.8:0/1622326056 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fd630038820 0x7fd63003acd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.161+0000 7fd64d42a700 1 --2- 172.21.15.8:0/1622326056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd64807bde0 0x7fd64819b780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.161+0000 7fd64d42a700 1 -- 172.21.15.8:0/1622326056 >> 172.21.15.8:0/1622326056 conn(0x7fd648100fa0 msgr2=0x7fd648101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.161+0000 7fd64d42a700 1 -- 172.21.15.8:0/1622326056 shutdown_connections 2023-11-27T15:16:22.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:22.161+0000 7fd64d42a700 1 -- 172.21.15.8:0/1622326056 wait complete. 2023-11-27T15:16:22.540 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying unmanaged mon service... 2023-11-27T15:16:23.507 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:23 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:23.507 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:23 smithi008 ceph-mon[128788]: Added host smithi008 2023-11-27T15:16:23.507 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:23 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:23.820 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-11-27T15:16:23.820 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.087+0000 7f7577d15700 1 Processor -- start 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.087+0000 7f7577d15700 1 -- start start 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.088+0000 7f7577d15700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.088+0000 7f7577d15700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757007a800 con 0x7f757007bdc0 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.088+0000 7f7575ab1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.088+0000 7f7575ab1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33500/0 (socket says 172.21.15.8:33500) 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.088+0000 7f7575ab1700 1 -- 172.21.15.8:0/3753858985 learned_addr learned my addr 172.21.15.8:0/3753858985 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.089+0000 7f7575ab1700 1 -- 172.21.15.8:0/3753858985 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757007a940 con 0x7f757007bdc0 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.089+0000 7f7575ab1700 1 --2- 172.21.15.8:0/3753858985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757007a2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f756c0098a0 tx=0x7f756c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d704217f46bf87 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.089+0000 7f7574aaf700 1 -- 172.21.15.8:0/3753858985 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f756c004160 con 0x7f757007bdc0 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.089+0000 7f7574aaf700 1 -- 172.21.15.8:0/3753858985 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f756c0042c0 con 0x7f757007bdc0 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.089+0000 7f7574aaf700 1 -- 172.21.15.8:0/3753858985 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f756c011a00 con 0x7f757007bdc0 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.090+0000 7f7577d15700 1 -- 172.21.15.8:0/3753858985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 msgr2=0x7f757007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.090+0000 7f7577d15700 1 --2- 172.21.15.8:0/3753858985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757007a2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f756c0098a0 tx=0x7f756c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.090+0000 7f7577d15700 1 -- 172.21.15.8:0/3753858985 shutdown_connections 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.090+0000 7f7577d15700 1 --2- 172.21.15.8:0/3753858985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757007a2c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.090+0000 7f7577d15700 1 -- 172.21.15.8:0/3753858985 >> 172.21.15.8:0/3753858985 conn(0x7f7570100fa0 msgr2=0x7f75701033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.090+0000 7f7577d15700 1 -- 172.21.15.8:0/3753858985 shutdown_connections 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.090+0000 7f7577d15700 1 -- 172.21.15.8:0/3753858985 wait complete. 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.091+0000 7f7577d15700 1 Processor -- start 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.092+0000 7f7577d15700 1 -- start start 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.092+0000 7f7577d15700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.092+0000 7f7577d15700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757019bcc0 con 0x7f757007bdc0 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.092+0000 7f7575ab1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.092+0000 7f7575ab1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757019b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33516/0 (socket says 172.21.15.8:33516) 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.092+0000 7f7575ab1700 1 -- 172.21.15.8:0/2164159080 learned_addr learned my addr 172.21.15.8:0/2164159080 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:23.822 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.093+0000 7f7575ab1700 1 -- 172.21.15.8:0/2164159080 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c009550 con 0x7f757007bdc0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.093+0000 7f7575ab1700 1 --2- 172.21.15.8:0/2164159080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757019b780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f756c006af0 tx=0x7f756c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.093+0000 7f7566ffd700 1 -- 172.21.15.8:0/2164159080 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f756c003a40 con 0x7f757007bdc0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.093+0000 7f7577d15700 1 -- 172.21.15.8:0/2164159080 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757010bdc0 con 0x7f757007bdc0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.093+0000 7f7566ffd700 1 -- 172.21.15.8:0/2164159080 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f756c003ba0 con 0x7f757007bdc0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.093+0000 7f7566ffd700 1 -- 172.21.15.8:0/2164159080 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f756c01a700 con 0x7f757007bdc0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.093+0000 7f7577d15700 1 -- 172.21.15.8:0/2164159080 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757010bc30 con 0x7f757007bdc0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.094+0000 7f7566ffd700 1 -- 172.21.15.8:0/2164159080 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f756c024ad0 con 0x7f757007bdc0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.094+0000 7f7577d15700 1 -- 172.21.15.8:0/2164159080 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7570062120 con 0x7f757007bdc0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.095+0000 7f7566ffd700 1 --2- 172.21.15.8:0/2164159080 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f755c038820 0x7f755c03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.095+0000 7f7566ffd700 1 -- 172.21.15.8:0/2164159080 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f756c04c780 con 0x7f757007bdc0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.095+0000 7f75752b0700 1 --2- 172.21.15.8:0/2164159080 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f755c038820 0x7f755c03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.096+0000 7f75752b0700 1 --2- 172.21.15.8:0/2164159080 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f755c038820 0x7f755c03acd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7560006fd0 tx=0x7f7560006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.100+0000 7f7566ffd700 1 -- 172.21.15.8:0/2164159080 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f756c018b80 con 0x7f757007bdc0 2023-11-27T15:16:23.823 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.327+0000 7f7577d15700 1 -- 172.21.15.8:0/2164159080 --> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7570102390 con 0x7f755c038820 2023-11-27T15:16:23.824 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.333+0000 7f7566ffd700 1 -- 172.21.15.8:0/2164159080 <== mgr.14120 v2:172.21.15.8:6800/2324276308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7570102390 con 0x7f755c038820 2023-11-27T15:16:23.824 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.336+0000 7f7577d15700 1 -- 172.21.15.8:0/2164159080 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f755c038820 msgr2=0x7f755c03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:23.824 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.336+0000 7f7577d15700 1 --2- 172.21.15.8:0/2164159080 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f755c038820 0x7f755c03acd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7560006fd0 tx=0x7f7560006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:23.824 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.336+0000 7f7577d15700 1 -- 172.21.15.8:0/2164159080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 msgr2=0x7f757019b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:23.824 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.336+0000 7f7577d15700 1 --2- 172.21.15.8:0/2164159080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757019b780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f756c006af0 tx=0x7f756c011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:23.824 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.336+0000 7f7577d15700 1 -- 172.21.15.8:0/2164159080 shutdown_connections 2023-11-27T15:16:23.824 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.336+0000 7f7577d15700 1 --2- 172.21.15.8:0/2164159080 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f755c038820 0x7f755c03acd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:23.824 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.336+0000 7f7577d15700 1 --2- 172.21.15.8:0/2164159080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f757007bdc0 0x7f757019b780 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:23.824 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.336+0000 7f7577d15700 1 -- 172.21.15.8:0/2164159080 >> 172.21.15.8:0/2164159080 conn(0x7f7570100fa0 msgr2=0x7f7570101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:23.824 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.336+0000 7f7577d15700 1 -- 172.21.15.8:0/2164159080 shutdown_connections 2023-11-27T15:16:23.825 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:23.336+0000 7f7577d15700 1 -- 172.21.15.8:0/2164159080 wait complete. 2023-11-27T15:16:23.825 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying unmanaged mgr service... 2023-11-27T15:16:24.508 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:24 smithi008 ceph-mon[128788]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:24.508 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:24 smithi008 ceph-mon[128788]: Saving service mon spec with placement count:5 2023-11-27T15:16:24.508 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:24 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:25.228 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-11-27T15:16:25.228 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.517+0000 7f17e1196700 1 Processor -- start 2023-11-27T15:16:25.228 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.517+0000 7f17e1196700 1 -- start start 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.518+0000 7f17e1196700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.518+0000 7f17e1196700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc07a840 con 0x7f17dc07be00 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.518+0000 7f17dad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.518+0000 7f17dad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33528/0 (socket says 172.21.15.8:33528) 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.518+0000 7f17dad9d700 1 -- 172.21.15.8:0/3245537570 learned_addr learned my addr 172.21.15.8:0/3245537570 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.519+0000 7f17dad9d700 1 -- 172.21.15.8:0/3245537570 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc07a980 con 0x7f17dc07be00 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.519+0000 7f17dad9d700 1 --2- 172.21.15.8:0/3245537570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc07a300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0098a0 tx=0x7f17cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a026dc1e2e7a83 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.519+0000 7f17d9d9b700 1 -- 172.21.15.8:0/3245537570 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17cc004160 con 0x7f17dc07be00 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.519+0000 7f17d9d9b700 1 -- 172.21.15.8:0/3245537570 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17cc0042c0 con 0x7f17dc07be00 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.519+0000 7f17d9d9b700 1 -- 172.21.15.8:0/3245537570 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17cc011a00 con 0x7f17dc07be00 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.520+0000 7f17e1196700 1 -- 172.21.15.8:0/3245537570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 msgr2=0x7f17dc07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.520+0000 7f17e1196700 1 --2- 172.21.15.8:0/3245537570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc07a300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0098a0 tx=0x7f17cc009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.229 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.520+0000 7f17e1196700 1 -- 172.21.15.8:0/3245537570 shutdown_connections 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.520+0000 7f17e1196700 1 --2- 172.21.15.8:0/3245537570 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc07a300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.520+0000 7f17e1196700 1 -- 172.21.15.8:0/3245537570 >> 172.21.15.8:0/3245537570 conn(0x7f17dc100fa0 msgr2=0x7f17dc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.520+0000 7f17e1196700 1 -- 172.21.15.8:0/3245537570 shutdown_connections 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.520+0000 7f17e1196700 1 -- 172.21.15.8:0/3245537570 wait complete. 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.521+0000 7f17e1196700 1 Processor -- start 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.521+0000 7f17e1196700 1 -- start start 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.521+0000 7f17e1196700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.521+0000 7f17e1196700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc19bce0 con 0x7f17dc07be00 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.522+0000 7f17dad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.522+0000 7f17dad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc19b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33538/0 (socket says 172.21.15.8:33538) 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.522+0000 7f17dad9d700 1 -- 172.21.15.8:0/448118293 learned_addr learned my addr 172.21.15.8:0/448118293 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:25.230 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.522+0000 7f17dad9d700 1 -- 172.21.15.8:0/448118293 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc009550 con 0x7f17dc07be00 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.522+0000 7f17dad9d700 1 --2- 172.21.15.8:0/448118293 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc19b7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f17cc006b20 tx=0x7f17cc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.522+0000 7f17c3fff700 1 -- 172.21.15.8:0/448118293 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17cc003b70 con 0x7f17dc07be00 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.522+0000 7f17c3fff700 1 -- 172.21.15.8:0/448118293 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17cc003cd0 con 0x7f17dc07be00 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.522+0000 7f17e1196700 1 -- 172.21.15.8:0/448118293 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc10bde0 con 0x7f17dc07be00 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.522+0000 7f17c3fff700 1 -- 172.21.15.8:0/448118293 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17cc0247c0 con 0x7f17dc07be00 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.523+0000 7f17e1196700 1 -- 172.21.15.8:0/448118293 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc10bc50 con 0x7f17dc07be00 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.524+0000 7f17c3fff700 1 -- 172.21.15.8:0/448118293 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f17cc01ab40 con 0x7f17dc07be00 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.524+0000 7f17e1196700 1 -- 172.21.15.8:0/448118293 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17dc062120 con 0x7f17dc07be00 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.524+0000 7f17c3fff700 1 --2- 172.21.15.8:0/448118293 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f17c4038820 0x7f17c403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.524+0000 7f17c3fff700 1 -- 172.21.15.8:0/448118293 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f17cc04c8e0 con 0x7f17dc07be00 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.524+0000 7f17da59c700 1 --2- 172.21.15.8:0/448118293 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f17c4038820 0x7f17c403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.525+0000 7f17da59c700 1 --2- 172.21.15.8:0/448118293 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f17c4038820 0x7f17c403acd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f17d0006fd0 tx=0x7f17d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:25.231 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.529+0000 7f17c3fff700 1 -- 172.21.15.8:0/448118293 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f17cc01a950 con 0x7f17dc07be00 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.716+0000 7f17c3fff700 1 -- 172.21.15.8:0/448118293 <== mon.0 v2:172.21.15.8:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17cc04c2f0 con 0x7f17dc07be00 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.755+0000 7f17e1196700 1 -- 172.21.15.8:0/448118293 --> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f17dc102380 con 0x7f17c4038820 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.760+0000 7f17c3fff700 1 -- 172.21.15.8:0/448118293 <== mgr.14120 v2:172.21.15.8:6800/2324276308 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f17dc102380 con 0x7f17c4038820 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.762+0000 7f17e1196700 1 -- 172.21.15.8:0/448118293 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f17c4038820 msgr2=0x7f17c403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.762+0000 7f17e1196700 1 --2- 172.21.15.8:0/448118293 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f17c4038820 0x7f17c403acd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f17d0006fd0 tx=0x7f17d0006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.763+0000 7f17e1196700 1 -- 172.21.15.8:0/448118293 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 msgr2=0x7f17dc19b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.763+0000 7f17e1196700 1 --2- 172.21.15.8:0/448118293 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc19b7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f17cc006b20 tx=0x7f17cc011ad0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.763+0000 7f17e1196700 1 -- 172.21.15.8:0/448118293 shutdown_connections 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.763+0000 7f17e1196700 1 --2- 172.21.15.8:0/448118293 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f17c4038820 0x7f17c403acd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.763+0000 7f17e1196700 1 --2- 172.21.15.8:0/448118293 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f17dc07be00 0x7f17dc19b7a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.763+0000 7f17e1196700 1 -- 172.21.15.8:0/448118293 >> 172.21.15.8:0/448118293 conn(0x7f17dc100fa0 msgr2=0x7f17dc101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.763+0000 7f17e1196700 1 -- 172.21.15.8:0/448118293 shutdown_connections 2023-11-27T15:16:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:24.763+0000 7f17e1196700 1 -- 172.21.15.8:0/448118293 wait complete. 2023-11-27T15:16:26.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:25 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:26.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:25 smithi008 ceph-mon[128788]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:26.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:25 smithi008 ceph-mon[128788]: Saving service mgr spec with placement count:2 2023-11-27T15:16:26.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:25 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:26.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:25 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:26.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.761+0000 7f2865bf1700 1 Processor -- start 2023-11-27T15:16:26.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.762+0000 7f2865bf1700 1 -- start start 2023-11-27T15:16:26.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.762+0000 7f2865bf1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:26.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.762+0000 7f2865bf1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286007a800 con 0x7f286007bdc0 2023-11-27T15:16:26.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.762+0000 7f285f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:26.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.762+0000 7f285f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33546/0 (socket says 172.21.15.8:33546) 2023-11-27T15:16:26.608 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.762+0000 7f285f7fe700 1 -- 172.21.15.8:0/2924493204 learned_addr learned my addr 172.21.15.8:0/2924493204 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:26.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.763+0000 7f285f7fe700 1 -- 172.21.15.8:0/2924493204 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286007a940 con 0x7f286007bdc0 2023-11-27T15:16:26.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.763+0000 7f285f7fe700 1 --2- 172.21.15.8:0/2924493204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286007a2c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f28500098a0 tx=0x7f2850009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9157a081da7a948 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:26.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.763+0000 7f285e7fc700 1 -- 172.21.15.8:0/2924493204 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2850004160 con 0x7f286007bdc0 2023-11-27T15:16:26.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.763+0000 7f285e7fc700 1 -- 172.21.15.8:0/2924493204 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28500042c0 con 0x7f286007bdc0 2023-11-27T15:16:26.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.763+0000 7f285e7fc700 1 -- 172.21.15.8:0/2924493204 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28500119b0 con 0x7f286007bdc0 2023-11-27T15:16:26.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.764+0000 7f2865bf1700 1 -- 172.21.15.8:0/2924493204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 msgr2=0x7f286007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:26.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.764+0000 7f2865bf1700 1 --2- 172.21.15.8:0/2924493204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286007a2c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f28500098a0 tx=0x7f2850009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:26.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.764+0000 7f2865bf1700 1 -- 172.21.15.8:0/2924493204 shutdown_connections 2023-11-27T15:16:26.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.764+0000 7f2865bf1700 1 --2- 172.21.15.8:0/2924493204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286007a2c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:26.609 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.764+0000 7f2865bf1700 1 -- 172.21.15.8:0/2924493204 >> 172.21.15.8:0/2924493204 conn(0x7f2860100fa0 msgr2=0x7f28601033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:26.610 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.764+0000 7f2865bf1700 1 -- 172.21.15.8:0/2924493204 shutdown_connections 2023-11-27T15:16:26.610 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.764+0000 7f2865bf1700 1 -- 172.21.15.8:0/2924493204 wait complete. 2023-11-27T15:16:26.610 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.765+0000 7f2865bf1700 1 Processor -- start 2023-11-27T15:16:26.610 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.765+0000 7f2865bf1700 1 -- start start 2023-11-27T15:16:26.610 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.766+0000 7f2865bf1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:26.610 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.766+0000 7f2865bf1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286019bd00 con 0x7f286007bdc0 2023-11-27T15:16:26.610 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.766+0000 7f285f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:26.610 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.766+0000 7f285f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286019b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33550/0 (socket says 172.21.15.8:33550) 2023-11-27T15:16:26.610 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.766+0000 7f285f7fe700 1 -- 172.21.15.8:0/711944953 learned_addr learned my addr 172.21.15.8:0/711944953 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.766+0000 7f285f7fe700 1 -- 172.21.15.8:0/711944953 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2850009550 con 0x7f286007bdc0 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.766+0000 7f285f7fe700 1 --2- 172.21.15.8:0/711944953 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286019b7c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2850000c00 tx=0x7f2850011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.767+0000 7f285cff9700 1 -- 172.21.15.8:0/711944953 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28500039f0 con 0x7f286007bdc0 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.767+0000 7f285cff9700 1 -- 172.21.15.8:0/711944953 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2850003b50 con 0x7f286007bdc0 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.767+0000 7f2865bf1700 1 -- 172.21.15.8:0/711944953 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286010be10 con 0x7f286007bdc0 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.767+0000 7f285cff9700 1 -- 172.21.15.8:0/711944953 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f285001a780 con 0x7f286007bdc0 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.767+0000 7f2865bf1700 1 -- 172.21.15.8:0/711944953 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286010bc80 con 0x7f286007bdc0 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.768+0000 7f2865bf1700 1 -- 172.21.15.8:0/711944953 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286010bfa0 con 0x7f286007bdc0 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.768+0000 7f285cff9700 1 -- 172.21.15.8:0/711944953 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f285001a9f0 con 0x7f286007bdc0 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.769+0000 7f285cff9700 1 --2- 172.21.15.8:0/711944953 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2848038820 0x7f284803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.769+0000 7f285cff9700 1 -- 172.21.15.8:0/711944953 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f285004c970 con 0x7f286007bdc0 2023-11-27T15:16:26.611 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.769+0000 7f285effd700 1 --2- 172.21.15.8:0/711944953 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2848038820 0x7f284803acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.770+0000 7f285effd700 1 --2- 172.21.15.8:0/711944953 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2848038820 0x7f284803acd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2854006fd0 tx=0x7f2854006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.774+0000 7f285cff9700 1 -- 172.21.15.8:0/711944953 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f285001aca0 con 0x7f286007bdc0 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.984+0000 7f2865bf1700 1 -- 172.21.15.8:0/711944953 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f286019f3d0 con 0x7f286007bdc0 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.988+0000 7f285cff9700 1 -- 172.21.15.8:0/711944953 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f28500187c0 con 0x7f286007bdc0 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.990+0000 7f2865bf1700 1 -- 172.21.15.8:0/711944953 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2848038820 msgr2=0x7f284803acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.990+0000 7f2865bf1700 1 --2- 172.21.15.8:0/711944953 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2848038820 0x7f284803acd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2854006fd0 tx=0x7f2854006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.990+0000 7f2865bf1700 1 -- 172.21.15.8:0/711944953 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 msgr2=0x7f286019b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.990+0000 7f2865bf1700 1 --2- 172.21.15.8:0/711944953 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286019b7c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2850000c00 tx=0x7f2850011a30 comp rx=0 tx=0).stop 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.990+0000 7f2865bf1700 1 -- 172.21.15.8:0/711944953 shutdown_connections 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.990+0000 7f2865bf1700 1 --2- 172.21.15.8:0/711944953 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2848038820 0x7f284803acd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.990+0000 7f2865bf1700 1 --2- 172.21.15.8:0/711944953 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f286007bdc0 0x7f286019b7c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.991+0000 7f2865bf1700 1 -- 172.21.15.8:0/711944953 >> 172.21.15.8:0/711944953 conn(0x7f2860100fa0 msgr2=0x7f2860101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:26.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.991+0000 7f2865bf1700 1 -- 172.21.15.8:0/711944953 shutdown_connections 2023-11-27T15:16:26.613 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:25.991+0000 7f2865bf1700 1 -- 172.21.15.8:0/711944953 wait complete. 2023-11-27T15:16:27.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:26 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/711944953' entity='client.admin' 2023-11-27T15:16:27.994 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.138+0000 7f2122478700 1 Processor -- start 2023-11-27T15:16:27.994 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.139+0000 7f2122478700 1 -- start start 2023-11-27T15:16:27.994 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.139+0000 7f2122478700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c108840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:27.994 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.139+0000 7f2122478700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c079d20 con 0x7f211c106460 2023-11-27T15:16:27.994 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.139+0000 7f211bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c108840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:27.994 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.139+0000 7f211bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c108840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33560/0 (socket says 172.21.15.8:33560) 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.139+0000 7f211bfff700 1 -- 172.21.15.8:0/4288884343 learned_addr learned my addr 172.21.15.8:0/4288884343 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.140+0000 7f211bfff700 1 -- 172.21.15.8:0/4288884343 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c079e60 con 0x7f211c106460 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.140+0000 7f211bfff700 1 --2- 172.21.15.8:0/4288884343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c108840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f210c0098a0 tx=0x7f210c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79168e600b620c33 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.140+0000 7f211affd700 1 -- 172.21.15.8:0/4288884343 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f210c004160 con 0x7f211c106460 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.140+0000 7f211affd700 1 -- 172.21.15.8:0/4288884343 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f210c0042c0 con 0x7f211c106460 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.140+0000 7f211affd700 1 -- 172.21.15.8:0/4288884343 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f210c0119b0 con 0x7f211c106460 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.141+0000 7f2122478700 1 -- 172.21.15.8:0/4288884343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 msgr2=0x7f211c108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.141+0000 7f2122478700 1 --2- 172.21.15.8:0/4288884343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c108840 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f210c0098a0 tx=0x7f210c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.141+0000 7f2122478700 1 -- 172.21.15.8:0/4288884343 shutdown_connections 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.141+0000 7f2122478700 1 --2- 172.21.15.8:0/4288884343 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c108840 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.141+0000 7f2122478700 1 -- 172.21.15.8:0/4288884343 >> 172.21.15.8:0/4288884343 conn(0x7f211c0ffe40 msgr2=0x7f211c102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:27.995 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.141+0000 7f2122478700 1 -- 172.21.15.8:0/4288884343 shutdown_connections 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.141+0000 7f2122478700 1 -- 172.21.15.8:0/4288884343 wait complete. 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.142+0000 7f2122478700 1 Processor -- start 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.142+0000 7f2122478700 1 -- start start 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.143+0000 7f2122478700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.143+0000 7f2122478700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c19bcc0 con 0x7f211c106460 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.143+0000 7f211bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.143+0000 7f211bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33568/0 (socket says 172.21.15.8:33568) 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.143+0000 7f211bfff700 1 -- 172.21.15.8:0/3254873478 learned_addr learned my addr 172.21.15.8:0/3254873478 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.143+0000 7f211bfff700 1 -- 172.21.15.8:0/3254873478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c009550 con 0x7f211c106460 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.143+0000 7f211bfff700 1 --2- 172.21.15.8:0/3254873478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c19b780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f210c000c00 tx=0x7f210c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.144+0000 7f21197fa700 1 -- 172.21.15.8:0/3254873478 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f210c0039f0 con 0x7f211c106460 2023-11-27T15:16:27.996 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.144+0000 7f21197fa700 1 -- 172.21.15.8:0/3254873478 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f210c003b50 con 0x7f211c106460 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.144+0000 7f2122478700 1 -- 172.21.15.8:0/3254873478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c109bc0 con 0x7f211c106460 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.144+0000 7f21197fa700 1 -- 172.21.15.8:0/3254873478 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f210c01a780 con 0x7f211c106460 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.144+0000 7f2122478700 1 -- 172.21.15.8:0/3254873478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c109a30 con 0x7f211c106460 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.145+0000 7f21197fa700 1 -- 172.21.15.8:0/3254873478 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f210c024440 con 0x7f211c106460 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.145+0000 7f2122478700 1 -- 172.21.15.8:0/3254873478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c109d50 con 0x7f211c106460 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.145+0000 7f21197fa700 1 --2- 172.21.15.8:0/3254873478 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2104038820 0x7f210403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.145+0000 7f21197fa700 1 -- 172.21.15.8:0/3254873478 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f210c04c820 con 0x7f211c106460 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.146+0000 7f211b7fe700 1 --2- 172.21.15.8:0/3254873478 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2104038820 0x7f210403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.146+0000 7f211b7fe700 1 --2- 172.21.15.8:0/3254873478 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2104038820 0x7f210403acd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2110006fd0 tx=0x7f2110006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.150+0000 7f21197fa700 1 -- 172.21.15.8:0/3254873478 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f210c0246f0 con 0x7f211c106460 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.366+0000 7f2122478700 1 -- 172.21.15.8:0/3254873478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f211c19f400 con 0x7f211c106460 2023-11-27T15:16:27.997 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.370+0000 7f21197fa700 1 -- 172.21.15.8:0/3254873478 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f210c018350 con 0x7f211c106460 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.372+0000 7f2122478700 1 -- 172.21.15.8:0/3254873478 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2104038820 msgr2=0x7f210403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.372+0000 7f2122478700 1 --2- 172.21.15.8:0/3254873478 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2104038820 0x7f210403acd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2110006fd0 tx=0x7f2110006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.372+0000 7f2122478700 1 -- 172.21.15.8:0/3254873478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 msgr2=0x7f211c19b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.372+0000 7f2122478700 1 --2- 172.21.15.8:0/3254873478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c19b780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f210c000c00 tx=0x7f210c011a30 comp rx=0 tx=0).stop 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.372+0000 7f2122478700 1 -- 172.21.15.8:0/3254873478 shutdown_connections 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.372+0000 7f2122478700 1 --2- 172.21.15.8:0/3254873478 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f2104038820 0x7f210403acd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.372+0000 7f2122478700 1 --2- 172.21.15.8:0/3254873478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c106460 0x7f211c19b780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.372+0000 7f2122478700 1 -- 172.21.15.8:0/3254873478 >> 172.21.15.8:0/3254873478 conn(0x7f211c0ffe40 msgr2=0x7f211c100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.373+0000 7f2122478700 1 -- 172.21.15.8:0/3254873478 shutdown_connections 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:27.373+0000 7f2122478700 1 -- 172.21.15.8:0/3254873478 wait complete. 2023-11-27T15:16:27.998 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling the dashboard module... 2023-11-27T15:16:28.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:28 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3254873478' entity='client.admin' 2023-11-27T15:16:28.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:28 smithi008 ceph-mon[128788]: from='mgr.14120 172.21.15.8:0/3538998608' entity='mgr.x' 2023-11-27T15:16:29.708 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:29 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4000740522' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-11-27T15:16:29.708 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:29 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: ignoring --setuser ceph since I am not root 2023-11-27T15:16:29.709 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:29 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: ignoring --setgroup ceph since I am not root 2023-11-27T15:16:29.709 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:29 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:29.703+0000 7f956d813200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:16:29.888 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.587+0000 7f77b5c41700 1 Processor -- start 2023-11-27T15:16:29.888 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.588+0000 7f77b5c41700 1 -- start start 2023-11-27T15:16:29.888 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.588+0000 7f77b5c41700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0079d00 0x7f77b0108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:29.888 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.588+0000 7f77b5c41700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b01089d0 con 0x7f77b0079d00 2023-11-27T15:16:29.888 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.588+0000 7f77af7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0079d00 0x7f77b0108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:29.888 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.589+0000 7f77af7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0079d00 0x7f77b0108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33574/0 (socket says 172.21.15.8:33574) 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.589+0000 7f77af7fe700 1 -- 172.21.15.8:0/398924013 learned_addr learned my addr 172.21.15.8:0/398924013 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.589+0000 7f77af7fe700 1 -- 172.21.15.8:0/398924013 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b0108b10 con 0x7f77b0079d00 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.589+0000 7f77af7fe700 1 --2- 172.21.15.8:0/398924013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0079d00 0x7f77b0108490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77a00098a0 tx=0x7f77a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5be2e1f5aca084 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.589+0000 7f77ae7fc700 1 -- 172.21.15.8:0/398924013 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77a0004160 con 0x7f77b0079d00 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.589+0000 7f77ae7fc700 1 -- 172.21.15.8:0/398924013 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77a00042c0 con 0x7f77b0079d00 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.589+0000 7f77ae7fc700 1 -- 172.21.15.8:0/398924013 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77a00119b0 con 0x7f77b0079d00 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.590+0000 7f77b5c41700 1 -- 172.21.15.8:0/398924013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0079d00 msgr2=0x7f77b0108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.590+0000 7f77b5c41700 1 --2- 172.21.15.8:0/398924013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0079d00 0x7f77b0108490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77a00098a0 tx=0x7f77a0009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.590+0000 7f77b5c41700 1 -- 172.21.15.8:0/398924013 shutdown_connections 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.590+0000 7f77b5c41700 1 --2- 172.21.15.8:0/398924013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0079d00 0x7f77b0108490 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.590+0000 7f77b5c41700 1 -- 172.21.15.8:0/398924013 >> 172.21.15.8:0/398924013 conn(0x7f77b00ffe40 msgr2=0x7f77b0102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.590+0000 7f77b5c41700 1 -- 172.21.15.8:0/398924013 shutdown_connections 2023-11-27T15:16:29.889 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.590+0000 7f77b5c41700 1 -- 172.21.15.8:0/398924013 wait complete. 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.591+0000 7f77b5c41700 1 Processor -- start 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.591+0000 7f77b5c41700 1 -- start start 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.592+0000 7f77b5c41700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0199530 0x7f77b0199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.592+0000 7f77b5c41700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b0199e80 con 0x7f77b0199530 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.592+0000 7f77af7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0199530 0x7f77b0199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.592+0000 7f77af7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0199530 0x7f77b0199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33588/0 (socket says 172.21.15.8:33588) 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.592+0000 7f77af7fe700 1 -- 172.21.15.8:0/4000740522 learned_addr learned my addr 172.21.15.8:0/4000740522 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.592+0000 7f77af7fe700 1 -- 172.21.15.8:0/4000740522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0009550 con 0x7f77b0199530 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.592+0000 7f77af7fe700 1 --2- 172.21.15.8:0/4000740522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0199530 0x7f77b0199940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77a0000c00 tx=0x7f77a0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.593+0000 7f77acff9700 1 -- 172.21.15.8:0/4000740522 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77a00039f0 con 0x7f77b0199530 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.593+0000 7f77acff9700 1 -- 172.21.15.8:0/4000740522 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77a0003b50 con 0x7f77b0199530 2023-11-27T15:16:29.890 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.593+0000 7f77acff9700 1 -- 172.21.15.8:0/4000740522 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77a001a780 con 0x7f77b0199530 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.593+0000 7f77b5c41700 1 -- 172.21.15.8:0/4000740522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b0109ba0 con 0x7f77b0199530 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.593+0000 7f77b5c41700 1 -- 172.21.15.8:0/4000740522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b0109a10 con 0x7f77b0199530 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.594+0000 7f77acff9700 1 -- 172.21.15.8:0/4000740522 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f77a0024440 con 0x7f77b0199530 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.594+0000 7f77b5c41700 1 -- 172.21.15.8:0/4000740522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b0062120 con 0x7f77b0199530 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.594+0000 7f77acff9700 1 --2- 172.21.15.8:0/4000740522 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f7798038820 0x7f779803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.594+0000 7f77acff9700 1 -- 172.21.15.8:0/4000740522 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f77a004c840 con 0x7f77b0199530 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.595+0000 7f77aeffd700 1 --2- 172.21.15.8:0/4000740522 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f7798038820 0x7f779803acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.595+0000 7f77aeffd700 1 --2- 172.21.15.8:0/4000740522 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f7798038820 0x7f779803acd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f77a4006fd0 tx=0x7f77a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.599+0000 7f77acff9700 1 -- 172.21.15.8:0/4000740522 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f77a0018d90 con 0x7f77b0199530 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:28.888+0000 7f77b5c41700 1 -- 172.21.15.8:0/4000740522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f77b0079d00 con 0x7f77b0199530 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.373+0000 7f77acff9700 1 -- 172.21.15.8:0/4000740522 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f77a002d910 con 0x7f77b0199530 2023-11-27T15:16:29.891 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.376+0000 7f77acff9700 1 -- 172.21.15.8:0/4000740522 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f77a004c190 con 0x7f77b0199530 2023-11-27T15:16:29.892 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.379+0000 7f77b5c41700 1 -- 172.21.15.8:0/4000740522 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f7798038820 msgr2=0x7f779803acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:29.892 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.379+0000 7f77b5c41700 1 --2- 172.21.15.8:0/4000740522 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f7798038820 0x7f779803acd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f77a4006fd0 tx=0x7f77a4006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:29.892 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.379+0000 7f77b5c41700 1 -- 172.21.15.8:0/4000740522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0199530 msgr2=0x7f77b0199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:29.892 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.379+0000 7f77b5c41700 1 --2- 172.21.15.8:0/4000740522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0199530 0x7f77b0199940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77a0000c00 tx=0x7f77a0011a30 comp rx=0 tx=0).stop 2023-11-27T15:16:29.892 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.379+0000 7f77b5c41700 1 -- 172.21.15.8:0/4000740522 shutdown_connections 2023-11-27T15:16:29.892 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.379+0000 7f77b5c41700 1 --2- 172.21.15.8:0/4000740522 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f7798038820 0x7f779803acd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:29.892 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.379+0000 7f77b5c41700 1 --2- 172.21.15.8:0/4000740522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77b0199530 0x7f77b0199940 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:29.892 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.379+0000 7f77b5c41700 1 -- 172.21.15.8:0/4000740522 >> 172.21.15.8:0/4000740522 conn(0x7f77b00ffe40 msgr2=0x7f77b0100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:29.892 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.379+0000 7f77b5c41700 1 -- 172.21.15.8:0/4000740522 shutdown_connections 2023-11-27T15:16:29.892 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:29.379+0000 7f77b5c41700 1 -- 172.21.15.8:0/4000740522 wait complete. 2023-11-27T15:16:29.995 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:29 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:29.810+0000 7f956d813200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:16:29.995 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:29 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:29.933+0000 7f956d813200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:16:30.246 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:30 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:30.073+0000 7f956d813200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:16:30.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:30 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4000740522' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-11-27T15:16:30.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:30 smithi008 ceph-mon[128788]: mgrmap e9: x(active, since 17s) 2023-11-27T15:16:30.614 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:30 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:30.318+0000 7f956d813200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:16:31.031 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:30 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:30.726+0000 7f956d813200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:16:31.031 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:30 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:30.854+0000 7f956d813200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:16:31.032 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:31 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:31.027+0000 7f956d813200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:16:31.179 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:16:31.179 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-11-27T15:16:31.179 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-27T15:16:31.179 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-27T15:16:31.179 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-27T15:16:31.179 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:16:31.179 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.444+0000 7f35d2f49700 1 Processor -- start 2023-11-27T15:16:31.179 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.444+0000 7f35d2f49700 1 -- start start 2023-11-27T15:16:31.179 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.444+0000 7f35d2f49700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc07a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:31.179 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.444+0000 7f35d2f49700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc07a800 con 0x7f35cc07bdc0 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.445+0000 7f35d0ce5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc07a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.445+0000 7f35d0ce5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc07a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46342/0 (socket says 172.21.15.8:46342) 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.445+0000 7f35d0ce5700 1 -- 172.21.15.8:0/3217917370 learned_addr learned my addr 172.21.15.8:0/3217917370 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.445+0000 7f35d0ce5700 1 -- 172.21.15.8:0/3217917370 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc07a940 con 0x7f35cc07bdc0 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.445+0000 7f35d0ce5700 1 --2- 172.21.15.8:0/3217917370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc07a2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35c80098a0 tx=0x7f35c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1735f24d50c32e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.446+0000 7f35c37fe700 1 -- 172.21.15.8:0/3217917370 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35c8004160 con 0x7f35cc07bdc0 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.446+0000 7f35c37fe700 1 -- 172.21.15.8:0/3217917370 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35c80042c0 con 0x7f35cc07bdc0 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.446+0000 7f35c37fe700 1 -- 172.21.15.8:0/3217917370 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35c80119b0 con 0x7f35cc07bdc0 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.446+0000 7f35d2f49700 1 -- 172.21.15.8:0/3217917370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 msgr2=0x7f35cc07a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.446+0000 7f35d2f49700 1 --2- 172.21.15.8:0/3217917370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc07a2c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f35c80098a0 tx=0x7f35c8009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.446+0000 7f35d2f49700 1 -- 172.21.15.8:0/3217917370 shutdown_connections 2023-11-27T15:16:31.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.446+0000 7f35d2f49700 1 --2- 172.21.15.8:0/3217917370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc07a2c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.446+0000 7f35d2f49700 1 -- 172.21.15.8:0/3217917370 >> 172.21.15.8:0/3217917370 conn(0x7f35cc100fa0 msgr2=0x7f35cc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.446+0000 7f35d2f49700 1 -- 172.21.15.8:0/3217917370 shutdown_connections 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.446+0000 7f35d2f49700 1 -- 172.21.15.8:0/3217917370 wait complete. 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.447+0000 7f35d2f49700 1 Processor -- start 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.448+0000 7f35d2f49700 1 -- start start 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.448+0000 7f35d2f49700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.448+0000 7f35d2f49700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc19bd00 con 0x7f35cc07bdc0 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.448+0000 7f35d0ce5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.448+0000 7f35d0ce5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc19b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46350/0 (socket says 172.21.15.8:46350) 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.448+0000 7f35d0ce5700 1 -- 172.21.15.8:0/190224290 learned_addr learned my addr 172.21.15.8:0/190224290 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.448+0000 7f35d0ce5700 1 -- 172.21.15.8:0/190224290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8009550 con 0x7f35cc07bdc0 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.449+0000 7f35d0ce5700 1 --2- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc19b7c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f35c8000c00 tx=0x7f35c8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:31.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.449+0000 7f35c1ffb700 1 -- 172.21.15.8:0/190224290 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35c80039f0 con 0x7f35cc07bdc0 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.449+0000 7f35c1ffb700 1 -- 172.21.15.8:0/190224290 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35c8003b50 con 0x7f35cc07bdc0 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.449+0000 7f35c1ffb700 1 -- 172.21.15.8:0/190224290 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35c801a780 con 0x7f35cc07bdc0 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.449+0000 7f35d2f49700 1 -- 172.21.15.8:0/190224290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35cc10be10 con 0x7f35cc07bdc0 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.449+0000 7f35d2f49700 1 -- 172.21.15.8:0/190224290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc10bc80 con 0x7f35cc07bdc0 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.450+0000 7f35d2f49700 1 -- 172.21.15.8:0/190224290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc10bfa0 con 0x7f35cc07bdc0 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.451+0000 7f35c1ffb700 1 -- 172.21.15.8:0/190224290 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f35c8024440 con 0x7f35cc07bdc0 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.451+0000 7f35c1ffb700 1 --2- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f35b4038870 0x7f35b403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.451+0000 7f35c1ffb700 1 -- 172.21.15.8:0/190224290 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f35c804c860 con 0x7f35cc07bdc0 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.451+0000 7f35c3fff700 1 -- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f35b4038870 msgr2=0x7f35b403ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/2324276308 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.451+0000 7f35c3fff700 1 --2- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f35b4038870 0x7f35b403ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:16:31.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.456+0000 7f35c1ffb700 1 -- 172.21.15.8:0/190224290 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f35c8018db0 con 0x7f35cc07bdc0 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.652+0000 7f35c3fff700 1 -- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f35b4038870 msgr2=0x7f35b403ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/2324276308 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.652+0000 7f35c3fff700 1 --2- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f35b4038870 0x7f35b403ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.721+0000 7f35d2f49700 1 -- 172.21.15.8:0/190224290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f35cc19f3f0 con 0x7f35cc07bdc0 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.722+0000 7f35c1ffb700 1 -- 172.21.15.8:0/190224290 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f35c8018350 con 0x7f35cc07bdc0 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.724+0000 7f35d2f49700 1 -- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f35b4038870 msgr2=0x7f35b403ad20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.724+0000 7f35d2f49700 1 --2- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f35b4038870 0x7f35b403ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.724+0000 7f35d2f49700 1 -- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 msgr2=0x7f35cc19b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.724+0000 7f35d2f49700 1 --2- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc19b7c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f35c8000c00 tx=0x7f35c8011a30 comp rx=0 tx=0).stop 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.724+0000 7f35d2f49700 1 -- 172.21.15.8:0/190224290 shutdown_connections 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.724+0000 7f35d2f49700 1 --2- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7f35b4038870 0x7f35b403ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.724+0000 7f35d2f49700 1 --2- 172.21.15.8:0/190224290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35cc07bdc0 0x7f35cc19b7c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.724+0000 7f35d2f49700 1 -- 172.21.15.8:0/190224290 >> 172.21.15.8:0/190224290 conn(0x7f35cc100fa0 msgr2=0x7f35cc101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:31.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.725+0000 7f35d2f49700 1 -- 172.21.15.8:0/190224290 shutdown_connections 2023-11-27T15:16:31.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:30.725+0000 7f35d2f49700 1 -- 172.21.15.8:0/190224290 wait complete. 2023-11-27T15:16:31.184 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for the mgr to restart... 2023-11-27T15:16:31.184 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr epoch 9... 2023-11-27T15:16:31.363 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:31 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:31.155+0000 7f956d813200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:16:31.747 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:31 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/190224290' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-27T15:16:32.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:31 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:31.743+0000 7f956d813200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:16:32.863 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:32 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:32.514+0000 7f956d813200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:16:32.864 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:32 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:32.729+0000 7f956d813200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:16:34.073 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:33 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:33.713+0000 7f956d813200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:16:34.073 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:33 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:33.837+0000 7f956d813200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:16:34.363 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:34.069+0000 7f956d813200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:16:34.735 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:34.445+0000 7f956d813200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:16:35.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:34.731+0000 7f956d813200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:16:35.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:34.931+0000 7f956d813200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:16:35.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:35 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:35.291+0000 7f956d813200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:16:36.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:36 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:36.256+0000 7f956d813200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:16:36.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:36 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:36.436+0000 7f956d813200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:16:37.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:36 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:36.693+0000 7f956d813200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:16:37.114 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:36 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:36.847+0000 7f956d813200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:16:37.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:37 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:37.136+0000 7f956d813200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:16:38.613 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:16:38.239+0000 7f956d813200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:16:39.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: Active manager daemon x restarted 2023-11-27T15:16:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: Activating manager daemon x 2023-11-27T15:16:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: osdmap e3: 0 total, 0 up, 0 in 2023-11-27T15:16:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: mgrmap e10: x(active, starting, since 0.00408401s) 2023-11-27T15:16:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:16:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:16:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:16:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:16:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: Manager daemon x is now available 2023-11-27T15:16:39.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:39.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:39.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:16:39.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.715+0000 7fa4e05b8700 1 Processor -- start 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.716+0000 7fa4e05b8700 1 -- start start 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.716+0000 7fa4e05b8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.716+0000 7fa4e05b8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d807a820 con 0x7fa4d807bde0 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.716+0000 7fa4de354700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.716+0000 7fa4de354700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46356/0 (socket says 172.21.15.8:46356) 2023-11-27T15:16:39.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.716+0000 7fa4de354700 1 -- 172.21.15.8:0/1998099457 learned_addr learned my addr 172.21.15.8:0/1998099457 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.717+0000 7fa4de354700 1 -- 172.21.15.8:0/1998099457 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d807a960 con 0x7fa4d807bde0 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.717+0000 7fa4de354700 1 --2- 172.21.15.8:0/1998099457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d807a2e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098a0 tx=0x7fa4d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8f8f0ec1b5d060 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.717+0000 7fa4dd352700 1 -- 172.21.15.8:0/1998099457 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4d4004160 con 0x7fa4d807bde0 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.717+0000 7fa4dd352700 1 -- 172.21.15.8:0/1998099457 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4d40042c0 con 0x7fa4d807bde0 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.717+0000 7fa4dd352700 1 -- 172.21.15.8:0/1998099457 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4d40119b0 con 0x7fa4d807bde0 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.718+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1998099457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 msgr2=0x7fa4d807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.718+0000 7fa4e05b8700 1 --2- 172.21.15.8:0/1998099457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d807a2e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40098a0 tx=0x7fa4d4009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.718+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1998099457 shutdown_connections 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.718+0000 7fa4e05b8700 1 --2- 172.21.15.8:0/1998099457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d807a2e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.718+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1998099457 >> 172.21.15.8:0/1998099457 conn(0x7fa4d8100fa0 msgr2=0x7fa4d81033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.718+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1998099457 shutdown_connections 2023-11-27T15:16:39.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.718+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1998099457 wait complete. 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.720+0000 7fa4e05b8700 1 Processor -- start 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.720+0000 7fa4e05b8700 1 -- start start 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.720+0000 7fa4e05b8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.721+0000 7fa4e05b8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d819bcf0 con 0x7fa4d807bde0 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.721+0000 7fa4de354700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.721+0000 7fa4de354700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46368/0 (socket says 172.21.15.8:46368) 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.721+0000 7fa4de354700 1 -- 172.21.15.8:0/1254220616 learned_addr learned my addr 172.21.15.8:0/1254220616 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.721+0000 7fa4de354700 1 -- 172.21.15.8:0/1254220616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d4009550 con 0x7fa4d807bde0 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.721+0000 7fa4de354700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d819b7b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4003e60 tx=0x7fa4d4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.722+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4d4003970 con 0x7fa4d807bde0 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.722+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4d4003ad0 con 0x7fa4d807bde0 2023-11-27T15:16:39.790 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.722+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4d40195e0 con 0x7fa4d807bde0 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.722+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1254220616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d810be50 con 0x7fa4d807bde0 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.722+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1254220616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d810bcc0 con 0x7fa4d807bde0 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.723+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fa4d4019850 con 0x7fa4d807bde0 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.724+0000 7fa4cf7fe700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 0x7fa4c403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.724+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 --> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4c403b3e0 con 0x7fa4c4038820 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.724+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa4d404c8f0 con 0x7fa4d807bde0 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.724+0000 7fa4ddb53700 1 -- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 msgr2=0x7fa4c403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/2324276308 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.724+0000 7fa4ddb53700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 0x7fa4c403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.925+0000 7fa4ddb53700 1 -- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 msgr2=0x7fa4c403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/2324276308 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:31.925+0000 7fa4ddb53700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 0x7fa4c403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:32.326+0000 7fa4ddb53700 1 -- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 msgr2=0x7fa4c403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/2324276308 2023-11-27T15:16:39.791 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:32.326+0000 7fa4ddb53700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 0x7fa4c403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:33.127+0000 7fa4ddb53700 1 -- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 msgr2=0x7fa4c403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/2324276308 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:33.127+0000 7fa4ddb53700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 0x7fa4c403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:34.728+0000 7fa4ddb53700 1 -- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 msgr2=0x7fa4c403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/2324276308 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:34.729+0000 7fa4ddb53700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 0x7fa4c403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:37.932+0000 7fa4ddb53700 1 -- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 msgr2=0x7fa4c403acd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/2324276308 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:37.932+0000 7fa4ddb53700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 0x7fa4c403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:38.245+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7fa4d4019b50 con 0x7fa4d807bde0 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:38.245+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 msgr2=0x7fa4c403acd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:38.245+0000 7fa4cf7fe700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 0x7fa4c403acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.248+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fa4d404e300 con 0x7fa4d807bde0 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.248+0000 7fa4cf7fe700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa4c403c270 0x7fa4c403e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.248+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4c403b3e0 con 0x7fa4c403c270 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.249+0000 7fa4ddb53700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa4c403c270 0x7fa4c403e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.251+0000 7fa4ddb53700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa4c403c270 0x7fa4c403e650 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8003a10 tx=0x7fa4c8009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.252+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fa4c403b3e0 con 0x7fa4c403c270 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.260+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1254220616 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa4d819be30 con 0x7fa4c403c270 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.261+0000 7fa4cf7fe700 1 -- 172.21.15.8:0/1254220616 <== mgr.14152 v2:172.21.15.8:6800/1756612424 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa4c800c9c0 con 0x7fa4c403c270 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.261+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa4c403c270 msgr2=0x7fa4c403e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.261+0000 7fa4e05b8700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa4c403c270 0x7fa4c403e650 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8003a10 tx=0x7fa4c8009270 comp rx=0 tx=0).stop 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.261+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 msgr2=0x7fa4d819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.261+0000 7fa4e05b8700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d819b7b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4003e60 tx=0x7fa4d4011aa0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.261+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1254220616 shutdown_connections 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.261+0000 7fa4e05b8700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa4c403c270 0x7fa4c403e650 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.261+0000 7fa4e05b8700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4d807bde0 0x7fa4d819b7b0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4003e60 tx=0x7fa4d4011aa0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.794 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.261+0000 7fa4e05b8700 1 --2- 172.21.15.8:0/1254220616 >> [v2:172.21.15.8:6800/2324276308,v1:172.21.15.8:6801/2324276308] conn(0x7fa4c4038820 0x7fa4c403acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:39.794 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.261+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1254220616 >> 172.21.15.8:0/1254220616 conn(0x7fa4d8100fa0 msgr2=0x7fa4d8101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:39.794 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.262+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1254220616 shutdown_connections 2023-11-27T15:16:39.794 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:39.262+0000 7fa4e05b8700 1 -- 172.21.15.8:0/1254220616 wait complete. 2023-11-27T15:16:39.794 INFO:teuthology.orchestra.run.smithi008.stdout:mgr epoch 9 is available 2023-11-27T15:16:39.794 INFO:teuthology.orchestra.run.smithi008.stdout:Generating a dashboard self-signed certificate... 2023-11-27T15:16:40.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:40 smithi008 ceph-mon[128788]: mgrmap e11: x(active, since 1.0082s) 2023-11-27T15:16:40.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:40 smithi008 ceph-mon[128788]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-27T15:16:40.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:40 smithi008 ceph-mon[128788]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-27T15:16:41.462 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:41 smithi008 ceph-mon[128788]: [27/Nov/2023:15:16:39] ENGINE Bus STARTING 2023-11-27T15:16:41.462 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:41 smithi008 ceph-mon[128788]: [27/Nov/2023:15:16:40] ENGINE Serving on http://172.21.15.8:8765 2023-11-27T15:16:41.462 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:41 smithi008 ceph-mon[128788]: [27/Nov/2023:15:16:40] ENGINE Serving on https://172.21.15.8:7150 2023-11-27T15:16:41.462 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:41 smithi008 ceph-mon[128788]: [27/Nov/2023:15:16:40] ENGINE Bus STARTED 2023-11-27T15:16:41.462 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:41 smithi008 ceph-mon[128788]: mgrmap e12: x(active, since 2s) 2023-11-27T15:16:41.462 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:41 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:41.462 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:41 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:41.463 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:41 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:41.480 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-11-27T15:16:41.480 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.601+0000 7ff256489700 1 Processor -- start 2023-11-27T15:16:41.480 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.601+0000 7ff256489700 1 -- start start 2023-11-27T15:16:41.481 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.601+0000 7ff256489700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff250105b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.481 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.601+0000 7ff256489700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2501060a0 con 0x7ff250105750 2023-11-27T15:16:41.481 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.602+0000 7ff255487700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff250105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.481 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.602+0000 7ff255487700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff250105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56730/0 (socket says 172.21.15.8:56730) 2023-11-27T15:16:41.481 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.602+0000 7ff255487700 1 -- 172.21.15.8:0/1910635974 learned_addr learned my addr 172.21.15.8:0/1910635974 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:41.481 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.602+0000 7ff255487700 1 -- 172.21.15.8:0/1910635974 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2501061e0 con 0x7ff250105750 2023-11-27T15:16:41.482 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.602+0000 7ff255487700 1 --2- 172.21.15.8:0/1910635974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff250105b60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0098a0 tx=0x7ff24c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b22d8ca6d041096 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:41.482 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.603+0000 7ff247fff700 1 -- 172.21.15.8:0/1910635974 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c004160 con 0x7ff250105750 2023-11-27T15:16:41.482 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.603+0000 7ff247fff700 1 -- 172.21.15.8:0/1910635974 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff24c0042c0 con 0x7ff250105750 2023-11-27T15:16:41.482 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.603+0000 7ff247fff700 1 -- 172.21.15.8:0/1910635974 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c0119b0 con 0x7ff250105750 2023-11-27T15:16:41.482 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.603+0000 7ff256489700 1 -- 172.21.15.8:0/1910635974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 msgr2=0x7ff250105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.482 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.603+0000 7ff256489700 1 --2- 172.21.15.8:0/1910635974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff250105b60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0098a0 tx=0x7ff24c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.483 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.603+0000 7ff256489700 1 -- 172.21.15.8:0/1910635974 shutdown_connections 2023-11-27T15:16:41.483 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.603+0000 7ff256489700 1 --2- 172.21.15.8:0/1910635974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff250105b60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.483 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.603+0000 7ff256489700 1 -- 172.21.15.8:0/1910635974 >> 172.21.15.8:0/1910635974 conn(0x7ff250100dc0 msgr2=0x7ff2501031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:41.483 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.603+0000 7ff256489700 1 -- 172.21.15.8:0/1910635974 shutdown_connections 2023-11-27T15:16:41.483 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.604+0000 7ff256489700 1 -- 172.21.15.8:0/1910635974 wait complete. 2023-11-27T15:16:41.483 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.604+0000 7ff256489700 1 Processor -- start 2023-11-27T15:16:41.483 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.605+0000 7ff256489700 1 -- start start 2023-11-27T15:16:41.484 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.605+0000 7ff256489700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff25019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.484 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.605+0000 7ff256489700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25019bb40 con 0x7ff250105750 2023-11-27T15:16:41.484 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.605+0000 7ff255487700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff25019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.484 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.605+0000 7ff255487700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff25019b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56734/0 (socket says 172.21.15.8:56734) 2023-11-27T15:16:41.484 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.605+0000 7ff255487700 1 -- 172.21.15.8:0/853863765 learned_addr learned my addr 172.21.15.8:0/853863765 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:41.484 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.605+0000 7ff255487700 1 -- 172.21.15.8:0/853863765 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c009550 con 0x7ff250105750 2023-11-27T15:16:41.485 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.606+0000 7ff255487700 1 --2- 172.21.15.8:0/853863765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff25019b600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff24c003f00 tx=0x7ff24c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:41.485 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.606+0000 7ff2467fc700 1 -- 172.21.15.8:0/853863765 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c003b20 con 0x7ff250105750 2023-11-27T15:16:41.485 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.606+0000 7ff2467fc700 1 -- 172.21.15.8:0/853863765 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff24c003c80 con 0x7ff250105750 2023-11-27T15:16:41.485 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.606+0000 7ff256489700 1 -- 172.21.15.8:0/853863765 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2501077e0 con 0x7ff250105750 2023-11-27T15:16:41.485 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.606+0000 7ff2467fc700 1 -- 172.21.15.8:0/853863765 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c019770 con 0x7ff250105750 2023-11-27T15:16:41.485 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.606+0000 7ff256489700 1 -- 172.21.15.8:0/853863765 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff250107650 con 0x7ff250105750 2023-11-27T15:16:41.485 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.607+0000 7ff2467fc700 1 -- 172.21.15.8:0/853863765 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7ff24c01ab60 con 0x7ff250105750 2023-11-27T15:16:41.486 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.607+0000 7ff256489700 1 -- 172.21.15.8:0/853863765 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff250109c20 con 0x7ff250105750 2023-11-27T15:16:41.486 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.608+0000 7ff2467fc700 1 --2- 172.21.15.8:0/853863765 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff23c038870 0x7ff23c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:41.486 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.608+0000 7ff2467fc700 1 -- 172.21.15.8:0/853863765 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff24c04cb20 con 0x7ff250105750 2023-11-27T15:16:41.486 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.608+0000 7ff254c86700 1 --2- 172.21.15.8:0/853863765 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff23c038870 0x7ff23c03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:41.486 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.609+0000 7ff254c86700 1 --2- 172.21.15.8:0/853863765 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff23c038870 0x7ff23c03ad20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff240006fd0 tx=0x7ff240006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:41.487 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.613+0000 7ff2467fc700 1 -- 172.21.15.8:0/853863765 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff24c01a430 con 0x7ff250105750 2023-11-27T15:16:41.487 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.834+0000 7ff256489700 1 -- 172.21.15.8:0/853863765 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2501021b0 con 0x7ff23c038870 2023-11-27T15:16:41.487 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.913+0000 7ff2467fc700 1 -- 172.21.15.8:0/853863765 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ff2501021b0 con 0x7ff23c038870 2023-11-27T15:16:41.487 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.915+0000 7ff256489700 1 -- 172.21.15.8:0/853863765 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff23c038870 msgr2=0x7ff23c03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.487 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.915+0000 7ff256489700 1 --2- 172.21.15.8:0/853863765 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff23c038870 0x7ff23c03ad20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff240006fd0 tx=0x7ff240006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:41.487 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.915+0000 7ff256489700 1 -- 172.21.15.8:0/853863765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 msgr2=0x7ff25019b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:41.487 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.915+0000 7ff256489700 1 --2- 172.21.15.8:0/853863765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff25019b600 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff24c003f00 tx=0x7ff24c011a30 comp rx=0 tx=0).stop 2023-11-27T15:16:41.488 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.916+0000 7ff256489700 1 -- 172.21.15.8:0/853863765 shutdown_connections 2023-11-27T15:16:41.488 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.916+0000 7ff256489700 1 --2- 172.21.15.8:0/853863765 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff23c038870 0x7ff23c03ad20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.488 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.916+0000 7ff256489700 1 --2- 172.21.15.8:0/853863765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff250105750 0x7ff25019b600 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:41.488 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.916+0000 7ff256489700 1 -- 172.21.15.8:0/853863765 >> 172.21.15.8:0/853863765 conn(0x7ff250100dc0 msgr2=0x7ff250101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:41.488 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.916+0000 7ff256489700 1 -- 172.21.15.8:0/853863765 shutdown_connections 2023-11-27T15:16:41.488 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:40.916+0000 7ff256489700 1 -- 172.21.15.8:0/853863765 wait complete. 2023-11-27T15:16:41.488 INFO:teuthology.orchestra.run.smithi008.stdout:Creating initial admin user... 2023-11-27T15:16:42.558 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:42 smithi008 ceph-mon[128788]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:42.967 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$nzPnVA6dSFAazc27l8ATu.lGEKx3v6yFGtayYyRPxy50cqo8cm9SO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1701098202, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-11-27T15:16:42.967 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.038+0000 7fb740189700 1 Processor -- start 2023-11-27T15:16:42.967 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.038+0000 7fb740189700 1 -- start start 2023-11-27T15:16:42.967 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.039+0000 7fb740189700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:42.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.039+0000 7fb740189700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73807a820 con 0x7fb73807bde0 2023-11-27T15:16:42.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.039+0000 7fb73df25700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:42.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.039+0000 7fb73df25700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56750/0 (socket says 172.21.15.8:56750) 2023-11-27T15:16:42.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.039+0000 7fb73df25700 1 -- 172.21.15.8:0/269791838 learned_addr learned my addr 172.21.15.8:0/269791838 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:42.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.039+0000 7fb73df25700 1 -- 172.21.15.8:0/269791838 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73807a960 con 0x7fb73807bde0 2023-11-27T15:16:42.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.040+0000 7fb73df25700 1 --2- 172.21.15.8:0/269791838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73807a2e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb7340098a0 tx=0x7fb734009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41a3e185d5ef074 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:42.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.040+0000 7fb73cf23700 1 -- 172.21.15.8:0/269791838 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb734004160 con 0x7fb73807bde0 2023-11-27T15:16:42.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.040+0000 7fb73cf23700 1 -- 172.21.15.8:0/269791838 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7340042c0 con 0x7fb73807bde0 2023-11-27T15:16:42.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.040+0000 7fb73cf23700 1 -- 172.21.15.8:0/269791838 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7340119b0 con 0x7fb73807bde0 2023-11-27T15:16:42.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.040+0000 7fb740189700 1 -- 172.21.15.8:0/269791838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 msgr2=0x7fb73807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:42.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.040+0000 7fb740189700 1 --2- 172.21.15.8:0/269791838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73807a2e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb7340098a0 tx=0x7fb734009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:42.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.041+0000 7fb740189700 1 -- 172.21.15.8:0/269791838 shutdown_connections 2023-11-27T15:16:42.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.041+0000 7fb740189700 1 --2- 172.21.15.8:0/269791838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73807a2e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:42.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.041+0000 7fb740189700 1 -- 172.21.15.8:0/269791838 >> 172.21.15.8:0/269791838 conn(0x7fb738100fa0 msgr2=0x7fb7381033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:42.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.041+0000 7fb740189700 1 -- 172.21.15.8:0/269791838 shutdown_connections 2023-11-27T15:16:42.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.041+0000 7fb740189700 1 -- 172.21.15.8:0/269791838 wait complete. 2023-11-27T15:16:42.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.042+0000 7fb740189700 1 Processor -- start 2023-11-27T15:16:42.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.043+0000 7fb740189700 1 -- start start 2023-11-27T15:16:42.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.043+0000 7fb740189700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:42.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.043+0000 7fb740189700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73819bce0 con 0x7fb73807bde0 2023-11-27T15:16:42.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.043+0000 7fb73df25700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.043+0000 7fb73df25700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73819b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56764/0 (socket says 172.21.15.8:56764) 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.043+0000 7fb73df25700 1 -- 172.21.15.8:0/212827876 learned_addr learned my addr 172.21.15.8:0/212827876 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.043+0000 7fb73df25700 1 -- 172.21.15.8:0/212827876 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb734009550 con 0x7fb73807bde0 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.044+0000 7fb73df25700 1 --2- 172.21.15.8:0/212827876 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73819b7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb734000c00 tx=0x7fb734011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.044+0000 7fb72effd700 1 -- 172.21.15.8:0/212827876 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7340039f0 con 0x7fb73807bde0 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.044+0000 7fb72effd700 1 -- 172.21.15.8:0/212827876 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb734003b50 con 0x7fb73807bde0 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.044+0000 7fb72effd700 1 -- 172.21.15.8:0/212827876 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb73401a780 con 0x7fb73807bde0 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.044+0000 7fb740189700 1 -- 172.21.15.8:0/212827876 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73810be50 con 0x7fb73807bde0 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.044+0000 7fb740189700 1 -- 172.21.15.8:0/212827876 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73810bcc0 con 0x7fb73807bde0 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.045+0000 7fb740189700 1 -- 172.21.15.8:0/212827876 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73810bfe0 con 0x7fb73807bde0 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.046+0000 7fb72effd700 1 -- 172.21.15.8:0/212827876 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fb734024440 con 0x7fb73807bde0 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.046+0000 7fb72effd700 1 --2- 172.21.15.8:0/212827876 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb724038870 0x7fb72403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:42.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.046+0000 7fb72effd700 1 -- 172.21.15.8:0/212827876 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb73404c960 con 0x7fb73807bde0 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.046+0000 7fb73d724700 1 --2- 172.21.15.8:0/212827876 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb724038870 0x7fb72403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.047+0000 7fb73d724700 1 --2- 172.21.15.8:0/212827876 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb724038870 0x7fb72403ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb728006fd0 tx=0x7fb728006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.052+0000 7fb72effd700 1 -- 172.21.15.8:0/212827876 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb7340189a0 con 0x7fb73807bde0 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.282+0000 7fb740189700 1 -- 172.21.15.8:0/212827876 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb738061f40 con 0x7fb724038870 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.530+0000 7fb72effd700 1 -- 172.21.15.8:0/212827876 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fb738061f40 con 0x7fb724038870 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.532+0000 7fb740189700 1 -- 172.21.15.8:0/212827876 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb724038870 msgr2=0x7fb72403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.532+0000 7fb740189700 1 --2- 172.21.15.8:0/212827876 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb724038870 0x7fb72403ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb728006fd0 tx=0x7fb728006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.532+0000 7fb740189700 1 -- 172.21.15.8:0/212827876 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 msgr2=0x7fb73819b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.532+0000 7fb740189700 1 --2- 172.21.15.8:0/212827876 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73819b7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb734000c00 tx=0x7fb734011a30 comp rx=0 tx=0).stop 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.532+0000 7fb740189700 1 -- 172.21.15.8:0/212827876 shutdown_connections 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.532+0000 7fb740189700 1 --2- 172.21.15.8:0/212827876 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb724038870 0x7fb72403ad20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:42.972 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.532+0000 7fb740189700 1 --2- 172.21.15.8:0/212827876 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb73807bde0 0x7fb73819b7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:42.973 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.532+0000 7fb740189700 1 -- 172.21.15.8:0/212827876 >> 172.21.15.8:0/212827876 conn(0x7fb738100fa0 msgr2=0x7fb738101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:42.973 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.533+0000 7fb740189700 1 -- 172.21.15.8:0/212827876 shutdown_connections 2023-11-27T15:16:42.973 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:42.533+0000 7fb740189700 1 -- 172.21.15.8:0/212827876 wait complete. 2023-11-27T15:16:42.973 INFO:teuthology.orchestra.run.smithi008.stdout:Fetching dashboard port number... 2023-11-27T15:16:43.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:43 smithi008 ceph-mon[128788]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:43.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:43 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:43.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:43 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:44.245 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 8443 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.513+0000 7fcccd9fe700 1 Processor -- start 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.514+0000 7fcccd9fe700 1 -- start start 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.514+0000 7fcccd9fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8079d00 0x7fccc8108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.514+0000 7fcccd9fe700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc81089d0 con 0x7fccc8079d00 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.514+0000 7fccc6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8079d00 0x7fccc8108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.514+0000 7fccc6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8079d00 0x7fccc8108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56778/0 (socket says 172.21.15.8:56778) 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.514+0000 7fccc6ffd700 1 -- 172.21.15.8:0/211336750 learned_addr learned my addr 172.21.15.8:0/211336750 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.515+0000 7fccc6ffd700 1 -- 172.21.15.8:0/211336750 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8108b10 con 0x7fccc8079d00 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.515+0000 7fccc6ffd700 1 --2- 172.21.15.8:0/211336750 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8079d00 0x7fccc8108490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fccb80098a0 tx=0x7fccb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b66ac6735e78ca6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.515+0000 7fccc5ffb700 1 -- 172.21.15.8:0/211336750 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccb8004160 con 0x7fccc8079d00 2023-11-27T15:16:44.251 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.515+0000 7fccc5ffb700 1 -- 172.21.15.8:0/211336750 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccb80042c0 con 0x7fccc8079d00 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.515+0000 7fccc5ffb700 1 -- 172.21.15.8:0/211336750 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccb80119b0 con 0x7fccc8079d00 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.516+0000 7fcccd9fe700 1 -- 172.21.15.8:0/211336750 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8079d00 msgr2=0x7fccc8108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.516+0000 7fcccd9fe700 1 --2- 172.21.15.8:0/211336750 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8079d00 0x7fccc8108490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fccb80098a0 tx=0x7fccb8009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.516+0000 7fcccd9fe700 1 -- 172.21.15.8:0/211336750 shutdown_connections 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.516+0000 7fcccd9fe700 1 --2- 172.21.15.8:0/211336750 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8079d00 0x7fccc8108490 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.516+0000 7fcccd9fe700 1 -- 172.21.15.8:0/211336750 >> 172.21.15.8:0/211336750 conn(0x7fccc80ffe40 msgr2=0x7fccc8102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.516+0000 7fcccd9fe700 1 -- 172.21.15.8:0/211336750 shutdown_connections 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.516+0000 7fcccd9fe700 1 -- 172.21.15.8:0/211336750 wait complete. 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.518+0000 7fcccd9fe700 1 Processor -- start 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.518+0000 7fcccd9fe700 1 -- start start 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.519+0000 7fcccd9fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8199530 0x7fccc8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:44.252 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.519+0000 7fcccd9fe700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc8199e80 con 0x7fccc8199530 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.519+0000 7fccc6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8199530 0x7fccc8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.519+0000 7fccc6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8199530 0x7fccc8199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56782/0 (socket says 172.21.15.8:56782) 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.519+0000 7fccc6ffd700 1 -- 172.21.15.8:0/1591975491 learned_addr learned my addr 172.21.15.8:0/1591975491 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.519+0000 7fccc6ffd700 1 -- 172.21.15.8:0/1591975491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8009550 con 0x7fccc8199530 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.519+0000 7fccc6ffd700 1 --2- 172.21.15.8:0/1591975491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8199530 0x7fccc8199940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fccb8000c00 tx=0x7fccb8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.520+0000 7fcccc9fc700 1 -- 172.21.15.8:0/1591975491 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccb80039f0 con 0x7fccc8199530 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.520+0000 7fcccc9fc700 1 -- 172.21.15.8:0/1591975491 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccb8003b50 con 0x7fccc8199530 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.520+0000 7fcccc9fc700 1 -- 172.21.15.8:0/1591975491 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fccb801a780 con 0x7fccc8199530 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.520+0000 7fcccd9fe700 1 -- 172.21.15.8:0/1591975491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc8109ba0 con 0x7fccc8199530 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.520+0000 7fcccd9fe700 1 -- 172.21.15.8:0/1591975491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc8109a10 con 0x7fccc8199530 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.521+0000 7fcccc9fc700 1 -- 172.21.15.8:0/1591975491 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fccb8024440 con 0x7fccc8199530 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.521+0000 7fcccd9fe700 1 -- 172.21.15.8:0/1591975491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc8062120 con 0x7fccc8199530 2023-11-27T15:16:44.253 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.522+0000 7fcccc9fc700 1 --2- 172.21.15.8:0/1591975491 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fccb0038870 0x7fccb003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.522+0000 7fcccc9fc700 1 -- 172.21.15.8:0/1591975491 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fccb804ca20 con 0x7fccc8199530 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.522+0000 7fccc67fc700 1 --2- 172.21.15.8:0/1591975491 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fccb0038870 0x7fccb003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.523+0000 7fccc67fc700 1 --2- 172.21.15.8:0/1591975491 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fccb0038870 0x7fccb003ad20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fccbc006fd0 tx=0x7fccbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.528+0000 7fcccc9fc700 1 -- 172.21.15.8:0/1591975491 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fccb80189a0 con 0x7fccc8199530 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.772+0000 7fcccd9fe700 1 -- 172.21.15.8:0/1591975491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fccc8079d00 con 0x7fccc8199530 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.773+0000 7fcccc9fc700 1 -- 172.21.15.8:0/1591975491 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fccb8018350 con 0x7fccc8199530 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.775+0000 7fcccd9fe700 1 -- 172.21.15.8:0/1591975491 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fccb0038870 msgr2=0x7fccb003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.775+0000 7fcccd9fe700 1 --2- 172.21.15.8:0/1591975491 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fccb0038870 0x7fccb003ad20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fccbc006fd0 tx=0x7fccbc006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.775+0000 7fcccd9fe700 1 -- 172.21.15.8:0/1591975491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8199530 msgr2=0x7fccc8199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.775+0000 7fcccd9fe700 1 --2- 172.21.15.8:0/1591975491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8199530 0x7fccc8199940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fccb8000c00 tx=0x7fccb8011a30 comp rx=0 tx=0).stop 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.776+0000 7fcccd9fe700 1 -- 172.21.15.8:0/1591975491 shutdown_connections 2023-11-27T15:16:44.254 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.776+0000 7fcccd9fe700 1 --2- 172.21.15.8:0/1591975491 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fccb0038870 0x7fccb003ad20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:44.255 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.776+0000 7fcccd9fe700 1 --2- 172.21.15.8:0/1591975491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fccc8199530 0x7fccc8199940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:44.255 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.776+0000 7fcccd9fe700 1 -- 172.21.15.8:0/1591975491 >> 172.21.15.8:0/1591975491 conn(0x7fccc80ffe40 msgr2=0x7fccc8100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:44.255 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.776+0000 7fcccd9fe700 1 -- 172.21.15.8:0/1591975491 shutdown_connections 2023-11-27T15:16:44.255 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:43.776+0000 7fcccd9fe700 1 -- 172.21.15.8:0/1591975491 wait complete. 2023-11-27T15:16:44.269 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:16:44.269 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2023-11-27T15:16:44.285 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:16:44.286 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2023-11-27T15:16:44.286 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2023-11-27T15:16:44.286 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-11-27T15:16:44.289 INFO:teuthology.orchestra.run.smithi008.stdout:Ceph Dashboard is now available at: 2023-11-27T15:16:44.290 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:16:44.290 INFO:teuthology.orchestra.run.smithi008.stdout: URL: https://smithi008.front.sepia.ceph.com:8443/ 2023-11-27T15:16:44.290 INFO:teuthology.orchestra.run.smithi008.stdout: User: admin 2023-11-27T15:16:44.290 INFO:teuthology.orchestra.run.smithi008.stdout: Password: 5kmotvhjdo 2023-11-27T15:16:44.290 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:16:44.290 INFO:teuthology.orchestra.run.smithi008.stdout:Saving cluster configuration to /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config directory 2023-11-27T15:16:44.291 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling autotune for osd_memory_target 2023-11-27T15:16:44.552 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:44 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1591975491' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-11-27T15:16:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:45 smithi008 ceph-mon[128788]: mgrmap e13: x(active, since 6s) 2023-11-27T15:16:45.718 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.015+0000 7f1cee7d5700 1 Processor -- start 2023-11-27T15:16:45.718 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.015+0000 7f1cee7d5700 1 -- start start 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.015+0000 7f1cee7d5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.015+0000 7f1cee7d5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce807a820 con 0x7f1ce807bde0 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.016+0000 7f1ce7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.016+0000 7f1ce7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56784/0 (socket says 172.21.15.8:56784) 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.016+0000 7f1ce7fff700 1 -- 172.21.15.8:0/1882971032 learned_addr learned my addr 172.21.15.8:0/1882971032 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.016+0000 7f1ce7fff700 1 -- 172.21.15.8:0/1882971032 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce807a960 con 0x7f1ce807bde0 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.017+0000 7f1ce7fff700 1 --2- 172.21.15.8:0/1882971032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce807a2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80098a0 tx=0x7f1cd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9856badc6aeaa52 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.017+0000 7f1ce6ffd700 1 -- 172.21.15.8:0/1882971032 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cd8004160 con 0x7f1ce807bde0 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.017+0000 7f1ce6ffd700 1 -- 172.21.15.8:0/1882971032 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cd80042c0 con 0x7f1ce807bde0 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.017+0000 7f1ce6ffd700 1 -- 172.21.15.8:0/1882971032 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cd80119b0 con 0x7f1ce807bde0 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.017+0000 7f1cee7d5700 1 -- 172.21.15.8:0/1882971032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 msgr2=0x7f1ce807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:45.719 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.017+0000 7f1cee7d5700 1 --2- 172.21.15.8:0/1882971032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce807a2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80098a0 tx=0x7f1cd8009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.017+0000 7f1cee7d5700 1 -- 172.21.15.8:0/1882971032 shutdown_connections 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.017+0000 7f1cee7d5700 1 --2- 172.21.15.8:0/1882971032 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce807a2e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.017+0000 7f1cee7d5700 1 -- 172.21.15.8:0/1882971032 >> 172.21.15.8:0/1882971032 conn(0x7f1ce8100fa0 msgr2=0x7f1ce81033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.018+0000 7f1cee7d5700 1 -- 172.21.15.8:0/1882971032 shutdown_connections 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.018+0000 7f1cee7d5700 1 -- 172.21.15.8:0/1882971032 wait complete. 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.019+0000 7f1cee7d5700 1 Processor -- start 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.019+0000 7f1cee7d5700 1 -- start start 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.020+0000 7f1cee7d5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.020+0000 7f1cee7d5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce819bcf0 con 0x7f1ce807bde0 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.020+0000 7f1ce7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.020+0000 7f1ce7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56800/0 (socket says 172.21.15.8:56800) 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.020+0000 7f1ce7fff700 1 -- 172.21.15.8:0/249989914 learned_addr learned my addr 172.21.15.8:0/249989914 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:45.720 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.020+0000 7f1ce7fff700 1 -- 172.21.15.8:0/249989914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd8009550 con 0x7f1ce807bde0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.020+0000 7f1ce7fff700 1 --2- 172.21.15.8:0/249989914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce819b7b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8003e60 tx=0x7f1cd8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.021+0000 7f1ce57fa700 1 -- 172.21.15.8:0/249989914 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cd8003970 con 0x7f1ce807bde0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.021+0000 7f1ce57fa700 1 -- 172.21.15.8:0/249989914 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cd8003ad0 con 0x7f1ce807bde0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.021+0000 7f1ce57fa700 1 -- 172.21.15.8:0/249989914 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1cd80195e0 con 0x7f1ce807bde0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.021+0000 7f1cee7d5700 1 -- 172.21.15.8:0/249989914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce810be50 con 0x7f1ce807bde0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.021+0000 7f1cee7d5700 1 -- 172.21.15.8:0/249989914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce810bcc0 con 0x7f1ce807bde0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.022+0000 7f1ce57fa700 1 -- 172.21.15.8:0/249989914 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f1cd801a430 con 0x7f1ce807bde0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.022+0000 7f1cee7d5700 1 -- 172.21.15.8:0/249989914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce810bfe0 con 0x7f1ce807bde0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.022+0000 7f1ce57fa700 1 --2- 172.21.15.8:0/249989914 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1cd0038870 0x7f1cd003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.022+0000 7f1ce57fa700 1 -- 172.21.15.8:0/249989914 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1cd804cb00 con 0x7f1ce807bde0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.023+0000 7f1ce77fe700 1 --2- 172.21.15.8:0/249989914 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1cd0038870 0x7f1cd003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:45.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.023+0000 7f1ce77fe700 1 --2- 172.21.15.8:0/249989914 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1cd0038870 0x7f1cd003ad20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc006fd0 tx=0x7f1cdc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.027+0000 7f1ce57fa700 1 -- 172.21.15.8:0/249989914 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1cd804ee60 con 0x7f1ce807bde0 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.236+0000 7f1cee7d5700 1 -- 172.21.15.8:0/249989914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f1ce819f430 con 0x7f1ce807bde0 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.236+0000 7f1ce57fa700 1 -- 172.21.15.8:0/249989914 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f1cd8003c40 con 0x7f1ce807bde0 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.238+0000 7f1cee7d5700 1 -- 172.21.15.8:0/249989914 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1cd0038870 msgr2=0x7f1cd003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.238+0000 7f1cee7d5700 1 --2- 172.21.15.8:0/249989914 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1cd0038870 0x7f1cd003ad20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc006fd0 tx=0x7f1cdc006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.238+0000 7f1cee7d5700 1 -- 172.21.15.8:0/249989914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 msgr2=0x7f1ce819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.238+0000 7f1cee7d5700 1 --2- 172.21.15.8:0/249989914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce819b7b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8003e60 tx=0x7f1cd8011aa0 comp rx=0 tx=0).stop 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.238+0000 7f1cee7d5700 1 -- 172.21.15.8:0/249989914 shutdown_connections 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.238+0000 7f1cee7d5700 1 --2- 172.21.15.8:0/249989914 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1cd0038870 0x7f1cd003ad20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.238+0000 7f1cee7d5700 1 --2- 172.21.15.8:0/249989914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ce807bde0 0x7f1ce819b7b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.238+0000 7f1cee7d5700 1 -- 172.21.15.8:0/249989914 >> 172.21.15.8:0/249989914 conn(0x7f1ce8100fa0 msgr2=0x7f1ce8101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:45.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.238+0000 7f1cee7d5700 1 -- 172.21.15.8:0/249989914 shutdown_connections 2023-11-27T15:16:45.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:45.239+0000 7f1cee7d5700 1 -- 172.21.15.8:0/249989914 wait complete. 2023-11-27T15:16:47.002 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.262+0000 7f9cbe83e700 1 Processor -- start 2023-11-27T15:16:47.002 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.263+0000 7f9cbe83e700 1 -- start start 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.263+0000 7f9cbe83e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.263+0000 7f9cbe83e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb807a840 con 0x7f9cb807be00 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.263+0000 7f9cb7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.264+0000 7f9cb7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56810/0 (socket says 172.21.15.8:56810) 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.264+0000 7f9cb7fff700 1 -- 172.21.15.8:0/4106748976 learned_addr learned my addr 172.21.15.8:0/4106748976 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.264+0000 7f9cb7fff700 1 -- 172.21.15.8:0/4106748976 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb807a980 con 0x7f9cb807be00 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.264+0000 7f9cb7fff700 1 --2- 172.21.15.8:0/4106748976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb807a300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80098a0 tx=0x7f9ca8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d600abb43417ce5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.264+0000 7f9cb6ffd700 1 -- 172.21.15.8:0/4106748976 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca8004160 con 0x7f9cb807be00 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.264+0000 7f9cb6ffd700 1 -- 172.21.15.8:0/4106748976 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca80042c0 con 0x7f9cb807be00 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.264+0000 7f9cb6ffd700 1 -- 172.21.15.8:0/4106748976 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca80119b0 con 0x7f9cb807be00 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.265+0000 7f9cbe83e700 1 -- 172.21.15.8:0/4106748976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 msgr2=0x7f9cb807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.265+0000 7f9cbe83e700 1 --2- 172.21.15.8:0/4106748976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb807a300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80098a0 tx=0x7f9ca8009bb0 comp rx=0 tx=0).stop 2023-11-27T15:16:47.003 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.265+0000 7f9cbe83e700 1 -- 172.21.15.8:0/4106748976 shutdown_connections 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.265+0000 7f9cbe83e700 1 --2- 172.21.15.8:0/4106748976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb807a300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.265+0000 7f9cbe83e700 1 -- 172.21.15.8:0/4106748976 >> 172.21.15.8:0/4106748976 conn(0x7f9cb8100fa0 msgr2=0x7f9cb81033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.265+0000 7f9cbe83e700 1 -- 172.21.15.8:0/4106748976 shutdown_connections 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.265+0000 7f9cbe83e700 1 -- 172.21.15.8:0/4106748976 wait complete. 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.266+0000 7f9cbe83e700 1 Processor -- start 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.266+0000 7f9cbe83e700 1 -- start start 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.267+0000 7f9cbe83e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.267+0000 7f9cbe83e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb819bcf0 con 0x7f9cb807be00 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.267+0000 7f9cb7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.267+0000 7f9cb7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56816/0 (socket says 172.21.15.8:56816) 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.267+0000 7f9cb7fff700 1 -- 172.21.15.8:0/2351317701 learned_addr learned my addr 172.21.15.8:0/2351317701 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.267+0000 7f9cb7fff700 1 -- 172.21.15.8:0/2351317701 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca8009550 con 0x7f9cb807be00 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.267+0000 7f9cb7fff700 1 --2- 172.21.15.8:0/2351317701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb819b7b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8003e60 tx=0x7f9ca8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:47.004 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.268+0000 7f9cb57fa700 1 -- 172.21.15.8:0/2351317701 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca8003b20 con 0x7f9cb807be00 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.268+0000 7f9cb57fa700 1 -- 172.21.15.8:0/2351317701 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ca8003c80 con 0x7f9cb807be00 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.268+0000 7f9cb57fa700 1 -- 172.21.15.8:0/2351317701 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca8024760 con 0x7f9cb807be00 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.268+0000 7f9cbe83e700 1 -- 172.21.15.8:0/2351317701 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb810be30 con 0x7f9cb807be00 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.268+0000 7f9cbe83e700 1 -- 172.21.15.8:0/2351317701 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb810bca0 con 0x7f9cb807be00 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.269+0000 7f9cbe83e700 1 -- 172.21.15.8:0/2351317701 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb810bfc0 con 0x7f9cb807be00 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.269+0000 7f9cb57fa700 1 -- 172.21.15.8:0/2351317701 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f9ca801ab10 con 0x7f9cb807be00 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.270+0000 7f9cb57fa700 1 --2- 172.21.15.8:0/2351317701 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ca0038910 0x7f9ca003adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.270+0000 7f9cb57fa700 1 -- 172.21.15.8:0/2351317701 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9ca804cbd0 con 0x7f9cb807be00 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.270+0000 7f9cb77fe700 1 --2- 172.21.15.8:0/2351317701 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ca0038910 0x7f9ca003adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.270+0000 7f9cb77fe700 1 --2- 172.21.15.8:0/2351317701 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ca0038910 0x7f9ca003adc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9cac006fd0 tx=0x7f9cac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.274+0000 7f9cb57fa700 1 -- 172.21.15.8:0/2351317701 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9ca8018530 con 0x7f9cb807be00 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.592+0000 7f9cbe83e700 1 -- 172.21.15.8:0/2351317701 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f9cb8062120 con 0x7f9cb807be00 2023-11-27T15:16:47.005 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.594+0000 7f9cb57fa700 1 -- 172.21.15.8:0/2351317701 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f9ca8018350 con 0x7f9cb807be00 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.597+0000 7f9cbe83e700 1 -- 172.21.15.8:0/2351317701 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ca0038910 msgr2=0x7f9ca003adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.597+0000 7f9cbe83e700 1 --2- 172.21.15.8:0/2351317701 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ca0038910 0x7f9ca003adc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9cac006fd0 tx=0x7f9cac006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.597+0000 7f9cbe83e700 1 -- 172.21.15.8:0/2351317701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 msgr2=0x7f9cb819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.597+0000 7f9cbe83e700 1 --2- 172.21.15.8:0/2351317701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb819b7b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8003e60 tx=0x7f9ca8011a30 comp rx=0 tx=0).stop 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.597+0000 7f9cbe83e700 1 -- 172.21.15.8:0/2351317701 shutdown_connections 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.597+0000 7f9cbe83e700 1 --2- 172.21.15.8:0/2351317701 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ca0038910 0x7f9ca003adc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.597+0000 7f9cbe83e700 1 --2- 172.21.15.8:0/2351317701 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9cb807be00 0x7f9cb819b7b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.597+0000 7f9cbe83e700 1 -- 172.21.15.8:0/2351317701 >> 172.21.15.8:0/2351317701 conn(0x7f9cb8100fa0 msgr2=0x7f9cb8101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.597+0000 7f9cbe83e700 1 -- 172.21.15.8:0/2351317701 shutdown_connections 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2023-11-27T15:16:46.597+0000 7f9cbe83e700 1 -- 172.21.15.8:0/2351317701 wait complete. 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-11-27T15:16:47.006 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout:Or, if you are only running a single cluster on this host: 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout:Please consider enabling telemetry to help improve Ceph: 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: ceph telemetry on 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout:For more information see: 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-11-27T15:16:47.007 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:16:47.008 INFO:teuthology.orchestra.run.smithi008.stdout:Bootstrap complete. 2023-11-27T15:16:47.063 INFO:tasks.cephadm:Fetching config... 2023-11-27T15:16:47.063 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:16:47.063 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-11-27T15:16:47.107 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-11-27T15:16:47.107 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:16:47.107 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-11-27T15:16:47.194 INFO:tasks.cephadm:Fetching mon keyring... 2023-11-27T15:16:47.194 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:16:47.194 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/keyring of=/dev/stdout 2023-11-27T15:16:47.306 INFO:tasks.cephadm:Fetching pub ssh key... 2023-11-27T15:16:47.306 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:16:47.306 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-11-27T15:16:47.391 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-11-27T15:16:47.391 DEBUG:teuthology.orchestra.run.smithi008:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbveisYWttW783R4QIwCRUPLXfL4xU3wXqMnf1aTMwH6zYaeqHj/veOzOWpYvGO2v/j8AOF2kEHh8ce8fKho320KndoLsEMk1TsSM5tgORvy1MElVufSDL8HsRXtWuYZWB0oKRNpypZQ9pLkvnQ6j2tmR3BPsusIlf63pFORmP8GhBVyta8EQQ5sZn1K3qM6lGOUzgcit4PishJFWKW0iQY8RZ7LeoexO2vXxWyj+WkcIWj3x4I0U5w/PGagCdkjWT6VpVRGaLoao3ZlcUoQbm7WCgxEmR21g70jHHXMbef5xjp25GgNVCWmWo9gXQbDzT5g05LNbnMYIsSuTQlAHi4gHUBHfWmuuD7mqEXhn2kH0/fr3Babz3V9QanqbpR7kdnD103TgfxVu2jKmjv91zidjGZr1mRU4dR6jvk3DrQp/3G0WGcqHWOuuhpO6hf3MpoUE+vBFsv1gn32FUTqWt3B8M0gjwwI2V61cgasgUe5Vjv4uPKPI9Dl7YUVUlJU0= ceph-9f287ba8-8d37-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-27T15:16:47.542 INFO:teuthology.orchestra.run.smithi008.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbveisYWttW783R4QIwCRUPLXfL4xU3wXqMnf1aTMwH6zYaeqHj/veOzOWpYvGO2v/j8AOF2kEHh8ce8fKho320KndoLsEMk1TsSM5tgORvy1MElVufSDL8HsRXtWuYZWB0oKRNpypZQ9pLkvnQ6j2tmR3BPsusIlf63pFORmP8GhBVyta8EQQ5sZn1K3qM6lGOUzgcit4PishJFWKW0iQY8RZ7LeoexO2vXxWyj+WkcIWj3x4I0U5w/PGagCdkjWT6VpVRGaLoao3ZlcUoQbm7WCgxEmR21g70jHHXMbef5xjp25GgNVCWmWo9gXQbDzT5g05LNbnMYIsSuTQlAHi4gHUBHfWmuuD7mqEXhn2kH0/fr3Babz3V9QanqbpR7kdnD103TgfxVu2jKmjv91zidjGZr1mRU4dR6jvk3DrQp/3G0WGcqHWOuuhpO6hf3MpoUE+vBFsv1gn32FUTqWt3B8M0gjwwI2V61cgasgUe5Vjv4uPKPI9Dl7YUVUlJU0= ceph-9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:16:47.574 DEBUG:teuthology.orchestra.run.smithi029:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbveisYWttW783R4QIwCRUPLXfL4xU3wXqMnf1aTMwH6zYaeqHj/veOzOWpYvGO2v/j8AOF2kEHh8ce8fKho320KndoLsEMk1TsSM5tgORvy1MElVufSDL8HsRXtWuYZWB0oKRNpypZQ9pLkvnQ6j2tmR3BPsusIlf63pFORmP8GhBVyta8EQQ5sZn1K3qM6lGOUzgcit4PishJFWKW0iQY8RZ7LeoexO2vXxWyj+WkcIWj3x4I0U5w/PGagCdkjWT6VpVRGaLoao3ZlcUoQbm7WCgxEmR21g70jHHXMbef5xjp25GgNVCWmWo9gXQbDzT5g05LNbnMYIsSuTQlAHi4gHUBHfWmuuD7mqEXhn2kH0/fr3Babz3V9QanqbpR7kdnD103TgfxVu2jKmjv91zidjGZr1mRU4dR6jvk3DrQp/3G0WGcqHWOuuhpO6hf3MpoUE+vBFsv1gn32FUTqWt3B8M0gjwwI2V61cgasgUe5Vjv4uPKPI9Dl7YUVUlJU0= ceph-9f287ba8-8d37-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-27T15:16:47.620 INFO:teuthology.orchestra.run.smithi029.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbveisYWttW783R4QIwCRUPLXfL4xU3wXqMnf1aTMwH6zYaeqHj/veOzOWpYvGO2v/j8AOF2kEHh8ce8fKho320KndoLsEMk1TsSM5tgORvy1MElVufSDL8HsRXtWuYZWB0oKRNpypZQ9pLkvnQ6j2tmR3BPsusIlf63pFORmP8GhBVyta8EQQ5sZn1K3qM6lGOUzgcit4PishJFWKW0iQY8RZ7LeoexO2vXxWyj+WkcIWj3x4I0U5w/PGagCdkjWT6VpVRGaLoao3ZlcUoQbm7WCgxEmR21g70jHHXMbef5xjp25GgNVCWmWo9gXQbDzT5g05LNbnMYIsSuTQlAHi4gHUBHfWmuuD7mqEXhn2kH0/fr3Babz3V9QanqbpR7kdnD103TgfxVu2jKmjv91zidjGZr1mRU4dR6jvk3DrQp/3G0WGcqHWOuuhpO6hf3MpoUE+vBFsv1gn32FUTqWt3B8M0gjwwI2V61cgasgUe5Vjv4uPKPI9Dl7YUVUlJU0= ceph-9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:16:47.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbveisYWttW783R4QIwCRUPLXfL4xU3wXqMnf1aTMwH6zYaeqHj/veOzOWpYvGO2v/j8AOF2kEHh8ce8fKho320KndoLsEMk1TsSM5tgORvy1MElVufSDL8HsRXtWuYZWB0oKRNpypZQ9pLkvnQ6j2tmR3BPsusIlf63pFORmP8GhBVyta8EQQ5sZn1K3qM6lGOUzgcit4PishJFWKW0iQY8RZ7LeoexO2vXxWyj+WkcIWj3x4I0U5w/PGagCdkjWT6VpVRGaLoao3ZlcUoQbm7WCgxEmR21g70jHHXMbef5xjp25GgNVCWmWo9gXQbDzT5g05LNbnMYIsSuTQlAHi4gHUBHfWmuuD7mqEXhn2kH0/fr3Babz3V9QanqbpR7kdnD103TgfxVu2jKmjv91zidjGZr1mRU4dR6jvk3DrQp/3G0WGcqHWOuuhpO6hf3MpoUE+vBFsv1gn32FUTqWt3B8M0gjwwI2V61cgasgUe5Vjv4uPKPI9Dl7YUVUlJU0= ceph-9f287ba8-8d37-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-27T15:16:47.680 INFO:teuthology.orchestra.run.smithi060.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbveisYWttW783R4QIwCRUPLXfL4xU3wXqMnf1aTMwH6zYaeqHj/veOzOWpYvGO2v/j8AOF2kEHh8ce8fKho320KndoLsEMk1TsSM5tgORvy1MElVufSDL8HsRXtWuYZWB0oKRNpypZQ9pLkvnQ6j2tmR3BPsusIlf63pFORmP8GhBVyta8EQQ5sZn1K3qM6lGOUzgcit4PishJFWKW0iQY8RZ7LeoexO2vXxWyj+WkcIWj3x4I0U5w/PGagCdkjWT6VpVRGaLoao3ZlcUoQbm7WCgxEmR21g70jHHXMbef5xjp25GgNVCWmWo9gXQbDzT5g05LNbnMYIsSuTQlAHi4gHUBHfWmuuD7mqEXhn2kH0/fr3Babz3V9QanqbpR7kdnD103TgfxVu2jKmjv91zidjGZr1mRU4dR6jvk3DrQp/3G0WGcqHWOuuhpO6hf3MpoUE+vBFsv1gn32FUTqWt3B8M0gjwwI2V61cgasgUe5Vjv4uPKPI9Dl7YUVUlJU0= ceph-9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:16:47.694 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-11-27T15:16:47.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:47 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2351317701' entity='client.admin' 2023-11-27T15:16:48.846 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:49.986 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:49.986 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:49.987 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi008", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:16:49.987 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:49.987 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:49.987 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:50.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.225+0000 7faea72e5700 1 -- 172.21.15.8:0/3354099171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea00ffd00 msgr2=0x7faea0100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:50.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.225+0000 7faea72e5700 1 --2- 172.21.15.8:0/3354099171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea00ffd00 0x7faea0100110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009910 tx=0x7fae9c009c20 comp rx=0 tx=0).stop 2023-11-27T15:16:50.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.225+0000 7faea72e5700 1 -- 172.21.15.8:0/3354099171 shutdown_connections 2023-11-27T15:16:50.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.225+0000 7faea72e5700 1 --2- 172.21.15.8:0/3354099171 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea00ffd00 0x7faea0100110 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.225+0000 7faea72e5700 1 -- 172.21.15.8:0/3354099171 >> 172.21.15.8:0/3354099171 conn(0x7faea00fb2b0 msgr2=0x7faea00fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:50.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.225+0000 7faea72e5700 1 -- 172.21.15.8:0/3354099171 shutdown_connections 2023-11-27T15:16:50.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.225+0000 7faea72e5700 1 -- 172.21.15.8:0/3354099171 wait complete. 2023-11-27T15:16:50.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.226+0000 7faea72e5700 1 Processor -- start 2023-11-27T15:16:50.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.226+0000 7faea72e5700 1 -- start start 2023-11-27T15:16:50.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.227+0000 7faea72e5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea00ffd00 0x7faea0074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:50.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.227+0000 7faea72e5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0074dd0 con 0x7faea00ffd00 2023-11-27T15:16:50.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.227+0000 7faea5081700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea00ffd00 0x7faea0074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:50.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.227+0000 7faea5081700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea00ffd00 0x7faea0074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60356/0 (socket says 172.21.15.8:60356) 2023-11-27T15:16:50.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.227+0000 7faea5081700 1 -- 172.21.15.8:0/1048641166 learned_addr learned my addr 172.21.15.8:0/1048641166 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:50.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.227+0000 7faea5081700 1 -- 172.21.15.8:0/1048641166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c0095f0 con 0x7faea00ffd00 2023-11-27T15:16:50.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.228+0000 7faea5081700 1 --2- 172.21.15.8:0/1048641166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea00ffd00 0x7faea0074890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae9c004f40 tx=0x7fae9c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:50.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.228+0000 7fae967fc700 1 -- 172.21.15.8:0/1048641166 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae9c01d070 con 0x7faea00ffd00 2023-11-27T15:16:50.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.228+0000 7fae967fc700 1 -- 172.21.15.8:0/1048641166 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae9c00bc10 con 0x7faea00ffd00 2023-11-27T15:16:50.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.228+0000 7fae967fc700 1 -- 172.21.15.8:0/1048641166 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae9c003a70 con 0x7faea00ffd00 2023-11-27T15:16:50.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.228+0000 7faea72e5700 1 -- 172.21.15.8:0/1048641166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea0105bd0 con 0x7faea00ffd00 2023-11-27T15:16:50.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.228+0000 7faea72e5700 1 -- 172.21.15.8:0/1048641166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea0105a40 con 0x7faea00ffd00 2023-11-27T15:16:50.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.229+0000 7fae967fc700 1 -- 172.21.15.8:0/1048641166 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fae9c003d30 con 0x7faea00ffd00 2023-11-27T15:16:50.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.229+0000 7faea72e5700 1 -- 172.21.15.8:0/1048641166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea0062120 con 0x7faea00ffd00 2023-11-27T15:16:50.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.229+0000 7fae967fc700 1 --2- 172.21.15.8:0/1048641166 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fae8c0388c0 0x7fae8c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:50.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.230+0000 7fae967fc700 1 -- 172.21.15.8:0/1048641166 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fae9c04c5a0 con 0x7faea00ffd00 2023-11-27T15:16:50.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.230+0000 7faea4880700 1 --2- 172.21.15.8:0/1048641166 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fae8c0388c0 0x7fae8c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:50.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.230+0000 7faea4880700 1 --2- 172.21.15.8:0/1048641166 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fae8c0388c0 0x7fae8c03ad70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae90006fd0 tx=0x7fae90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:50.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.235+0000 7fae967fc700 1 -- 172.21.15.8:0/1048641166 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fae9c02ae20 con 0x7faea00ffd00 2023-11-27T15:16:50.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.442+0000 7faea72e5700 1 -- 172.21.15.8:0/1048641166 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7faea018e730 con 0x7faea00ffd00 2023-11-27T15:16:50.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.446+0000 7fae967fc700 1 -- 172.21.15.8:0/1048641166 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fae9c026070 con 0x7faea00ffd00 2023-11-27T15:16:50.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.448+0000 7faea72e5700 1 -- 172.21.15.8:0/1048641166 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fae8c0388c0 msgr2=0x7fae8c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:50.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.448+0000 7faea72e5700 1 --2- 172.21.15.8:0/1048641166 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fae8c0388c0 0x7fae8c03ad70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fae90006fd0 tx=0x7fae90006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:50.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.448+0000 7faea72e5700 1 -- 172.21.15.8:0/1048641166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea00ffd00 msgr2=0x7faea0074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:50.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.448+0000 7faea72e5700 1 --2- 172.21.15.8:0/1048641166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea00ffd00 0x7faea0074890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae9c004f40 tx=0x7fae9c0057c0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.448+0000 7faea72e5700 1 -- 172.21.15.8:0/1048641166 shutdown_connections 2023-11-27T15:16:50.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.448+0000 7faea72e5700 1 --2- 172.21.15.8:0/1048641166 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fae8c0388c0 0x7fae8c03ad70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.448+0000 7faea72e5700 1 --2- 172.21.15.8:0/1048641166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea00ffd00 0x7faea0074890 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:50.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.448+0000 7faea72e5700 1 -- 172.21.15.8:0/1048641166 >> 172.21.15.8:0/1048641166 conn(0x7faea00fb2b0 msgr2=0x7faea00fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:50.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.449+0000 7faea72e5700 1 -- 172.21.15.8:0/1048641166 shutdown_connections 2023-11-27T15:16:50.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:50.449+0000 7faea72e5700 1 -- 172.21.15.8:0/1048641166 wait complete. 2023-11-27T15:16:51.026 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-11-27T15:16:51.026 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-11-27T15:16:51.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:51 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1048641166' entity='client.admin' 2023-11-27T15:16:51.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:51.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:51.884 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:53.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.403+0000 7f25c679a700 1 -- 172.21.15.8:0/1731555650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00ffd00 msgr2=0x7f25c0100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.403+0000 7f25c679a700 1 --2- 172.21.15.8:0/1731555650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00ffd00 0x7f25c0100110 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009910 tx=0x7f25b0009c20 comp rx=0 tx=0).stop 2023-11-27T15:16:53.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.403+0000 7f25c679a700 1 -- 172.21.15.8:0/1731555650 shutdown_connections 2023-11-27T15:16:53.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.403+0000 7f25c679a700 1 --2- 172.21.15.8:0/1731555650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00ffd00 0x7f25c0100110 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.403+0000 7f25c679a700 1 -- 172.21.15.8:0/1731555650 >> 172.21.15.8:0/1731555650 conn(0x7f25c00fb2b0 msgr2=0x7f25c00fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:53.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.404+0000 7f25c679a700 1 -- 172.21.15.8:0/1731555650 shutdown_connections 2023-11-27T15:16:53.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.404+0000 7f25c679a700 1 -- 172.21.15.8:0/1731555650 wait complete. 2023-11-27T15:16:53.410 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.404+0000 7f25c679a700 1 Processor -- start 2023-11-27T15:16:53.410 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.405+0000 7f25c679a700 1 -- start start 2023-11-27T15:16:53.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.405+0000 7f25c679a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00ffd00 0x7f25c0196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.405+0000 7f25c679a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0197530 con 0x7f25c00ffd00 2023-11-27T15:16:53.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.405+0000 7f25bffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00ffd00 0x7f25c0196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.405+0000 7f25bffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00ffd00 0x7f25c0196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60374/0 (socket says 172.21.15.8:60374) 2023-11-27T15:16:53.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.405+0000 7f25bffff700 1 -- 172.21.15.8:0/2279070426 learned_addr learned my addr 172.21.15.8:0/2279070426 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:53.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.406+0000 7f25bffff700 1 -- 172.21.15.8:0/2279070426 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b00095f0 con 0x7f25c00ffd00 2023-11-27T15:16:53.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.406+0000 7f25bffff700 1 --2- 172.21.15.8:0/2279070426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00ffd00 0x7f25c0196ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25b0004f40 tx=0x7f25b00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:53.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.406+0000 7f25bd7fa700 1 -- 172.21.15.8:0/2279070426 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b001d070 con 0x7f25c00ffd00 2023-11-27T15:16:53.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.406+0000 7f25bd7fa700 1 -- 172.21.15.8:0/2279070426 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25b000bc10 con 0x7f25c00ffd00 2023-11-27T15:16:53.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.406+0000 7f25bd7fa700 1 -- 172.21.15.8:0/2279070426 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b0003a70 con 0x7f25c00ffd00 2023-11-27T15:16:53.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.406+0000 7f25c679a700 1 -- 172.21.15.8:0/2279070426 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c0105bd0 con 0x7f25c00ffd00 2023-11-27T15:16:53.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.406+0000 7f25c679a700 1 -- 172.21.15.8:0/2279070426 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c0105a40 con 0x7f25c00ffd00 2023-11-27T15:16:53.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.408+0000 7f25bd7fa700 1 -- 172.21.15.8:0/2279070426 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f25b0003ce0 con 0x7f25c00ffd00 2023-11-27T15:16:53.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.408+0000 7f25c679a700 1 -- 172.21.15.8:0/2279070426 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c0105d60 con 0x7f25c00ffd00 2023-11-27T15:16:53.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.408+0000 7f25bd7fa700 1 --2- 172.21.15.8:0/2279070426 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f25a80388c0 0x7f25a803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:53.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.408+0000 7f25bd7fa700 1 -- 172.21.15.8:0/2279070426 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f25b004c470 con 0x7f25c00ffd00 2023-11-27T15:16:53.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.408+0000 7f25bf7fe700 1 --2- 172.21.15.8:0/2279070426 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f25a80388c0 0x7f25a803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:53.413 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.409+0000 7f25bf7fe700 1 --2- 172.21.15.8:0/2279070426 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f25a80388c0 0x7f25a803ad70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f25b4006fd0 tx=0x7f25b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:53.417 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.413+0000 7f25bd7fa700 1 -- 172.21.15.8:0/2279070426 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f25b002aba0 con 0x7f25c00ffd00 2023-11-27T15:16:53.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.625+0000 7f25c679a700 1 -- 172.21.15.8:0/2279070426 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f25c0060f30 con 0x7f25a80388c0 2023-11-27T15:16:53.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.629+0000 7f25bd7fa700 1 -- 172.21.15.8:0/2279070426 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f25c0060f30 con 0x7f25a80388c0 2023-11-27T15:16:53.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.631+0000 7f25c679a700 1 -- 172.21.15.8:0/2279070426 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f25a80388c0 msgr2=0x7f25a803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.631+0000 7f25c679a700 1 --2- 172.21.15.8:0/2279070426 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f25a80388c0 0x7f25a803ad70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f25b4006fd0 tx=0x7f25b4006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:53.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.631+0000 7f25c679a700 1 -- 172.21.15.8:0/2279070426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00ffd00 msgr2=0x7f25c0196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.631+0000 7f25c679a700 1 --2- 172.21.15.8:0/2279070426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00ffd00 0x7f25c0196ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25b0004f40 tx=0x7f25b00057c0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.631+0000 7f25c679a700 1 -- 172.21.15.8:0/2279070426 shutdown_connections 2023-11-27T15:16:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.631+0000 7f25c679a700 1 --2- 172.21.15.8:0/2279070426 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f25a80388c0 0x7f25a803ad70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.631+0000 7f25c679a700 1 --2- 172.21.15.8:0/2279070426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25c00ffd00 0x7f25c0196ff0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.631+0000 7f25c679a700 1 -- 172.21.15.8:0/2279070426 >> 172.21.15.8:0/2279070426 conn(0x7f25c00fb2b0 msgr2=0x7f25c00fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.631+0000 7f25c679a700 1 -- 172.21.15.8:0/2279070426 shutdown_connections 2023-11-27T15:16:53.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:53.631+0000 7f25c679a700 1 -- 172.21.15.8:0/2279070426 wait complete. 2023-11-27T15:16:54.166 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi029 2023-11-27T15:16:54.166 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T15:16:54.166 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/etc/ceph/ceph.conf 2023-11-27T15:16:54.187 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T15:16:54.187 DEBUG:teuthology.orchestra.run.smithi029:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:16:54.247 INFO:tasks.cephadm:Adding host smithi029 to orchestrator... 2023-11-27T15:16:54.248 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch host add smithi029 2023-11-27T15:16:55.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:54 smithi008 ceph-mon[128788]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:55.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:54 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:16:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:54 smithi008 ceph-mon[128788]: Updating smithi008:/etc/ceph/ceph.conf 2023-11-27T15:16:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:54 smithi008 ceph-mon[128788]: Updating smithi008:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:16:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:55.687 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:55 smithi008 ceph-mon[128788]: Updating smithi008:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:16:55.687 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:55 smithi008 ceph-mon[128788]: Updating smithi008:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:16:56.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.622+0000 7fbbee6f0700 1 -- 172.21.15.8:0/634087494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbbe80fff90 msgr2=0x7fbbe8100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:56.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.622+0000 7fbbee6f0700 1 --2- 172.21.15.8:0/634087494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbbe80fff90 0x7fbbe8100360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009910 tx=0x7fbbd8009c20 comp rx=0 tx=0).stop 2023-11-27T15:16:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.622+0000 7fbbee6f0700 1 -- 172.21.15.8:0/634087494 shutdown_connections 2023-11-27T15:16:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.622+0000 7fbbee6f0700 1 --2- 172.21.15.8:0/634087494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbbe80fff90 0x7fbbe8100360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.622+0000 7fbbee6f0700 1 -- 172.21.15.8:0/634087494 >> 172.21.15.8:0/634087494 conn(0x7fbbe80fb850 msgr2=0x7fbbe80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.622+0000 7fbbee6f0700 1 -- 172.21.15.8:0/634087494 shutdown_connections 2023-11-27T15:16:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.622+0000 7fbbee6f0700 1 -- 172.21.15.8:0/634087494 wait complete. 2023-11-27T15:16:56.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.623+0000 7fbbee6f0700 1 Processor -- start 2023-11-27T15:16:56.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.623+0000 7fbbee6f0700 1 -- start start 2023-11-27T15:16:56.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.623+0000 7fbbee6f0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbbe80fff90 0x7fbbe81951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:56.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.623+0000 7fbbee6f0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8195700 con 0x7fbbe80fff90 2023-11-27T15:16:56.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.624+0000 7fbbe7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbbe80fff90 0x7fbbe81951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:56.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.624+0000 7fbbe7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbbe80fff90 0x7fbbe81951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60400/0 (socket says 172.21.15.8:60400) 2023-11-27T15:16:56.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.624+0000 7fbbe7fff700 1 -- 172.21.15.8:0/3490169223 learned_addr learned my addr 172.21.15.8:0/3490169223 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:16:56.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.624+0000 7fbbe7fff700 1 -- 172.21.15.8:0/3490169223 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80095f0 con 0x7fbbe80fff90 2023-11-27T15:16:56.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.624+0000 7fbbe7fff700 1 --2- 172.21.15.8:0/3490169223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbbe80fff90 0x7fbbe81951c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8006010 tx=0x7fbbd8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:56.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.624+0000 7fbbe57fa700 1 -- 172.21.15.8:0/3490169223 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbd801d070 con 0x7fbbe80fff90 2023-11-27T15:16:56.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.624+0000 7fbbe57fa700 1 -- 172.21.15.8:0/3490169223 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbd800bc10 con 0x7fbbe80fff90 2023-11-27T15:16:56.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.624+0000 7fbbee6f0700 1 -- 172.21.15.8:0/3490169223 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe810a170 con 0x7fbbe80fff90 2023-11-27T15:16:56.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.624+0000 7fbbe57fa700 1 -- 172.21.15.8:0/3490169223 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbd8003910 con 0x7fbbe80fff90 2023-11-27T15:16:56.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.625+0000 7fbbee6f0700 1 -- 172.21.15.8:0/3490169223 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe8109fe0 con 0x7fbbe80fff90 2023-11-27T15:16:56.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.626+0000 7fbbee6f0700 1 -- 172.21.15.8:0/3490169223 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe810a300 con 0x7fbbe80fff90 2023-11-27T15:16:56.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.626+0000 7fbbe57fa700 1 -- 172.21.15.8:0/3490169223 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fbbd800bd80 con 0x7fbbe80fff90 2023-11-27T15:16:56.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.626+0000 7fbbe57fa700 1 --2- 172.21.15.8:0/3490169223 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbbd00388c0 0x7fbbd003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:56.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.626+0000 7fbbe57fa700 1 -- 172.21.15.8:0/3490169223 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbbd804c5a0 con 0x7fbbe80fff90 2023-11-27T15:16:56.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.626+0000 7fbbe77fe700 1 --2- 172.21.15.8:0/3490169223 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbbd00388c0 0x7fbbd003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:56.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.627+0000 7fbbe77fe700 1 --2- 172.21.15.8:0/3490169223 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbbd00388c0 0x7fbbd003ad70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc006fd0 tx=0x7fbbdc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:56.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.631+0000 7fbbe57fa700 1 -- 172.21.15.8:0/3490169223 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbbd8029360 con 0x7fbbe80fff90 2023-11-27T15:16:56.846 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:56.841+0000 7fbbee6f0700 1 -- 172.21.15.8:0/3490169223 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi029", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbe8195d00 con 0x7fbbd00388c0 2023-11-27T15:16:57.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:56 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:57.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:56 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:57.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:56 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:58.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:57 smithi008 ceph-mon[128788]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi029", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:58.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:57 smithi008 ceph-mon[128788]: Deploying cephadm binary to smithi029 2023-11-27T15:16:59.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.031+0000 7fbbe57fa700 1 -- 172.21.15.8:0/3490169223 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fbbe8195d00 con 0x7fbbd00388c0 2023-11-27T15:16:59.036 INFO:teuthology.orchestra.run.smithi008.stdout:Added host 'smithi029' with addr '172.21.15.29' 2023-11-27T15:16:59.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.033+0000 7fbbee6f0700 1 -- 172.21.15.8:0/3490169223 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbbd00388c0 msgr2=0x7fbbd003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:59.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.033+0000 7fbbee6f0700 1 --2- 172.21.15.8:0/3490169223 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbbd00388c0 0x7fbbd003ad70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc006fd0 tx=0x7fbbdc006e40 comp rx=0 tx=0).stop 2023-11-27T15:16:59.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.033+0000 7fbbee6f0700 1 -- 172.21.15.8:0/3490169223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbbe80fff90 msgr2=0x7fbbe81951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:59.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.033+0000 7fbbee6f0700 1 --2- 172.21.15.8:0/3490169223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbbe80fff90 0x7fbbe81951c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8006010 tx=0x7fbbd8004ff0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.033+0000 7fbbee6f0700 1 -- 172.21.15.8:0/3490169223 shutdown_connections 2023-11-27T15:16:59.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.033+0000 7fbbee6f0700 1 --2- 172.21.15.8:0/3490169223 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbbd00388c0 0x7fbbd003ad70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.033+0000 7fbbee6f0700 1 --2- 172.21.15.8:0/3490169223 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbbe80fff90 0x7fbbe81951c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:59.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.033+0000 7fbbee6f0700 1 -- 172.21.15.8:0/3490169223 >> 172.21.15.8:0/3490169223 conn(0x7fbbe80fb850 msgr2=0x7fbbe8104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:59.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.034+0000 7fbbee6f0700 1 -- 172.21.15.8:0/3490169223 shutdown_connections 2023-11-27T15:16:59.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:16:59.034+0000 7fbbee6f0700 1 -- 172.21.15.8:0/3490169223 wait complete. 2023-11-27T15:16:59.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:59 smithi008 ceph-mon[128788]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:16:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:59 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:16:59.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:16:59 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:59.521 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-11-27T15:17:00.378 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:00.786 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:00 smithi008 ceph-mon[128788]: Added host smithi029 2023-11-27T15:17:00.786 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:00.786 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:01.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:01 smithi008 ceph-mon[128788]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:01.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.896+0000 7f024fff0700 1 -- 172.21.15.8:0/2751053677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02480fff90 msgr2=0x7f0248100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:01.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.896+0000 7f024fff0700 1 --2- 172.21.15.8:0/2751053677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02480fff90 0x7f0248100360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0244009910 tx=0x7f0244009c20 comp rx=0 tx=0).stop 2023-11-27T15:17:01.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.896+0000 7f024fff0700 1 -- 172.21.15.8:0/2751053677 shutdown_connections 2023-11-27T15:17:01.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.896+0000 7f024fff0700 1 --2- 172.21.15.8:0/2751053677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02480fff90 0x7f0248100360 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:01.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.896+0000 7f024fff0700 1 -- 172.21.15.8:0/2751053677 >> 172.21.15.8:0/2751053677 conn(0x7f02480fb850 msgr2=0x7f02480fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:01.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.896+0000 7f024fff0700 1 -- 172.21.15.8:0/2751053677 shutdown_connections 2023-11-27T15:17:01.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.896+0000 7f024fff0700 1 -- 172.21.15.8:0/2751053677 wait complete. 2023-11-27T15:17:01.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.897+0000 7f024fff0700 1 Processor -- start 2023-11-27T15:17:01.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.897+0000 7f024fff0700 1 -- start start 2023-11-27T15:17:01.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.897+0000 7f024fff0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02480fff90 0x7f02481951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:01.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.897+0000 7f024fff0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0248195700 con 0x7f02480fff90 2023-11-27T15:17:01.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.897+0000 7f024dd8c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02480fff90 0x7f02481951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:01.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.897+0000 7f024dd8c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02480fff90 0x7f02481951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:52114/0 (socket says 172.21.15.8:52114) 2023-11-27T15:17:01.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.897+0000 7f024dd8c700 1 -- 172.21.15.8:0/1446882843 learned_addr learned my addr 172.21.15.8:0/1446882843 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:17:01.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.898+0000 7f024dd8c700 1 -- 172.21.15.8:0/1446882843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02440095f0 con 0x7f02480fff90 2023-11-27T15:17:01.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.898+0000 7f024dd8c700 1 --2- 172.21.15.8:0/1446882843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02480fff90 0x7f02481951c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0244006010 tx=0x7f0244004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:01.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.898+0000 7f023effd700 1 -- 172.21.15.8:0/1446882843 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f024401d070 con 0x7f02480fff90 2023-11-27T15:17:01.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.898+0000 7f023effd700 1 -- 172.21.15.8:0/1446882843 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f024400bc10 con 0x7f02480fff90 2023-11-27T15:17:01.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.898+0000 7f023effd700 1 -- 172.21.15.8:0/1446882843 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0244003910 con 0x7f02480fff90 2023-11-27T15:17:01.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.898+0000 7f024fff0700 1 -- 172.21.15.8:0/1446882843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024810a170 con 0x7f02480fff90 2023-11-27T15:17:01.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.898+0000 7f024fff0700 1 -- 172.21.15.8:0/1446882843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0248109fe0 con 0x7f02480fff90 2023-11-27T15:17:01.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.899+0000 7f023effd700 1 -- 172.21.15.8:0/1446882843 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f024400bd80 con 0x7f02480fff90 2023-11-27T15:17:01.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.899+0000 7f024fff0700 1 -- 172.21.15.8:0/1446882843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024810a300 con 0x7f02480fff90 2023-11-27T15:17:01.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.900+0000 7f023effd700 1 --2- 172.21.15.8:0/1446882843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0234038870 0x7f023403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:01.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.900+0000 7f023effd700 1 -- 172.21.15.8:0/1446882843 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f024404c600 con 0x7f02480fff90 2023-11-27T15:17:01.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.900+0000 7f024d58b700 1 --2- 172.21.15.8:0/1446882843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0234038870 0x7f023403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:01.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.901+0000 7f024d58b700 1 --2- 172.21.15.8:0/1446882843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0234038870 0x7f023403ad20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0238006fd0 tx=0x7f0238006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:01.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:01.905+0000 7f023effd700 1 -- 172.21.15.8:0/1446882843 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0244029360 con 0x7f02480fff90 2023-11-27T15:17:02.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.117+0000 7f024fff0700 1 -- 172.21.15.8:0/1446882843 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02481080d0 con 0x7f0234038870 2023-11-27T15:17:02.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.118+0000 7f023effd700 1 -- 172.21.15.8:0/1446882843 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7f02481080d0 con 0x7f0234038870 2023-11-27T15:17:02.124 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:17:02.124 INFO:teuthology.orchestra.run.smithi008.stdout:[{"addr": "172.21.15.8", "hostname": "smithi008", "labels": [], "status": ""}, {"addr": "172.21.15.29", "hostname": "smithi029", "labels": [], "status": ""}] 2023-11-27T15:17:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.121+0000 7f024fff0700 1 -- 172.21.15.8:0/1446882843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0234038870 msgr2=0x7f023403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.121+0000 7f024fff0700 1 --2- 172.21.15.8:0/1446882843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0234038870 0x7f023403ad20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0238006fd0 tx=0x7f0238006e40 comp rx=0 tx=0).stop 2023-11-27T15:17:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.121+0000 7f024fff0700 1 -- 172.21.15.8:0/1446882843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02480fff90 msgr2=0x7f02481951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.121+0000 7f024fff0700 1 --2- 172.21.15.8:0/1446882843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02480fff90 0x7f02481951c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0244006010 tx=0x7f0244004ff0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.121+0000 7f024fff0700 1 -- 172.21.15.8:0/1446882843 shutdown_connections 2023-11-27T15:17:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.121+0000 7f024fff0700 1 --2- 172.21.15.8:0/1446882843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0234038870 0x7f023403ad20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.121+0000 7f024fff0700 1 --2- 172.21.15.8:0/1446882843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02480fff90 0x7f02481951c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:02.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.121+0000 7f024fff0700 1 -- 172.21.15.8:0/1446882843 >> 172.21.15.8:0/1446882843 conn(0x7f02480fb850 msgr2=0x7f0248104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:02.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.121+0000 7f024fff0700 1 -- 172.21.15.8:0/1446882843 shutdown_connections 2023-11-27T15:17:02.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:02.121+0000 7f024fff0700 1 -- 172.21.15.8:0/1446882843 wait complete. 2023-11-27T15:17:02.656 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi060 2023-11-27T15:17:02.657 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T15:17:02.657 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/etc/ceph/ceph.conf 2023-11-27T15:17:02.677 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T15:17:02.678 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:17:02.738 INFO:tasks.cephadm:Adding host smithi060 to orchestrator... 2023-11-27T15:17:02.741 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch host add smithi060 2023-11-27T15:17:02.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:02 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:02.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:02 smithi008 ceph-mon[128788]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:02.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:02 smithi008 ceph-mon[128788]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:03.600 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:05.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.131+0000 7f9bc567a700 1 -- 172.21.15.8:0/1812629257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bc00fff90 msgr2=0x7f9bc0100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:05.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.131+0000 7f9bc567a700 1 --2- 172.21.15.8:0/1812629257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bc00fff90 0x7f9bc0100360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009910 tx=0x7f9bb0009c20 comp rx=0 tx=0).stop 2023-11-27T15:17:05.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.131+0000 7f9bc567a700 1 -- 172.21.15.8:0/1812629257 shutdown_connections 2023-11-27T15:17:05.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.131+0000 7f9bc567a700 1 --2- 172.21.15.8:0/1812629257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bc00fff90 0x7f9bc0100360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:05.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.131+0000 7f9bc567a700 1 -- 172.21.15.8:0/1812629257 >> 172.21.15.8:0/1812629257 conn(0x7f9bc00fb850 msgr2=0x7f9bc00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:05.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.131+0000 7f9bc567a700 1 -- 172.21.15.8:0/1812629257 shutdown_connections 2023-11-27T15:17:05.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.131+0000 7f9bc567a700 1 -- 172.21.15.8:0/1812629257 wait complete. 2023-11-27T15:17:05.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.132+0000 7f9bc567a700 1 Processor -- start 2023-11-27T15:17:05.137 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.132+0000 7f9bc567a700 1 -- start start 2023-11-27T15:17:05.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.133+0000 7f9bc567a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bc00fff90 0x7f9bc01951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:05.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.133+0000 7f9bc567a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc0195700 con 0x7f9bc00fff90 2023-11-27T15:17:05.138 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.133+0000 7f9bbeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bc00fff90 0x7f9bc01951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:05.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.133+0000 7f9bbeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bc00fff90 0x7f9bc01951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:52138/0 (socket says 172.21.15.8:52138) 2023-11-27T15:17:05.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.133+0000 7f9bbeffd700 1 -- 172.21.15.8:0/2003085777 learned_addr learned my addr 172.21.15.8:0/2003085777 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:17:05.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.133+0000 7f9bbeffd700 1 -- 172.21.15.8:0/2003085777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb00095f0 con 0x7f9bc00fff90 2023-11-27T15:17:05.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.133+0000 7f9bbeffd700 1 --2- 172.21.15.8:0/2003085777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bc00fff90 0x7f9bc01951c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0006010 tx=0x7f9bb0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:05.139 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.134+0000 7f9ba7fff700 1 -- 172.21.15.8:0/2003085777 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bb001d070 con 0x7f9bc00fff90 2023-11-27T15:17:05.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.134+0000 7f9ba7fff700 1 -- 172.21.15.8:0/2003085777 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bb000bc10 con 0x7f9bc00fff90 2023-11-27T15:17:05.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.134+0000 7f9bc567a700 1 -- 172.21.15.8:0/2003085777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc010a170 con 0x7f9bc00fff90 2023-11-27T15:17:05.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.134+0000 7f9ba7fff700 1 -- 172.21.15.8:0/2003085777 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bb0003910 con 0x7f9bc00fff90 2023-11-27T15:17:05.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.134+0000 7f9bc567a700 1 -- 172.21.15.8:0/2003085777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc0109fe0 con 0x7f9bc00fff90 2023-11-27T15:17:05.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.135+0000 7f9bc567a700 1 -- 172.21.15.8:0/2003085777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc010a300 con 0x7f9bc00fff90 2023-11-27T15:17:05.140 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.135+0000 7f9ba7fff700 1 -- 172.21.15.8:0/2003085777 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f9bb000bd80 con 0x7f9bc00fff90 2023-11-27T15:17:05.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.136+0000 7f9ba7fff700 1 --2- 172.21.15.8:0/2003085777 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ba80388c0 0x7f9ba803ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:05.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.136+0000 7f9ba7fff700 1 -- 172.21.15.8:0/2003085777 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9bb004c540 con 0x7f9bc00fff90 2023-11-27T15:17:05.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.136+0000 7f9bbe7fc700 1 --2- 172.21.15.8:0/2003085777 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ba80388c0 0x7f9ba803ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:05.141 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.137+0000 7f9bbe7fc700 1 --2- 172.21.15.8:0/2003085777 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ba80388c0 0x7f9ba803ad70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4006fd0 tx=0x7f9bb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:05.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.141+0000 7f9ba7fff700 1 -- 172.21.15.8:0/2003085777 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9bb002ad40 con 0x7f9bc00fff90 2023-11-27T15:17:05.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:05.369+0000 7f9bc567a700 1 -- 172.21.15.8:0/2003085777 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi060", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bc01080d0 con 0x7f9ba80388c0 2023-11-27T15:17:05.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:05.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi029", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: Updating smithi029:/etc/ceph/ceph.conf 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: Updating smithi029:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:05.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:06.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:06 smithi008 ceph-mon[128788]: Updating smithi029:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:17:06.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:06 smithi008 ceph-mon[128788]: Updating smithi029:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:17:06.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:06 smithi008 ceph-mon[128788]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi060", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:07.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.550+0000 7f9ba7fff700 1 -- 172.21.15.8:0/2003085777 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f9bc01080d0 con 0x7f9ba80388c0 2023-11-27T15:17:07.554 INFO:teuthology.orchestra.run.smithi008.stdout:Added host 'smithi060' with addr '172.21.15.60' 2023-11-27T15:17:07.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.552+0000 7f9bc567a700 1 -- 172.21.15.8:0/2003085777 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ba80388c0 msgr2=0x7f9ba803ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.552+0000 7f9bc567a700 1 --2- 172.21.15.8:0/2003085777 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ba80388c0 0x7f9ba803ad70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4006fd0 tx=0x7f9bb4006e40 comp rx=0 tx=0).stop 2023-11-27T15:17:07.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.552+0000 7f9bc567a700 1 -- 172.21.15.8:0/2003085777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bc00fff90 msgr2=0x7f9bc01951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:07.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.552+0000 7f9bc567a700 1 --2- 172.21.15.8:0/2003085777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bc00fff90 0x7f9bc01951c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0006010 tx=0x7f9bb0004ff0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.553+0000 7f9bc567a700 1 -- 172.21.15.8:0/2003085777 shutdown_connections 2023-11-27T15:17:07.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.553+0000 7f9bc567a700 1 --2- 172.21.15.8:0/2003085777 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ba80388c0 0x7f9ba803ad70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.553+0000 7f9bc567a700 1 --2- 172.21.15.8:0/2003085777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bc00fff90 0x7f9bc01951c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:07.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.553+0000 7f9bc567a700 1 -- 172.21.15.8:0/2003085777 >> 172.21.15.8:0/2003085777 conn(0x7f9bc00fb850 msgr2=0x7f9bc0104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:07.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.553+0000 7f9bc567a700 1 -- 172.21.15.8:0/2003085777 shutdown_connections 2023-11-27T15:17:07.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:07.553+0000 7f9bc567a700 1 -- 172.21.15.8:0/2003085777 wait complete. 2023-11-27T15:17:07.580 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:07 smithi008 ceph-mon[128788]: Deploying cephadm binary to smithi060 2023-11-27T15:17:07.580 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:07 smithi008 ceph-mon[128788]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:07.989 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-11-27T15:17:08.853 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:08 smithi008 ceph-mon[128788]: Added host smithi060 2023-11-27T15:17:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:08.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:08.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:08 smithi008 ceph-mon[128788]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:10.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:10.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.363+0000 7f32599bb700 1 -- 172.21.15.8:0/1243741005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3254072e90 msgr2=0x7f3254073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.363+0000 7f32599bb700 1 --2- 172.21.15.8:0/1243741005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3254072e90 0x7f3254073260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3244009910 tx=0x7f3244009c20 comp rx=0 tx=0).stop 2023-11-27T15:17:10.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.363+0000 7f32599bb700 1 -- 172.21.15.8:0/1243741005 shutdown_connections 2023-11-27T15:17:10.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.363+0000 7f32599bb700 1 --2- 172.21.15.8:0/1243741005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3254072e90 0x7f3254073260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.363+0000 7f32599bb700 1 -- 172.21.15.8:0/1243741005 >> 172.21.15.8:0/1243741005 conn(0x7f32540fb850 msgr2=0x7f32540fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.363+0000 7f32599bb700 1 -- 172.21.15.8:0/1243741005 shutdown_connections 2023-11-27T15:17:10.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.363+0000 7f32599bb700 1 -- 172.21.15.8:0/1243741005 wait complete. 2023-11-27T15:17:10.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.364+0000 7f32599bb700 1 Processor -- start 2023-11-27T15:17:10.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.364+0000 7f32599bb700 1 -- start start 2023-11-27T15:17:10.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.364+0000 7f32599bb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3254072e90 0x7f3254197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.364+0000 7f32599bb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3254197880 con 0x7f3254072e90 2023-11-27T15:17:10.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.365+0000 7f3252ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3254072e90 0x7f3254197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.365+0000 7f3252ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3254072e90 0x7f3254197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55206/0 (socket says 172.21.15.8:55206) 2023-11-27T15:17:10.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.365+0000 7f3252ffd700 1 -- 172.21.15.8:0/2741407835 learned_addr learned my addr 172.21.15.8:0/2741407835 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:17:10.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.365+0000 7f3252ffd700 1 -- 172.21.15.8:0/2741407835 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32440095f0 con 0x7f3254072e90 2023-11-27T15:17:10.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.365+0000 7f3252ffd700 1 --2- 172.21.15.8:0/2741407835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3254072e90 0x7f3254197340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3244005850 tx=0x7f3244004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.365+0000 7f32589b9700 1 -- 172.21.15.8:0/2741407835 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f324401d070 con 0x7f3254072e90 2023-11-27T15:17:10.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.366+0000 7f32589b9700 1 -- 172.21.15.8:0/2741407835 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3244004370 con 0x7f3254072e90 2023-11-27T15:17:10.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.366+0000 7f32589b9700 1 -- 172.21.15.8:0/2741407835 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3244003ab0 con 0x7f3254072e90 2023-11-27T15:17:10.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.366+0000 7f32599bb700 1 -- 172.21.15.8:0/2741407835 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325410c390 con 0x7f3254072e90 2023-11-27T15:17:10.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.366+0000 7f32599bb700 1 -- 172.21.15.8:0/2741407835 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325410c200 con 0x7f3254072e90 2023-11-27T15:17:10.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.367+0000 7f32599bb700 1 -- 172.21.15.8:0/2741407835 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325410c520 con 0x7f3254072e90 2023-11-27T15:17:10.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.367+0000 7f32589b9700 1 -- 172.21.15.8:0/2741407835 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f3244003d20 con 0x7f3254072e90 2023-11-27T15:17:10.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.367+0000 7f32589b9700 1 --2- 172.21.15.8:0/2741407835 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f323c0388c0 0x7f323c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:10.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.367+0000 7f32589b9700 1 -- 172.21.15.8:0/2741407835 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f324404c5b0 con 0x7f3254072e90 2023-11-27T15:17:10.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.367+0000 7f32527fc700 1 --2- 172.21.15.8:0/2741407835 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f323c0388c0 0x7f323c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:10.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.368+0000 7f32527fc700 1 --2- 172.21.15.8:0/2741407835 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f323c0388c0 0x7f323c03ad70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3248006fd0 tx=0x7f3248006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:10.376 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.372+0000 7f32589b9700 1 -- 172.21.15.8:0/2741407835 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f32440044e0 con 0x7f3254072e90 2023-11-27T15:17:10.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.584+0000 7f32599bb700 1 -- 172.21.15.8:0/2741407835 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3254109610 con 0x7f323c0388c0 2023-11-27T15:17:10.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.586+0000 7f32589b9700 1 -- 172.21.15.8:0/2741407835 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+237 (secure 0 0 0) 0x7f3254109610 con 0x7f323c0388c0 2023-11-27T15:17:10.590 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:17:10.590 INFO:teuthology.orchestra.run.smithi008.stdout:[{"addr": "172.21.15.8", "hostname": "smithi008", "labels": [], "status": ""}, {"addr": "172.21.15.29", "hostname": "smithi029", "labels": [], "status": ""}, {"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}] 2023-11-27T15:17:10.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.588+0000 7f32599bb700 1 -- 172.21.15.8:0/2741407835 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f323c0388c0 msgr2=0x7f323c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.588+0000 7f32599bb700 1 --2- 172.21.15.8:0/2741407835 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f323c0388c0 0x7f323c03ad70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3248006fd0 tx=0x7f3248006e40 comp rx=0 tx=0).stop 2023-11-27T15:17:10.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.588+0000 7f32599bb700 1 -- 172.21.15.8:0/2741407835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3254072e90 msgr2=0x7f3254197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:10.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.588+0000 7f32599bb700 1 --2- 172.21.15.8:0/2741407835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3254072e90 0x7f3254197340 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3244005850 tx=0x7f3244004ff0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.588+0000 7f32599bb700 1 -- 172.21.15.8:0/2741407835 shutdown_connections 2023-11-27T15:17:10.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.588+0000 7f32599bb700 1 --2- 172.21.15.8:0/2741407835 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f323c0388c0 0x7f323c03ad70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.588+0000 7f32599bb700 1 --2- 172.21.15.8:0/2741407835 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3254072e90 0x7f3254197340 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:10.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.588+0000 7f32599bb700 1 -- 172.21.15.8:0/2741407835 >> 172.21.15.8:0/2741407835 conn(0x7f32540fb850 msgr2=0x7f3254106870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:10.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.588+0000 7f32599bb700 1 -- 172.21.15.8:0/2741407835 shutdown_connections 2023-11-27T15:17:10.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:10.588+0000 7f32599bb700 1 -- 172.21.15.8:0/2741407835 wait complete. 2023-11-27T15:17:11.135 INFO:tasks.cephadm:Setting crush tunables to default 2023-11-27T15:17:11.135 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd crush tunables default 2023-11-27T15:17:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:11 smithi008 ceph-mon[128788]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:11.994 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:12.298 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:12 smithi008 ceph-mon[128788]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:13.240 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:13 smithi008 ceph-mon[128788]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:13.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.480+0000 7fa4786b7700 1 -- 172.21.15.8:0/3453236047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4700fff90 msgr2=0x7fa470100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:13.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.480+0000 7fa4786b7700 1 --2- 172.21.15.8:0/3453236047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4700fff90 0x7fa470100360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009910 tx=0x7fa46c009c20 comp rx=0 tx=0).stop 2023-11-27T15:17:13.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.481+0000 7fa4786b7700 1 -- 172.21.15.8:0/3453236047 shutdown_connections 2023-11-27T15:17:13.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.481+0000 7fa4786b7700 1 --2- 172.21.15.8:0/3453236047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4700fff90 0x7fa470100360 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:13.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.481+0000 7fa4786b7700 1 -- 172.21.15.8:0/3453236047 >> 172.21.15.8:0/3453236047 conn(0x7fa4700fb850 msgr2=0x7fa4700fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:13.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.481+0000 7fa4786b7700 1 -- 172.21.15.8:0/3453236047 shutdown_connections 2023-11-27T15:17:13.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.481+0000 7fa4786b7700 1 -- 172.21.15.8:0/3453236047 wait complete. 2023-11-27T15:17:13.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.482+0000 7fa4786b7700 1 Processor -- start 2023-11-27T15:17:13.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.482+0000 7fa4786b7700 1 -- start start 2023-11-27T15:17:13.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.482+0000 7fa4786b7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4700fff90 0x7fa47018f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:13.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.482+0000 7fa4786b7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701937b0 con 0x7fa4700fff90 2023-11-27T15:17:13.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.483+0000 7fa476453700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4700fff90 0x7fa47018f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:13.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.483+0000 7fa476453700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4700fff90 0x7fa47018f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55228/0 (socket says 172.21.15.8:55228) 2023-11-27T15:17:13.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.483+0000 7fa476453700 1 -- 172.21.15.8:0/2905297735 learned_addr learned my addr 172.21.15.8:0/2905297735 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:17:13.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.483+0000 7fa476453700 1 -- 172.21.15.8:0/2905297735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c0095f0 con 0x7fa4700fff90 2023-11-27T15:17:13.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.483+0000 7fa476453700 1 --2- 172.21.15.8:0/2905297735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4700fff90 0x7fa47018f7c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa46c006010 tx=0x7fa46c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:13.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.484+0000 7fa4677fe700 1 -- 172.21.15.8:0/2905297735 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa46c01d070 con 0x7fa4700fff90 2023-11-27T15:17:13.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.484+0000 7fa4677fe700 1 -- 172.21.15.8:0/2905297735 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46c00bc10 con 0x7fa4700fff90 2023-11-27T15:17:13.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.484+0000 7fa4786b7700 1 -- 172.21.15.8:0/2905297735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47010a170 con 0x7fa4700fff90 2023-11-27T15:17:13.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.484+0000 7fa4677fe700 1 -- 172.21.15.8:0/2905297735 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa46c003910 con 0x7fa4700fff90 2023-11-27T15:17:13.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.484+0000 7fa4786b7700 1 -- 172.21.15.8:0/2905297735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa470109fe0 con 0x7fa4700fff90 2023-11-27T15:17:13.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.485+0000 7fa4677fe700 1 -- 172.21.15.8:0/2905297735 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fa46c003b80 con 0x7fa4700fff90 2023-11-27T15:17:13.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.485+0000 7fa4786b7700 1 -- 172.21.15.8:0/2905297735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47010a300 con 0x7fa4700fff90 2023-11-27T15:17:13.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.486+0000 7fa4677fe700 1 --2- 172.21.15.8:0/2905297735 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa45c0388c0 0x7fa45c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:13.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.486+0000 7fa4677fe700 1 -- 172.21.15.8:0/2905297735 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa46c04c520 con 0x7fa4700fff90 2023-11-27T15:17:13.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.486+0000 7fa475c52700 1 --2- 172.21.15.8:0/2905297735 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa45c0388c0 0x7fa45c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:13.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.486+0000 7fa475c52700 1 --2- 172.21.15.8:0/2905297735 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa45c0388c0 0x7fa45c03ad70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa460006fd0 tx=0x7fa460006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:13.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.491+0000 7fa4677fe700 1 -- 172.21.15.8:0/2905297735 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa46c00ba70 con 0x7fa4700fff90 2023-11-27T15:17:13.708 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:13.704+0000 7fa4786b7700 1 -- 172.21.15.8:0/2905297735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fa470190350 con 0x7fa4700fff90 2023-11-27T15:17:14.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.054+0000 7fa4677fe700 1 -- 172.21.15.8:0/2905297735 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fa46c026020 con 0x7fa4700fff90 2023-11-27T15:17:14.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.056+0000 7fa4786b7700 1 -- 172.21.15.8:0/2905297735 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa45c0388c0 msgr2=0x7fa45c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:14.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.056+0000 7fa4786b7700 1 --2- 172.21.15.8:0/2905297735 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa45c0388c0 0x7fa45c03ad70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa460006fd0 tx=0x7fa460006e40 comp rx=0 tx=0).stop 2023-11-27T15:17:14.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.056+0000 7fa4786b7700 1 -- 172.21.15.8:0/2905297735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4700fff90 msgr2=0x7fa47018f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:14.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.056+0000 7fa4786b7700 1 --2- 172.21.15.8:0/2905297735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4700fff90 0x7fa47018f7c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa46c006010 tx=0x7fa46c004ff0 comp rx=0 tx=0).stop 2023-11-27T15:17:14.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.056+0000 7fa4786b7700 1 -- 172.21.15.8:0/2905297735 shutdown_connections 2023-11-27T15:17:14.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.056+0000 7fa4786b7700 1 --2- 172.21.15.8:0/2905297735 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa45c0388c0 0x7fa45c03ad70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:14.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.056+0000 7fa4786b7700 1 --2- 172.21.15.8:0/2905297735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4700fff90 0x7fa47018f7c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:14.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.056+0000 7fa4786b7700 1 -- 172.21.15.8:0/2905297735 >> 172.21.15.8:0/2905297735 conn(0x7fa4700fb850 msgr2=0x7fa4701038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:14.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.056+0000 7fa4786b7700 1 -- 172.21.15.8:0/2905297735 shutdown_connections 2023-11-27T15:17:14.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:14.057+0000 7fa4786b7700 1 -- 172.21.15.8:0/2905297735 wait complete. 2023-11-27T15:17:14.062 INFO:teuthology.orchestra.run.smithi008.stderr:adjusted tunables profile to default 2023-11-27T15:17:14.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:14.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:14.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:14.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:14.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:14.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:14.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:14.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:14.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: Updating smithi060:/etc/ceph/ceph.conf 2023-11-27T15:17:14.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: Updating smithi060:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:14.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2905297735' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-27T15:17:14.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:14.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:14.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:14.576 INFO:tasks.cephadm:Adding mon.a on smithi008 2023-11-27T15:17:14.577 INFO:tasks.cephadm:Adding mon.b on smithi029 2023-11-27T15:17:14.577 INFO:tasks.cephadm:Adding mon.c on smithi060 2023-11-27T15:17:14.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch apply mon '3;smithi008:172.21.15.8=a;smithi029:172.21.15.29=b;smithi060:172.21.15.60=c' 2023-11-27T15:17:15.252 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:15.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:15 smithi008 ceph-mon[128788]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:17:15.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:15 smithi008 ceph-mon[128788]: Updating smithi060:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:17:15.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:15 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2905297735' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-27T15:17:15.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:15 smithi008 ceph-mon[128788]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:15.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:15 smithi008 ceph-mon[128788]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:16.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.455+0000 7fb1791e7700 1 -- 172.21.15.60:0/4257932848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1740fff90 msgr2=0x7fb174100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:16.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.455+0000 7fb1791e7700 1 --2- 172.21.15.60:0/4257932848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1740fff90 0x7fb174100360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb168009910 tx=0x7fb168009c20 comp rx=0 tx=0).stop 2023-11-27T15:17:16.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.455+0000 7fb1791e7700 1 -- 172.21.15.60:0/4257932848 shutdown_connections 2023-11-27T15:17:16.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.455+0000 7fb1791e7700 1 --2- 172.21.15.60:0/4257932848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1740fff90 0x7fb174100360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:16.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.455+0000 7fb1791e7700 1 -- 172.21.15.60:0/4257932848 >> 172.21.15.60:0/4257932848 conn(0x7fb1740fb870 msgr2=0x7fb1740fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:16.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.455+0000 7fb1791e7700 1 -- 172.21.15.60:0/4257932848 shutdown_connections 2023-11-27T15:17:16.460 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.455+0000 7fb1791e7700 1 -- 172.21.15.60:0/4257932848 wait complete. 2023-11-27T15:17:16.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.456+0000 7fb1791e7700 1 Processor -- start 2023-11-27T15:17:16.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.456+0000 7fb1791e7700 1 -- start start 2023-11-27T15:17:16.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.457+0000 7fb1791e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1740fff90 0x7fb1741951b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:16.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.457+0000 7fb1791e7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741956f0 con 0x7fb1740fff90 2023-11-27T15:17:16.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.457+0000 7fb172d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1740fff90 0x7fb1741951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:16.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.457+0000 7fb172d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1740fff90 0x7fb1741951b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.60:58702/0 (socket says 172.21.15.60:58702) 2023-11-27T15:17:16.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.457+0000 7fb172d9d700 1 -- 172.21.15.60:0/2643637155 learned_addr learned my addr 172.21.15.60:0/2643637155 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-27T15:17:16.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.457+0000 7fb172d9d700 1 -- 172.21.15.60:0/2643637155 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680095f0 con 0x7fb1740fff90 2023-11-27T15:17:16.462 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.458+0000 7fb172d9d700 1 --2- 172.21.15.60:0/2643637155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1740fff90 0x7fb1741951b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb168004f40 tx=0x7fb168005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:16.463 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.458+0000 7fb15bfff700 1 -- 172.21.15.60:0/2643637155 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb16801d070 con 0x7fb1740fff90 2023-11-27T15:17:16.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.458+0000 7fb1791e7700 1 -- 172.21.15.60:0/2643637155 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17410a170 con 0x7fb1740fff90 2023-11-27T15:17:16.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.458+0000 7fb15bfff700 1 -- 172.21.15.60:0/2643637155 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb168004370 con 0x7fb1740fff90 2023-11-27T15:17:16.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.458+0000 7fb15bfff700 1 -- 172.21.15.60:0/2643637155 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1680039b0 con 0x7fb1740fff90 2023-11-27T15:17:16.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.458+0000 7fb1791e7700 1 -- 172.21.15.60:0/2643637155 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb174109fe0 con 0x7fb1740fff90 2023-11-27T15:17:16.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.459+0000 7fb15bfff700 1 -- 172.21.15.60:0/2643637155 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fb168003c70 con 0x7fb1740fff90 2023-11-27T15:17:16.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.459+0000 7fb1791e7700 1 -- 172.21.15.60:0/2643637155 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb17410a300 con 0x7fb1740fff90 2023-11-27T15:17:16.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.459+0000 7fb15bfff700 1 --2- 172.21.15.60:0/2643637155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb15c038870 0x7fb15c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:16.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.459+0000 7fb15bfff700 1 -- 172.21.15.60:0/2643637155 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb16804c670 con 0x7fb1740fff90 2023-11-27T15:17:16.464 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.460+0000 7fb17259c700 1 --2- 172.21.15.60:0/2643637155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb15c038870 0x7fb15c03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:16.465 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.460+0000 7fb17259c700 1 --2- 172.21.15.60:0/2643637155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb15c038870 0x7fb15c03ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb164006fd0 tx=0x7fb164006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:16.468 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.464+0000 7fb15bfff700 1 -- 172.21.15.60:0/2643637155 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb168029360 con 0x7fb1740fff90 2023-11-27T15:17:16.677 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.672+0000 7fb1791e7700 1 -- 172.21.15.60:0/2643637155 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi008:172.21.15.8=a;smithi029:172.21.15.29=b;smithi060:172.21.15.60=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fb174108180 con 0x7fb15c038870 2023-11-27T15:17:16.683 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.679+0000 7fb15bfff700 1 -- 172.21.15.60:0/2643637155 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb174108180 con 0x7fb15c038870 2023-11-27T15:17:16.684 INFO:teuthology.orchestra.run.smithi060.stdout:Scheduled mon update... 2023-11-27T15:17:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.682+0000 7fb1791e7700 1 -- 172.21.15.60:0/2643637155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb15c038870 msgr2=0x7fb15c03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.682+0000 7fb1791e7700 1 --2- 172.21.15.60:0/2643637155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb15c038870 0x7fb15c03ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb164006fd0 tx=0x7fb164006e40 comp rx=0 tx=0).stop 2023-11-27T15:17:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.682+0000 7fb1791e7700 1 -- 172.21.15.60:0/2643637155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1740fff90 msgr2=0x7fb1741951b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.682+0000 7fb1791e7700 1 --2- 172.21.15.60:0/2643637155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1740fff90 0x7fb1741951b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb168004f40 tx=0x7fb168005020 comp rx=0 tx=0).stop 2023-11-27T15:17:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.682+0000 7fb1791e7700 1 -- 172.21.15.60:0/2643637155 shutdown_connections 2023-11-27T15:17:16.686 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.682+0000 7fb1791e7700 1 --2- 172.21.15.60:0/2643637155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb15c038870 0x7fb15c03ad20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:16.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.682+0000 7fb1791e7700 1 --2- 172.21.15.60:0/2643637155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb1740fff90 0x7fb1741951b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:16.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.682+0000 7fb1791e7700 1 -- 172.21.15.60:0/2643637155 >> 172.21.15.60:0/2643637155 conn(0x7fb1740fb870 msgr2=0x7fb174104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:16.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.682+0000 7fb1791e7700 1 -- 172.21.15.60:0/2643637155 shutdown_connections 2023-11-27T15:17:16.687 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:16.682+0000 7fb1791e7700 1 -- 172.21.15.60:0/2643637155 wait complete. 2023-11-27T15:17:17.119 DEBUG:teuthology.orchestra.run.smithi029:mon.b> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.b.service 2023-11-27T15:17:17.121 DEBUG:teuthology.orchestra.run.smithi060:mon.c> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.c.service 2023-11-27T15:17:17.123 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-27T15:17:17.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-11-27T15:17:17.154 INFO:journalctl@ceph.mon.c.smithi060.stdout:-- Logs begin at Mon 2023-11-27 14:51:59 UTC. -- 2023-11-27T15:17:17.158 INFO:journalctl@ceph.mon.b.smithi029.stdout:-- Logs begin at Mon 2023-11-27 14:52:42 UTC. -- 2023-11-27T15:17:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:17 smithi008 ceph-mon[128788]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:17 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:17 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:17 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:17 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:17 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:17 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:17 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:18.005 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:18 smithi008 ceph-mon[128788]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi008:172.21.15.8=a;smithi029:172.21.15.29=b;smithi060:172.21.15.60=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:18.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:18 smithi008 ceph-mon[128788]: Saving service mon spec with placement smithi008:172.21.15.8=a;smithi029:172.21.15.29=b;smithi060:172.21.15.60=c;count:3 2023-11-27T15:17:18.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:18 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:18.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:18 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:18.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:18 smithi008 ceph-mon[128788]: Deploying daemon mon.c on smithi060 2023-11-27T15:17:19.425 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.420+0000 7f7d67a7a700 1 -- 172.21.15.60:0/2375203054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d600fff90 msgr2=0x7f7d60100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.420+0000 7f7d67a7a700 1 --2- 172.21.15.60:0/2375203054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d600fff90 0x7f7d60100360 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009910 tx=0x7f7d5c009c20 comp rx=0 tx=0).stop 2023-11-27T15:17:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.420+0000 7f7d67a7a700 1 -- 172.21.15.60:0/2375203054 shutdown_connections 2023-11-27T15:17:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.420+0000 7f7d67a7a700 1 --2- 172.21.15.60:0/2375203054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d600fff90 0x7f7d60100360 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.420+0000 7f7d67a7a700 1 -- 172.21.15.60:0/2375203054 >> 172.21.15.60:0/2375203054 conn(0x7f7d600fb870 msgr2=0x7f7d600fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.420+0000 7f7d67a7a700 1 -- 172.21.15.60:0/2375203054 shutdown_connections 2023-11-27T15:17:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.420+0000 7f7d67a7a700 1 -- 172.21.15.60:0/2375203054 wait complete. 2023-11-27T15:17:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.421+0000 7f7d67a7a700 1 Processor -- start 2023-11-27T15:17:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.421+0000 7f7d67a7a700 1 -- start start 2023-11-27T15:17:19.426 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.421+0000 7f7d67a7a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d600fff90 0x7f7d601951b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:19.427 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.421+0000 7f7d67a7a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d601956f0 con 0x7f7d600fff90 2023-11-27T15:17:19.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.422+0000 7f7d65816700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d600fff90 0x7f7d601951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:19.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.422+0000 7f7d65816700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d600fff90 0x7f7d601951b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.60:58708/0 (socket says 172.21.15.60:58708) 2023-11-27T15:17:19.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.422+0000 7f7d65816700 1 -- 172.21.15.60:0/3499283675 learned_addr learned my addr 172.21.15.60:0/3499283675 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-27T15:17:19.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.422+0000 7f7d65816700 1 -- 172.21.15.60:0/3499283675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c0095f0 con 0x7f7d600fff90 2023-11-27T15:17:19.428 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.422+0000 7f7d65816700 1 --2- 172.21.15.60:0/3499283675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d600fff90 0x7f7d601951b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c006010 tx=0x7f7d5c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.422+0000 7f7d52ffd700 1 -- 172.21.15.60:0/3499283675 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d5c01d070 con 0x7f7d600fff90 2023-11-27T15:17:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.422+0000 7f7d67a7a700 1 -- 172.21.15.60:0/3499283675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6010a170 con 0x7f7d600fff90 2023-11-27T15:17:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.422+0000 7f7d52ffd700 1 -- 172.21.15.60:0/3499283675 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d5c00bc10 con 0x7f7d600fff90 2023-11-27T15:17:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.422+0000 7f7d52ffd700 1 -- 172.21.15.60:0/3499283675 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d5c003910 con 0x7f7d600fff90 2023-11-27T15:17:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.423+0000 7f7d67a7a700 1 -- 172.21.15.60:0/3499283675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d60109fe0 con 0x7f7d600fff90 2023-11-27T15:17:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.423+0000 7f7d52ffd700 1 -- 172.21.15.60:0/3499283675 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f7d5c00bd80 con 0x7f7d600fff90 2023-11-27T15:17:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.423+0000 7f7d67a7a700 1 -- 172.21.15.60:0/3499283675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6010a300 con 0x7f7d600fff90 2023-11-27T15:17:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.424+0000 7f7d52ffd700 1 --2- 172.21.15.60:0/3499283675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7d4c0388c0 0x7f7d4c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.424+0000 7f7d52ffd700 1 -- 172.21.15.60:0/3499283675 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7d5c04c600 con 0x7f7d600fff90 2023-11-27T15:17:19.429 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.424+0000 7f7d65015700 1 --2- 172.21.15.60:0/3499283675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7d4c0388c0 0x7f7d4c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:19.430 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.425+0000 7f7d65015700 1 --2- 172.21.15.60:0/3499283675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7d4c0388c0 0x7f7d4c03ad70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006fd0 tx=0x7f7d54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:19.433 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.429+0000 7f7d52ffd700 1 -- 172.21.15.60:0/3499283675 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7d5c029360 con 0x7f7d600fff90 2023-11-27T15:17:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:19 smithi008 ceph-mon[128788]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:19.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.761+0000 7f7d67a7a700 1 -- 172.21.15.60:0/3499283675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7d60062120 con 0x7f7d600fff90 2023-11-27T15:17:19.766 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.762+0000 7f7d52ffd700 1 -- 172.21.15.60:0/3499283675 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+732 (secure 0 0 0) 0x7f7d5c026070 con 0x7f7d600fff90 2023-11-27T15:17:19.767 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:17:19.767 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":1,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","modified":"2023-11-27T15:15:39.309333Z","created":"2023-11-27T15:15:39.309333Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-11-27T15:17:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.764+0000 7f7d67a7a700 1 -- 172.21.15.60:0/3499283675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7d4c0388c0 msgr2=0x7f7d4c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.764+0000 7f7d67a7a700 1 --2- 172.21.15.60:0/3499283675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7d4c0388c0 0x7f7d4c03ad70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7d54006fd0 tx=0x7f7d54006e40 comp rx=0 tx=0).stop 2023-11-27T15:17:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.764+0000 7f7d67a7a700 1 -- 172.21.15.60:0/3499283675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d600fff90 msgr2=0x7f7d601951b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.764+0000 7f7d67a7a700 1 --2- 172.21.15.60:0/3499283675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d600fff90 0x7f7d601951b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c006010 tx=0x7f7d5c004ff0 comp rx=0 tx=0).stop 2023-11-27T15:17:19.769 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.764+0000 7f7d67a7a700 1 -- 172.21.15.60:0/3499283675 shutdown_connections 2023-11-27T15:17:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.764+0000 7f7d67a7a700 1 --2- 172.21.15.60:0/3499283675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7d4c0388c0 0x7f7d4c03ad70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.764+0000 7f7d67a7a700 1 --2- 172.21.15.60:0/3499283675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d600fff90 0x7f7d601951b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.764+0000 7f7d67a7a700 1 -- 172.21.15.60:0/3499283675 >> 172.21.15.60:0/3499283675 conn(0x7f7d600fb870 msgr2=0x7f7d60104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.764+0000 7f7d67a7a700 1 -- 172.21.15.60:0/3499283675 shutdown_connections 2023-11-27T15:17:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:19.765+0000 7f7d67a7a700 1 -- 172.21.15.60:0/3499283675 wait complete. 2023-11-27T15:17:19.770 INFO:teuthology.orchestra.run.smithi060.stderr:dumped monmap epoch 1 2023-11-27T15:17:20.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:20 smithi008 ceph-mon[128788]: from='client.? 172.21.15.60:0/3499283675' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:17:20.681 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:20 smithi060 ceph-mon[131518]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-11-27T15:17:21.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:20 smithi060 systemd[1]: Started Ceph mon.c for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:17:21.176 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-27T15:17:21.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-11-27T15:17:21.835 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:17:25.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.920+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/1554872641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f080035c0 msgr2=0x7f4f08005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:25.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.920+0000 7f4f1eb0b700 1 --2- 172.21.15.60:0/1554872641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f080035c0 0x7f4f08005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4f180fec80 tx=0x7f4f0c00feb0 comp rx=0 tx=0).stop 2023-11-27T15:17:25.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.920+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/1554872641 shutdown_connections 2023-11-27T15:17:25.925 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.920+0000 7f4f1eb0b700 1 --2- 172.21.15.60:0/1554872641 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f080035c0 0x7f4f08005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:25.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.920+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/1554872641 >> 172.21.15.60:0/1554872641 conn(0x7f4f180fa870 msgr2=0x7f4f180fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:25.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.920+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/1554872641 shutdown_connections 2023-11-27T15:17:25.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.920+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/1554872641 wait complete. 2023-11-27T15:17:25.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.921+0000 7f4f1eb0b700 1 Processor -- start 2023-11-27T15:17:25.926 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.921+0000 7f4f1eb0b700 1 -- start start 2023-11-27T15:17:25.927 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.921+0000 7f4f1eb0b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f080035c0 0x7f4f18194230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:25.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.921+0000 7f4f1eb0b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f18194770 0x7f4f18198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:25.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.921+0000 7f4f1eb0b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f18194cf0 con 0x7f4f080035c0 2023-11-27T15:17:25.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.921+0000 7f4f1eb0b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f18194e60 con 0x7f4f18194770 2023-11-27T15:17:25.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f17fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f18194770 0x7f4f18198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:25.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f17fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f18194770 0x7f4f18198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34718/0 (socket says 172.21.15.60:34718) 2023-11-27T15:17:25.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f17fff700 1 -- 172.21.15.60:0/2259535155 learned_addr learned my addr 172.21.15.60:0/2259535155 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-27T15:17:25.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f1c8a7700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f080035c0 0x7f4f18194230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:25.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f17fff700 1 -- 172.21.15.60:0/2259535155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f18194770 msgr2=0x7f4f18198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-11-27T15:17:25.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f17fff700 1 -- 172.21.15.60:0/2259535155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f18194770 msgr2=0x7f4f18198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-27T15:17:25.928 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f17fff700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f18194770 0x7f4f18198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-27T15:17:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f17fff700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f18194770 0x7f4f18198be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:17:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f1c8a7700 1 -- 172.21.15.60:0/2259535155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f18194770 msgr2=0x7f4f18198be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:17:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f1c8a7700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f18194770 0x7f4f18198be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f1c8a7700 1 -- 172.21.15.60:0/2259535155 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c00f900 con 0x7f4f080035c0 2023-11-27T15:17:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.922+0000 7f4f1c8a7700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f080035c0 0x7f4f18194230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c010010 tx=0x7f4f0c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.923+0000 7f4f15ffb700 1 -- 172.21.15.60:0/2259535155 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4f0c013960 con 0x7f4f080035c0 2023-11-27T15:17:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.923+0000 7f4f15ffb700 1 -- 172.21.15.60:0/2259535155 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0c004030 con 0x7f4f080035c0 2023-11-27T15:17:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.923+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/2259535155 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f18106ba0 con 0x7f4f080035c0 2023-11-27T15:17:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.923+0000 7f4f15ffb700 1 -- 172.21.15.60:0/2259535155 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4f0c003a80 con 0x7f4f080035c0 2023-11-27T15:17:25.929 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.923+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/2259535155 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f18106a10 con 0x7f4f080035c0 2023-11-27T15:17:25.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.924+0000 7f4f15ffb700 1 -- 172.21.15.60:0/2259535155 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f4f0c003cf0 con 0x7f4f080035c0 2023-11-27T15:17:25.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.924+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/2259535155 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f18106d30 con 0x7f4f080035c0 2023-11-27T15:17:25.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.925+0000 7f4f15ffb700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4efc0386b0 0x7f4efc03ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:25.930 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.925+0000 7f4f15ffb700 1 -- 172.21.15.60:0/2259535155 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4f0c000e40 con 0x7f4f080035c0 2023-11-27T15:17:25.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.925+0000 7f4f17fff700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4efc0386b0 0x7f4efc03ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:25.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.926+0000 7f4f17fff700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4efc0386b0 0x7f4efc03ab60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009180 tx=0x7f4f04008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:25.935 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:25.931+0000 7f4f15ffb700 1 -- 172.21.15.60:0/2259535155 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4f0c080730 con 0x7f4f080035c0 2023-11-27T15:17:26.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:17:26.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: mon.a calling monitor election 2023-11-27T15:17:26.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:26.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: mon.c calling monitor election 2023-11-27T15:17:26.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:26.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: monmap e2: 2 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],c=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: fsmap 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: mgrmap e13: x(active, since 47s) 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: overall HEALTH_OK 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:26.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: mon.a calling monitor election 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: mon.c calling monitor election 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:26.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: monmap e2: 2 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],c=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: fsmap 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: mgrmap e13: x(active, since 47s) 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: overall HEALTH_OK 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:26.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:26.216 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.212+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/2259535155 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4f1819ee80 con 0x7f4f080035c0 2023-11-27T15:17:26.217 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.212+0000 7f4f15ffb700 1 -- 172.21.15.60:0/2259535155 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+998 (secure 0 0 0) 0x7f4f0c011540 con 0x7f4f080035c0 2023-11-27T15:17:26.217 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:17:26.217 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":2,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","modified":"2023-11-27T15:17:20.631222Z","created":"2023-11-27T15:15:39.309333Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-27T15:17:26.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4efc0386b0 msgr2=0x7f4efc03ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:26.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4efc0386b0 0x7f4efc03ab60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009180 tx=0x7f4f04008040 comp rx=0 tx=0).stop 2023-11-27T15:17:26.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f080035c0 msgr2=0x7f4f18194230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:26.219 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f080035c0 0x7f4f18194230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c010010 tx=0x7f4f0c0047a0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/2259535155 shutdown_connections 2023-11-27T15:17:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f080035c0 0x7f4f18194230 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4efc0386b0 0x7f4efc03ab60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 --2- 172.21.15.60:0/2259535155 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f18194770 0x7f4f18198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/2259535155 >> 172.21.15.60:0/2259535155 conn(0x7f4f180fa870 msgr2=0x7f4f181008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:26.220 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/2259535155 shutdown_connections 2023-11-27T15:17:26.221 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:26.215+0000 7f4f1eb0b700 1 -- 172.21.15.60:0/2259535155 wait complete. 2023-11-27T15:17:26.221 INFO:teuthology.orchestra.run.smithi060.stderr:dumped monmap epoch 2 2023-11-27T15:17:26.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:26 smithi060 ceph-mon[131518]: Deploying daemon mon.b on smithi029 2023-11-27T15:17:26.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:26 smithi060 ceph-mon[131518]: from='client.? 172.21.15.60:0/2259535155' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:17:26.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:26 smithi060 ceph-mon[131518]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:26.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:26 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:27.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:26 smithi008 ceph-mon[128788]: Deploying daemon mon.b on smithi029 2023-11-27T15:17:27.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:26 smithi008 ceph-mon[128788]: from='client.? 172.21.15.60:0/2259535155' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:17:27.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:26 smithi008 ceph-mon[128788]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:27.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:26 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:27.591 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-27T15:17:27.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-11-27T15:17:28.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:17:27 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:17:27.631+0000 7f952719b700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-27T15:17:28.255 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:17:28.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 systemd[1]: Starting Ceph mon.b for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:17:28.950 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 podman[131127]: 2023-11-27T15:17:28.950 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: set uid:gid to 167:167 (ceph:ceph) 2023-11-27T15:17:28.950 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-27T15:17:28.950 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pidfile_write: ignore empty --pid-file 2023-11-27T15:17:28.950 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: load: jerasure load: lrc 2023-11-27T15:17:28.950 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: RocksDB version: 7.9.2 2023-11-27T15:17:28.950 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Git sha 0 2023-11-27T15:17:28.950 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: DB SUMMARY 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: DB Session ID: BRIJQO89FG27MJJHJNRQ 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: CURRENT file: CURRENT 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: IDENTITY file: IDENTITY 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.error_if_exists: 0 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.create_if_missing: 0 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.paranoid_checks: 1 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-27T15:17:28.951 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.env: 0x55b13e401720 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.fs: PosixFileSystem 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.info_log: 0x55b13fd4d3c0 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_file_opening_threads: 16 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.statistics: (nil) 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.use_fsync: 0 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_log_file_size: 0 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.keep_log_file_num: 1000 2023-11-27T15:17:28.952 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.recycle_log_file_num: 0 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.allow_fallocate: 1 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.allow_mmap_reads: 0 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.allow_mmap_writes: 0 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.use_direct_reads: 0 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.create_missing_column_families: 0 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.db_log_dir: 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.wal_dir: 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-27T15:17:28.953 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.advise_random_on_open: 1 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.db_write_buffer_size: 0 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.write_buffer_manager: 0x55b13efbe3c0 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.rate_limiter: (nil) 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.wal_recovery_mode: 2 2023-11-27T15:17:28.954 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.enable_thread_tracking: 0 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.enable_pipelined_write: 0 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.unordered_write: 0 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.row_cache: None 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.wal_filter: None 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.allow_ingest_behind: 0 2023-11-27T15:17:28.955 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.two_write_queues: 0 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.manual_wal_flush: 0 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.wal_compression: 0 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.atomic_flush: 0 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.log_readahead_size: 0 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.best_efforts_recovery: 0 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.allow_data_in_errors: 0 2023-11-27T15:17:28.956 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.db_host_id: __hostname__ 2023-11-27T15:17:28.958 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-27T15:17:28.958 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_background_jobs: 2 2023-11-27T15:17:28.958 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_background_compactions: -1 2023-11-27T15:17:28.958 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_subcompactions: 1 2023-11-27T15:17:28.958 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-27T15:17:28.958 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-27T15:17:28.958 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_total_wal_size: 0 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 bash[131127]: 7f0ac6f859528f9205a98521bddaa2fafdcf568afef57707b1cf2fb5267426c8 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_open_files: -1 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bytes_per_sync: 0 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_readahead_size: 0 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_background_flushes: -1 2023-11-27T15:17:28.959 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Compression algorithms supported: 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: kZSTD supported: 0 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: kXpressCompression supported: 0 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: kLZ4HCCompression supported: 1 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: kZlibCompression supported: 1 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: kSnappyCompression supported: 1 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: kLZ4Compression supported: 1 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: kBZip2Compression supported: 0 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-11-27T15:17:28.960 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.merge_operator: 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_filter: None 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_filter_factory: None 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.sst_partitioner_factory: None 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b13fd4d4e0) 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout: cache_index_and_filter_blocks: 1 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout: pin_top_level_index_and_filter: 1 2023-11-27T15:17:28.961 INFO:journalctl@ceph.mon.b.smithi029.stdout: index_type: 0 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: data_block_index_type: 0 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: index_shortening: 1 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: checksum: 4 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: no_block_cache: 0 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: block_cache: 0x55b13f05f1f0 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: block_cache_name: BinnedLRUCache 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: block_cache_options: 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: capacity : 536870912 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: num_shard_bits : 4 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: strict_capacity_limit : 0 2023-11-27T15:17:28.962 INFO:journalctl@ceph.mon.b.smithi029.stdout: high_pri_pool_ratio: 0.000 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: block_cache_compressed: (nil) 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: persistent_cache: (nil) 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: block_size: 4096 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: block_size_deviation: 10 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: block_restart_interval: 16 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: index_block_restart_interval: 1 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: metadata_block_size: 4096 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: partition_filters: 0 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: use_delta_encoding: 1 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: filter_policy: bloomfilter 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: whole_key_filtering: 1 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: verify_compression: 0 2023-11-27T15:17:28.963 INFO:journalctl@ceph.mon.b.smithi029.stdout: read_amp_bytes_per_bit: 0 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout: format_version: 5 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout: enable_index_compression: 1 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout: block_align: 0 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout: max_auto_readahead_size: 262144 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout: prepopulate_block_cache: 0 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout: initial_auto_readahead_size: 8192 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout: num_file_reads_for_auto_readahead: 2 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.write_buffer_size: 33554432 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_write_buffer_number: 2 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compression: NoCompression 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bottommost_compression: Disabled 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.prefix_extractor: nullptr 2023-11-27T15:17:28.964 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.num_levels: 7 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-27T15:17:28.965 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compression_opts.level: 32767 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compression_opts.strategy: 0 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compression_opts.enabled: false 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-27T15:17:28.966 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.target_file_size_base: 67108864 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-27T15:17:28.967 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.arena_block_size: 1048576 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.disable_auto_compactions: 0 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-27T15:17:28.968 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.table_properties_collectors: 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.inplace_update_support: 0 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-27T15:17:28.969 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.bloom_locality: 0 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.max_successive_merges: 0 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.paranoid_file_checks: 0 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.force_consistency_checks: 1 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.report_bg_io_stats: 0 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.ttl: 2592000 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.enable_blob_files: false 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.min_blob_size: 0 2023-11-27T15:17:28.970 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.blob_file_size: 268435456 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.blob_file_starting_level: 0 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9547183c-4367-4143-88db-d3fcac27524e 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098248844797, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-27T15:17:28.971 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098248845216, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701098248, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9547183c-4367-4143-88db-d3fcac27524e", "db_session_id": "BRIJQO89FG27MJJHJNRQ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098248845286, "job": 1, "event": "recovery_finished"} 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b13f0fc000 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: DB pointer 0x55b13f0e8000 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout: ** DB Stats ** 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:17:28.972 INFO:journalctl@ceph.mon.b.smithi029.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: ** Compaction Stats [default] ** 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: ** Compaction Stats [default] ** 2023-11-27T15:17:28.973 INFO:journalctl@ceph.mon.b.smithi029.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-27T15:17:28.974 INFO:journalctl@ceph.mon.b.smithi029.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout: Block cache BinnedLRUCache@0x55b13f05f1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout: 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: using public_addr v2:172.21.15.29:0/0 -> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: starting mon.b rank -1 at public addrs [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] at bind addrs [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(???) e0 preinit fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).mds e1 new map 2023-11-27T15:17:28.975 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).mds e1 print_map 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout: e1 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout: legacy client fscid: -1 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout: 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout: No filesystems configured 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-27T15:17:28.976 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi029", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:28.977 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Updating smithi029:/etc/ceph/ceph.conf 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Updating smithi029:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Updating smithi029:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Updating smithi029:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi060", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Deploying cephadm binary to smithi060 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.978 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Added host smithi060 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.979 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Updating smithi060:/etc/ceph/ceph.conf 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Updating smithi060:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2905297735' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Updating smithi060:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:17:28.980 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2905297735' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi008:172.21.15.8=a;smithi029:172.21.15.29=b;smithi060:172.21.15.60=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:28.981 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Saving service mon spec with placement smithi008:172.21.15.8=a;smithi029:172.21.15.29=b;smithi060:172.21.15.60=c;count:3 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Deploying daemon mon.c on smithi060 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='client.? 172.21.15.60:0/3499283675' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.a calling monitor election 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.982 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.c calling monitor election 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: monmap e2: 2 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],c=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: fsmap 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mgrmap e13: x(active, since 47s) 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: overall HEALTH_OK 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.983 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: Deploying daemon mon.b on smithi029 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='client.? 172.21.15.60:0/2259535155' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta expand map: {default=false} 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta from 'false' to 'false' 2023-11-27T15:17:28.984 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta expanded map: {default=false} 2023-11-27T15:17:28.985 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta expand map: {default=info} 2023-11-27T15:17:28.985 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta from 'info' to 'info' 2023-11-27T15:17:28.985 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta expanded map: {default=info} 2023-11-27T15:17:28.985 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta expand map: {default=daemon} 2023-11-27T15:17:28.985 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-27T15:17:28.985 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta expanded map: {default=daemon} 2023-11-27T15:17:28.985 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta expand map: {default=debug} 2023-11-27T15:17:28.985 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta from 'debug' to 'debug' 2023-11-27T15:17:28.985 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 ceph-mon[131166]: expand_channel_meta expanded map: {default=debug} 2023-11-27T15:17:28.985 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:28 smithi029 systemd[1]: Started Ceph mon.b for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:17:29.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:28 smithi060 ceph-mon[131518]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:29.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:28 smithi008 ceph-mon[128788]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:34.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:17:34.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: mon.a calling monitor election 2023-11-27T15:17:34.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: mon.c calling monitor election 2023-11-27T15:17:34.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:34.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:34.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:34.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:17:34.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0],c=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} 2023-11-27T15:17:34.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: fsmap 2023-11-27T15:17:34.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:34.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: mgrmap e13: x(active, since 55s) 2023-11-27T15:17:34.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-27T15:17:34.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-27T15:17:34.327 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-27T15:17:34.327 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: mon.b (rank 2) addr [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] is down (out of quorum) 2023-11-27T15:17:34.327 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:34.327 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:34.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:17:34.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: mon.a calling monitor election 2023-11-27T15:17:34.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: mon.c calling monitor election 2023-11-27T15:17:34.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:34.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:34.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:34.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0],c=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: fsmap 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: mgrmap e13: x(active, since 55s) 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: mon.b (rank 2) addr [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] is down (out of quorum) 2023-11-27T15:17:34.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:34.366 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:35.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:34 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:35.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:34 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:35.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:34 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:35.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:34 smithi060 ceph-mon[131518]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:35.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:34 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.322+0000 7fd30bb69700 1 -- 172.21.15.60:0/521962473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2f0006230 msgr2=0x7fd2f0008690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.322+0000 7fd30bb69700 1 --2- 172.21.15.60:0/521962473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2f0006230 0x7fd2f0008690 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd3040ffe40 tx=0x7fd2f400fe90 comp rx=0 tx=0).stop 2023-11-27T15:17:35.326 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.322+0000 7fd30bb69700 1 -- 172.21.15.60:0/521962473 shutdown_connections 2023-11-27T15:17:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.322+0000 7fd30bb69700 1 --2- 172.21.15.60:0/521962473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2f0006230 0x7fd2f0008690 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.322+0000 7fd30bb69700 1 -- 172.21.15.60:0/521962473 >> 172.21.15.60:0/521962473 conn(0x7fd304074ab0 msgr2=0x7fd304074eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.322+0000 7fd30bb69700 1 -- 172.21.15.60:0/521962473 shutdown_connections 2023-11-27T15:17:35.327 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.322+0000 7fd30bb69700 1 -- 172.21.15.60:0/521962473 wait complete. 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.323+0000 7fd30bb69700 1 Processor -- start 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.323+0000 7fd30bb69700 1 -- start start 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.323+0000 7fd30bb69700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2f0006230 0x7fd3041940f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.323+0000 7fd30bb69700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd304194630 0x7fd304194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.323+0000 7fd30bb69700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3041a1760 0x7fd3041a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.323+0000 7fd30bb69700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3041a4080 con 0x7fd2f0006230 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.323+0000 7fd30bb69700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3041a41f0 con 0x7fd3041a1760 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.323+0000 7fd30bb69700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3041a43b0 con 0x7fd304194630 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.323+0000 7fd309104700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd304194630 0x7fd304194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309104700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd304194630 0x7fd304194aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34748/0 (socket says 172.21.15.60:34748) 2023-11-27T15:17:35.329 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309104700 1 -- 172.21.15.60:0/2209391552 learned_addr learned my addr 172.21.15.60:0/2209391552 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309104700 1 -- 172.21.15.60:0/2209391552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd304194630 msgr2=0x7fd304194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309104700 1 -- 172.21.15.60:0/2209391552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd304194630 msgr2=0x7fd304194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309104700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd304194630 0x7fd304194aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309104700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd304194630 0x7fd304194aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309905700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2f0006230 0x7fd3041940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd30a106700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3041a1760 0x7fd3041a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd30a106700 1 -- 172.21.15.60:0/2209391552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3041a1760 msgr2=0x7fd3041a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd30a106700 1 -- 172.21.15.60:0/2209391552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3041a1760 msgr2=0x7fd3041a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd30a106700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3041a1760 0x7fd3041a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd30a106700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3041a1760 0x7fd3041a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:17:35.330 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309905700 1 -- 172.21.15.60:0/2209391552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd304194630 msgr2=0x7fd304194aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:17:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309905700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd304194630 0x7fd304194aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309905700 1 -- 172.21.15.60:0/2209391552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3041a1760 msgr2=0x7fd3041a3b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:17:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309905700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3041a1760 0x7fd3041a3b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309905700 1 -- 172.21.15.60:0/2209391552 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f400f8e0 con 0x7fd2f0006230 2023-11-27T15:17:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.324+0000 7fd309905700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2f0006230 0x7fd3041940f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd3040ffe40 tx=0x7fd2f4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.325+0000 7fd2faffd700 1 -- 172.21.15.60:0/2209391552 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4011d60 con 0x7fd2f0006230 2023-11-27T15:17:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.325+0000 7fd30bb69700 1 -- 172.21.15.60:0/2209391552 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd304106b80 con 0x7fd2f0006230 2023-11-27T15:17:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.325+0000 7fd2faffd700 1 -- 172.21.15.60:0/2209391552 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2f4011ec0 con 0x7fd2f0006230 2023-11-27T15:17:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.325+0000 7fd2faffd700 1 -- 172.21.15.60:0/2209391552 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40038c0 con 0x7fd2f0006230 2023-11-27T15:17:35.331 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.325+0000 7fd30bb69700 1 -- 172.21.15.60:0/2209391552 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3041069f0 con 0x7fd2f0006230 2023-11-27T15:17:35.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.326+0000 7fd2faffd700 1 -- 172.21.15.60:0/2209391552 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fd2f4003a20 con 0x7fd2f0006230 2023-11-27T15:17:35.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.326+0000 7fd30bb69700 1 -- 172.21.15.60:0/2209391552 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd304106d10 con 0x7fd2f0006230 2023-11-27T15:17:35.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.326+0000 7fd2faffd700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd2e8038cf0 0x7fd2e803b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:35.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.326+0000 7fd2faffd700 1 -- 172.21.15.60:0/2209391552 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd2f404e8a0 con 0x7fd2f0006230 2023-11-27T15:17:35.332 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.326+0000 7fd309104700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd2e8038cf0 0x7fd2e803b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:35.333 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.327+0000 7fd309104700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd2e8038cf0 0x7fd2e803b1a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd300009180 tx=0x7fd300008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:35.335 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.331+0000 7fd2faffd700 1 -- 172.21.15.60:0/2209391552 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd2f40305c0 con 0x7fd2f0006230 2023-11-27T15:17:35.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:34 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:35.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:34 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:35.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:34 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:35.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:34 smithi008 ceph-mon[128788]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:35.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:34 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.625+0000 7fd30bb69700 1 -- 172.21.15.60:0/2209391552 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd304062120 con 0x7fd2f0006230 2023-11-27T15:17:35.630 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.626+0000 7fd2faffd700 1 -- 172.21.15.60:0/2209391552 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1262 (secure 0 0 0) 0x7fd2f40303e0 con 0x7fd2f0006230 2023-11-27T15:17:35.631 INFO:teuthology.orchestra.run.smithi060.stdout: 2023-11-27T15:17:35.631 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":3,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","modified":"2023-11-27T15:17:28.867648Z","created":"2023-11-27T15:15:39.309333Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:3300","nonce":0},{"type":"v1","addr":"172.21.15.29:6789","nonce":0}]},"addr":"172.21.15.29:6789/0","public_addr":"172.21.15.29:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 -- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd2e8038cf0 msgr2=0x7fd2e803b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd2e8038cf0 0x7fd2e803b1a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd300009180 tx=0x7fd300008040 comp rx=0 tx=0).stop 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 -- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2f0006230 msgr2=0x7fd3041940f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2f0006230 0x7fd3041940f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd3040ffe40 tx=0x7fd2f4005e50 comp rx=0 tx=0).stop 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 -- 172.21.15.60:0/2209391552 shutdown_connections 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3041a1760 0x7fd3041a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2f0006230 0x7fd3041940f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd2e8038cf0 0x7fd2e803b1a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 --2- 172.21.15.60:0/2209391552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd304194630 0x7fd304194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 -- 172.21.15.60:0/2209391552 >> 172.21.15.60:0/2209391552 conn(0x7fd304074ab0 msgr2=0x7fd3040fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:35.633 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 -- 172.21.15.60:0/2209391552 shutdown_connections 2023-11-27T15:17:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:35.628+0000 7fd30bb69700 1 -- 172.21.15.60:0/2209391552 wait complete. 2023-11-27T15:17:35.634 INFO:teuthology.orchestra.run.smithi060.stderr:dumped monmap epoch 3 2023-11-27T15:17:36.020 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-11-27T15:17:36.020 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph config generate-minimal-conf 2023-11-27T15:17:36.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:17:36.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: mon.a calling monitor election 2023-11-27T15:17:36.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: mon.c calling monitor election 2023-11-27T15:17:36.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:36.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:17:36.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:36.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:36.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:36.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0],c=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: fsmap 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: mgrmap e13: x(active, since 55s) 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: mon.b (rank 2) addr [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] is down (out of quorum) 2023-11-27T15:17:36.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:36.150 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:36.150 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:36.150 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:36.150 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:36.150 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:36.150 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:36.924 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: mon.b calling monitor election 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: Updating smithi008:/etc/ceph/ceph.conf 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: Updating smithi029:/etc/ceph/ceph.conf 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: Updating smithi060:/etc/ceph/ceph.conf 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: mon.b calling monitor election 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: mon.c calling monitor election 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: mon.a calling monitor election 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0],c=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} 2023-11-27T15:17:37.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: fsmap 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: mgrmap e13: x(active, since 57s) 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: Cluster is now healthy 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: overall HEALTH_OK 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: Updating smithi060:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: Updating smithi029:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: Updating smithi008:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:37.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:36 smithi060 ceph-mon[131518]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:37.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: mon.b calling monitor election 2023-11-27T15:17:37.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:37.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: Updating smithi008:/etc/ceph/ceph.conf 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: Updating smithi029:/etc/ceph/ceph.conf 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: Updating smithi060:/etc/ceph/ceph.conf 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: mon.b calling monitor election 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: mon.c calling monitor election 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: mon.a calling monitor election 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0],c=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: fsmap 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: mgrmap e13: x(active, since 57s) 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-27T15:17:37.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: Cluster is now healthy 2023-11-27T15:17:37.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: overall HEALTH_OK 2023-11-27T15:17:37.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: Updating smithi060:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:37.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: Updating smithi029:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:37.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: Updating smithi008:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:37.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:36 smithi008 ceph-mon[128788]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: mon.b calling monitor election 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: Updating smithi008:/etc/ceph/ceph.conf 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: Updating smithi029:/etc/ceph/ceph.conf 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: Updating smithi060:/etc/ceph/ceph.conf 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: mon.b calling monitor election 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: mon.c calling monitor election 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: mon.a calling monitor election 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: monmap e3: 3 mons at {a=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],b=[v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0],c=[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]} removed_ranks: {} 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: fsmap 2023-11-27T15:17:37.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: mgrmap e13: x(active, since 57s) 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: Cluster is now healthy 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: overall HEALTH_OK 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: Updating smithi060:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: Updating smithi029:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: Updating smithi008:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:17:37.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:36 smithi029 ceph-mon[131166]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:37.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:37.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:37 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:37.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:37.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:37 smithi008 ceph-mon[128788]: Reconfiguring mon.a (unknown last config time)... 2023-11-27T15:17:37.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:37.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:17:37.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:37.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:37 smithi008 ceph-mon[128788]: Reconfiguring daemon mon.a on smithi008 2023-11-27T15:17:38.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:38.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:38.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:38.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:37 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:17:38.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:38.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:37 smithi060 ceph-mon[131518]: Reconfiguring mon.a (unknown last config time)... 2023-11-27T15:17:38.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:38.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:17:38.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:38.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:37 smithi060 ceph-mon[131518]: Reconfiguring daemon mon.a on smithi008 2023-11-27T15:17:38.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:38.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:38.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:38.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:37 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:38.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:17:38.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:37 smithi029 ceph-mon[131166]: Reconfiguring mon.a (unknown last config time)... 2023-11-27T15:17:38.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:38.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:17:38.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:38.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:37 smithi029 ceph-mon[131166]: Reconfiguring daemon mon.a on smithi008 2023-11-27T15:17:38.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.625+0000 7f3ce5431700 1 -- 172.21.15.8:0/812116964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ce00fff90 msgr2=0x7f3ce0100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:38.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.625+0000 7f3ce5431700 1 --2- 172.21.15.8:0/812116964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ce00fff90 0x7f3ce0100360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009910 tx=0x7f3cd0009c20 comp rx=0 tx=0).stop 2023-11-27T15:17:38.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.625+0000 7f3ce5431700 1 -- 172.21.15.8:0/812116964 shutdown_connections 2023-11-27T15:17:38.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.625+0000 7f3ce5431700 1 --2- 172.21.15.8:0/812116964 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ce00fff90 0x7f3ce0100360 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:38.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.625+0000 7f3ce5431700 1 -- 172.21.15.8:0/812116964 >> 172.21.15.8:0/812116964 conn(0x7f3ce00fb850 msgr2=0x7f3ce00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:38.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.625+0000 7f3ce5431700 1 -- 172.21.15.8:0/812116964 shutdown_connections 2023-11-27T15:17:38.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.625+0000 7f3ce5431700 1 -- 172.21.15.8:0/812116964 wait complete. 2023-11-27T15:17:38.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.626+0000 7f3ce5431700 1 Processor -- start 2023-11-27T15:17:38.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.626+0000 7f3ce5431700 1 -- start start 2023-11-27T15:17:38.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3ce5431700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ce00fff90 0x7f3ce0195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3ce5431700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ce0195880 0x7f3ce0199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3ce5431700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ce019a230 0x7f3ce019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3ce5431700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce0195f50 con 0x7f3ce019a230 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3ce5431700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce01a1fa0 con 0x7f3ce0195880 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3ce5431700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce01a20e0 con 0x7f3ce00fff90 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ce019a230 0x7f3ce019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3cdf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ce019a230 0x7f3ce019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60900/0 (socket says 172.21.15.8:60900) 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3cdf7fe700 1 -- 172.21.15.8:0/3936489675 learned_addr learned my addr 172.21.15.8:0/3936489675 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3cde7fc700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ce0195880 0x7f3ce0199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.627+0000 7f3cdeffd700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ce00fff90 0x7f3ce0195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:38.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3cdf7fe700 1 -- 172.21.15.8:0/3936489675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ce00fff90 msgr2=0x7f3ce0195340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:38.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3cdf7fe700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ce00fff90 0x7f3ce0195340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:38.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3cdf7fe700 1 -- 172.21.15.8:0/3936489675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ce0195880 msgr2=0x7f3ce0199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:38.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3cdf7fe700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ce0195880 0x7f3ce0199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:38.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3cdf7fe700 1 -- 172.21.15.8:0/3936489675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd00095f0 con 0x7f3ce019a230 2023-11-27T15:17:38.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3cdf7fe700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ce019a230 0x7f3ce019a6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800ba80 tx=0x7f3cd800bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:38.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3cc7fff700 1 -- 172.21.15.8:0/3936489675 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800c930 con 0x7f3ce019a230 2023-11-27T15:17:38.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3cc7fff700 1 -- 172.21.15.8:0/3936489675 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd8014440 con 0x7f3ce019a230 2023-11-27T15:17:38.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3cc7fff700 1 -- 172.21.15.8:0/3936489675 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8007570 con 0x7f3ce019a230 2023-11-27T15:17:38.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3ce5431700 1 -- 172.21.15.8:0/3936489675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce010a170 con 0x7f3ce019a230 2023-11-27T15:17:38.634 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.628+0000 7f3ce5431700 1 -- 172.21.15.8:0/3936489675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce0109fe0 con 0x7f3ce019a230 2023-11-27T15:17:38.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.629+0000 7f3cc7fff700 1 -- 172.21.15.8:0/3936489675 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f3cd80145b0 con 0x7f3ce019a230 2023-11-27T15:17:38.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.629+0000 7f3ce5431700 1 -- 172.21.15.8:0/3936489675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce010a300 con 0x7f3ce019a230 2023-11-27T15:17:38.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.630+0000 7f3cc7fff700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3cc8038d40 0x7f3cc803b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:38.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.630+0000 7f3cc7fff700 1 -- 172.21.15.8:0/3936489675 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3cd804aa40 con 0x7f3ce019a230 2023-11-27T15:17:38.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.630+0000 7f3cdeffd700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3cc8038d40 0x7f3cc803b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:38.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.631+0000 7f3cdeffd700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3cc8038d40 0x7f3cc803b1f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009fd0 tx=0x7f3cd00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:38.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.636+0000 7f3cc7fff700 1 -- 172.21.15.8:0/3936489675 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3cd800ca90 con 0x7f3ce019a230 2023-11-27T15:17:38.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.848+0000 7f3ce5431700 1 -- 172.21.15.8:0/3936489675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f3ce0062120 con 0x7f3ce019a230 2023-11-27T15:17:38.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.848+0000 7f3cc7fff700 1 -- 172.21.15.8:0/3936489675 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+269 (secure 0 0 0) 0x7f3cd8012340 con 0x7f3ce019a230 2023-11-27T15:17:38.852 INFO:teuthology.orchestra.run.smithi008.stdout:# minimal ceph.conf for 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:17:38.853 INFO:teuthology.orchestra.run.smithi008.stdout:[global] 2023-11-27T15:17:38.853 INFO:teuthology.orchestra.run.smithi008.stdout: fsid = 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:17:38.853 INFO:teuthology.orchestra.run.smithi008.stdout: mon_host = [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] 2023-11-27T15:17:38.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 -- 172.21.15.8:0/3936489675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3cc8038d40 msgr2=0x7f3cc803b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:38.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3cc8038d40 0x7f3cc803b1f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009fd0 tx=0x7f3cd00058e0 comp rx=0 tx=0).stop 2023-11-27T15:17:38.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 -- 172.21.15.8:0/3936489675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ce019a230 msgr2=0x7f3ce019a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:38.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ce019a230 0x7f3ce019a6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800ba80 tx=0x7f3cd800bd90 comp rx=0 tx=0).stop 2023-11-27T15:17:38.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 -- 172.21.15.8:0/3936489675 shutdown_connections 2023-11-27T15:17:38.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3cc8038d40 0x7f3cc803b1f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:38.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ce00fff90 0x7f3ce0195340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:38.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ce019a230 0x7f3ce019a6d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:38.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 --2- 172.21.15.8:0/3936489675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ce0195880 0x7f3ce0199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:38.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 -- 172.21.15.8:0/3936489675 >> 172.21.15.8:0/3936489675 conn(0x7f3ce00fb850 msgr2=0x7f3ce0105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:38.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 -- 172.21.15.8:0/3936489675 shutdown_connections 2023-11-27T15:17:38.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:38.851+0000 7f3ce5431700 1 -- 172.21.15.8:0/3936489675 wait complete. 2023-11-27T15:17:39.364 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-11-27T15:17:39.364 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:17:39.364 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.conf 2023-11-27T15:17:39.437 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:17:39.437 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:17:39.548 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T15:17:39.548 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.conf 2023-11-27T15:17:39.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:39 smithi060 ceph-mon[131518]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:39.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:39.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:39 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3936489675' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:39.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:39.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:39.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:39.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:17:39.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:39.576 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:39 smithi029 ceph-mon[131166]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:39.576 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:39.576 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:39 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3936489675' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:39.576 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:39.576 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:39.576 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:39.576 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:17:39.576 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:39.580 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T15:17:39.580 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:17:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:39 smithi008 ceph-mon[128788]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:39 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3936489675' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:39.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:39.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:17:39.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:39.650 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T15:17:39.650 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.conf 2023-11-27T15:17:39.688 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T15:17:39.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:17:39.759 INFO:tasks.cephadm:Adding mgr.x on smithi008 2023-11-27T15:17:39.759 INFO:tasks.cephadm:Adding mgr.y on smithi029 2023-11-27T15:17:39.759 INFO:tasks.cephadm:Adding mgr.z on smithi060 2023-11-27T15:17:39.760 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch apply mgr '3;smithi008=x;smithi029=y;smithi060=z' 2023-11-27T15:17:40.449 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:17:40.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:40 smithi060 ceph-mon[131518]: Reconfiguring mon.b (monmap changed)... 2023-11-27T15:17:40.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:40 smithi060 ceph-mon[131518]: Reconfiguring daemon mon.b on smithi029 2023-11-27T15:17:40.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:40 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:40.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:40 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:40.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:40 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:40.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:40 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:17:40.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:40 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:40.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:40 smithi008 ceph-mon[128788]: Reconfiguring mon.b (monmap changed)... 2023-11-27T15:17:40.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:40 smithi008 ceph-mon[128788]: Reconfiguring daemon mon.b on smithi029 2023-11-27T15:17:40.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:40 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:40.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:40 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:40.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:40 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:40.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:40 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:17:40.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:40 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:40.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:40 smithi029 ceph-mon[131166]: Reconfiguring mon.b (monmap changed)... 2023-11-27T15:17:40.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:40 smithi029 ceph-mon[131166]: Reconfiguring daemon mon.b on smithi029 2023-11-27T15:17:40.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:40 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:40.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:40 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:40.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:40 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:17:40.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:40 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:17:40.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:40 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:41.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:41 smithi060 ceph-mon[131518]: Reconfiguring mon.c (monmap changed)... 2023-11-27T15:17:41.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:41 smithi060 ceph-mon[131518]: Reconfiguring daemon mon.c on smithi060 2023-11-27T15:17:41.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:41 smithi060 ceph-mon[131518]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:41.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:41 smithi008 ceph-mon[128788]: Reconfiguring mon.c (monmap changed)... 2023-11-27T15:17:41.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:41 smithi008 ceph-mon[128788]: Reconfiguring daemon mon.c on smithi060 2023-11-27T15:17:41.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:41 smithi008 ceph-mon[128788]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:41.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:41 smithi029 ceph-mon[131166]: Reconfiguring mon.c (monmap changed)... 2023-11-27T15:17:41.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:41 smithi029 ceph-mon[131166]: Reconfiguring daemon mon.c on smithi060 2023-11-27T15:17:41.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:41 smithi029 ceph-mon[131166]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:41.943 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.939+0000 7ff1193c2700 1 -- 172.21.15.60:0/2163101986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1140ff1e0 msgr2=0x7ff1140ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.939+0000 7ff1193c2700 1 --2- 172.21.15.60:0/2163101986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1140ff1e0 0x7ff1140ff5f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff104009990 tx=0x7ff104009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.939+0000 7ff1193c2700 1 -- 172.21.15.60:0/2163101986 shutdown_connections 2023-11-27T15:17:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.939+0000 7ff1193c2700 1 --2- 172.21.15.60:0/2163101986 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1140ff1e0 0x7ff1140ff5f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.939+0000 7ff1193c2700 1 --2- 172.21.15.60:0/2163101986 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff114100d70 0x7ff114107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.939+0000 7ff1193c2700 1 --2- 172.21.15.60:0/2163101986 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1141003e0 0x7ff114100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.939+0000 7ff1193c2700 1 -- 172.21.15.60:0/2163101986 >> 172.21.15.60:0/2163101986 conn(0x7ff1140fa7f0 msgr2=0x7ff1140fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.939+0000 7ff1193c2700 1 -- 172.21.15.60:0/2163101986 shutdown_connections 2023-11-27T15:17:41.944 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.939+0000 7ff1193c2700 1 -- 172.21.15.60:0/2163101986 wait complete. 2023-11-27T15:17:41.946 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1193c2700 1 Processor -- start 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1193c2700 1 -- start start 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1193c2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1140ff1e0 0x7ff114069110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1193c2700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1141003e0 0x7ff11419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1193c2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff114100d70 0x7ff11419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1193c2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419dbd0 con 0x7ff114100d70 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1193c2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419dd40 con 0x7ff1140ff1e0 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1193c2700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11419df00 con 0x7ff1141003e0 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1127fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1141003e0 0x7ff11419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1127fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1141003e0 0x7ff11419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34762/0 (socket says 172.21.15.60:34762) 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1127fc700 1 -- 172.21.15.60:0/3828982668 learned_addr learned my addr 172.21.15.60:0/3828982668 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.940+0000 7ff1137fe700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff114100d70 0x7ff11419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff112ffd700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1140ff1e0 0x7ff114069110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.947 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff1127fc700 1 -- 172.21.15.60:0/3828982668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1140ff1e0 msgr2=0x7ff114069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff1127fc700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1140ff1e0 0x7ff114069110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff1127fc700 1 -- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff114100d70 msgr2=0x7ff11419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff1127fc700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff114100d70 0x7ff11419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff1127fc700 1 -- 172.21.15.60:0/3828982668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1040095f0 con 0x7ff1141003e0 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff1137fe700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff114100d70 0x7ff11419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff112ffd700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1140ff1e0 0x7ff114069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff1127fc700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1141003e0 0x7ff11419adf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff10800c920 tx=0x7ff10800cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff0fbfff700 1 -- 172.21.15.60:0/3828982668 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108007b00 con 0x7ff1141003e0 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff0fbfff700 1 -- 172.21.15.60:0/3828982668 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff108004cf0 con 0x7ff1141003e0 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff1193c2700 1 -- 172.21.15.60:0/3828982668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11410ab30 con 0x7ff1141003e0 2023-11-27T15:17:41.948 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff0fbfff700 1 -- 172.21.15.60:0/3828982668 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108005610 con 0x7ff1141003e0 2023-11-27T15:17:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.941+0000 7ff1193c2700 1 -- 172.21.15.60:0/3828982668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11410a9a0 con 0x7ff1141003e0 2023-11-27T15:17:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.942+0000 7ff0fbfff700 1 -- 172.21.15.60:0/3828982668 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7ff10800f450 con 0x7ff1141003e0 2023-11-27T15:17:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.942+0000 7ff1193c2700 1 -- 172.21.15.60:0/3828982668 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11419e690 con 0x7ff1141003e0 2023-11-27T15:17:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.942+0000 7ff0fbfff700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff0fc0389e0 0x7ff0fc03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.942+0000 7ff0fbfff700 1 -- 172.21.15.60:0/3828982668 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff10804b280 con 0x7ff1141003e0 2023-11-27T15:17:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.943+0000 7ff112ffd700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff0fc0389e0 0x7ff0fc03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:41.949 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.943+0000 7ff112ffd700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff0fc0389e0 0x7ff0fc03ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1040095c0 tx=0x7ff104009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:41.951 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:41.947+0000 7ff0fbfff700 1 -- 172.21.15.60:0/3828982668 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff10800d700 con 0x7ff1141003e0 2023-11-27T15:17:42.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.156+0000 7ff1193c2700 1 -- 172.21.15.60:0/3828982668 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi008=x;smithi029=y;smithi060=z", "target": ["mon-mgr", ""]}) v1 -- 0x7ff114060f30 con 0x7ff0fc0389e0 2023-11-27T15:17:42.166 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.162+0000 7ff0fbfff700 1 -- 172.21.15.60:0/3828982668 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff114060f30 con 0x7ff0fc0389e0 2023-11-27T15:17:42.166 INFO:teuthology.orchestra.run.smithi060.stdout:Scheduled mgr update... 2023-11-27T15:17:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 -- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff0fc0389e0 msgr2=0x7ff0fc03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.168 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff0fc0389e0 0x7ff0fc03ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1040095c0 tx=0x7ff104009510 comp rx=0 tx=0).stop 2023-11-27T15:17:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 -- 172.21.15.60:0/3828982668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1141003e0 msgr2=0x7ff11419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1141003e0 0x7ff11419adf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff10800c920 tx=0x7ff10800cce0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 -- 172.21.15.60:0/3828982668 shutdown_connections 2023-11-27T15:17:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff114100d70 0x7ff11419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff1140ff1e0 0x7ff114069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff0fc0389e0 0x7ff0fc03ae90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 --2- 172.21.15.60:0/3828982668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff1141003e0 0x7ff11419adf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 -- 172.21.15.60:0/3828982668 >> 172.21.15.60:0/3828982668 conn(0x7ff1140fa7f0 msgr2=0x7ff1140fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 -- 172.21.15.60:0/3828982668 shutdown_connections 2023-11-27T15:17:42.169 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:17:42.164+0000 7ff1193c2700 1 -- 172.21.15.60:0/3828982668 wait complete. 2023-11-27T15:17:42.579 DEBUG:teuthology.orchestra.run.smithi029:mgr.y> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.y.service 2023-11-27T15:17:42.582 DEBUG:teuthology.orchestra.run.smithi060:mgr.z> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.z.service 2023-11-27T15:17:42.584 INFO:tasks.cephadm:Deploying OSDs... 2023-11-27T15:17:42.585 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:17:42.585 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/scratch_devs of=/dev/stdout 2023-11-27T15:17:42.614 INFO:journalctl@ceph.mgr.z.smithi060.stdout:-- Logs begin at Mon 2023-11-27 14:51:59 UTC. -- 2023-11-27T15:17:42.617 INFO:journalctl@ceph.mgr.y.smithi029.stdout:-- Logs begin at Mon 2023-11-27 14:52:42 UTC. -- 2023-11-27T15:17:42.629 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-27T15:17:42.629 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_1 2023-11-27T15:17:42.718 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-27T15:17:42.719 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:42.719 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-11-27T15:17:42.719 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:42.719 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:42.719 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-11-27 15:16:48.010062429 +0000 2023-11-27T15:17:42.719 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-11-27 15:10:09.954583482 +0000 2023-11-27T15:17:42.719 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-11-27 15:10:09.954583482 +0000 2023-11-27T15:17:42.719 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2023-11-27 15:10:09.931583718 +0000 2023-11-27T15:17:42.719 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-27T15:17:42.786 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-11-27T15:17:42.786 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-11-27T15:17:42.786 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000176895 s, 2.9 MB/s 2023-11-27T15:17:42.788 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-27T15:17:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi008=x;smithi029=y;smithi060=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: Saving service mgr spec with placement smithi008=x;smithi029=y;smithi060=z;count:3 2023-11-27T15:17:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:42.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:17:42.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:17:42.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:17:42.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:42.834 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_2 2023-11-27T15:17:42.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:42.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:42.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:42.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi008=x;smithi029=y;smithi060=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:42.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: Saving service mgr spec with placement smithi008=x;smithi029=y;smithi060=z;count:3 2023-11-27T15:17:42.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:42.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:42.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:42.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:42.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:42.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:17:42.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:17:42.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:17:42.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:42.918 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-27T15:17:42.918 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:42.918 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-11-27T15:17:42.918 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:42.918 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:42.918 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-11-27 15:16:48.010062429 +0000 2023-11-27T15:17:42.918 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-11-27 15:10:10.780574990 +0000 2023-11-27T15:17:42.918 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-11-27 15:10:10.780574990 +0000 2023-11-27T15:17:42.918 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2023-11-27 15:10:10.758575216 +0000 2023-11-27T15:17:42.919 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-27T15:17:43.017 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:43.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:43.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:43.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi008=x;smithi029=y;smithi060=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:43.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: Saving service mgr spec with placement smithi008=x;smithi029=y;smithi060=z;count:3 2023-11-27T15:17:43.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:43.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:43.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:43.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:43.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:43.018 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:17:43.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:17:43.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:17:43.019 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:43.024 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-11-27T15:17:43.024 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-11-27T15:17:43.024 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000169217 s, 3.0 MB/s 2023-11-27T15:17:43.027 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-27T15:17:43.072 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_3 2023-11-27T15:17:43.156 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-27T15:17:43.156 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:43.156 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-11-27T15:17:43.156 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:43.156 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:43.156 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-11-27 15:16:48.010062429 +0000 2023-11-27T15:17:43.156 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-11-27 15:10:11.604566519 +0000 2023-11-27T15:17:43.156 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-11-27 15:10:11.604566519 +0000 2023-11-27T15:17:43.156 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2023-11-27 15:10:11.581566755 +0000 2023-11-27T15:17:43.156 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-27T15:17:43.261 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-11-27T15:17:43.261 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-11-27T15:17:43.261 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000192495 s, 2.7 MB/s 2023-11-27T15:17:43.264 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-27T15:17:43.309 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_4 2023-11-27T15:17:43.393 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-27T15:17:43.393 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:43.393 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-11-27T15:17:43.394 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:43.394 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:43.394 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-11-27 15:16:48.010062429 +0000 2023-11-27T15:17:43.394 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-11-27 15:10:12.416558171 +0000 2023-11-27T15:17:43.394 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-11-27 15:10:12.416558171 +0000 2023-11-27T15:17:43.394 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2023-11-27 15:10:12.394558397 +0000 2023-11-27T15:17:43.394 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-27T15:17:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2023-11-27T15:17:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2023-11-27T15:17:43.499 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000189684 s, 2.7 MB/s 2023-11-27T15:17:43.502 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-27T15:17:43.548 DEBUG:teuthology.orchestra.run.smithi029:> set -ex 2023-11-27T15:17:43.548 DEBUG:teuthology.orchestra.run.smithi029:> dd if=/scratch_devs of=/dev/stdout 2023-11-27T15:17:43.569 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-27T15:17:43.570 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_1 2023-11-27T15:17:43.630 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-27T15:17:43.630 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:43.630 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 391299 Links: 1 2023-11-27T15:17:43.630 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:43.631 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:43.631 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2023-11-27 15:17:28.308922886 +0000 2023-11-27T15:17:43.631 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2023-11-27 15:07:09.061409010 +0000 2023-11-27T15:17:43.631 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2023-11-27 15:07:09.061409010 +0000 2023-11-27T15:17:43.631 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2023-11-27T15:17:43.631 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-27T15:17:43.701 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2023-11-27T15:17:43.701 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2023-11-27T15:17:43.701 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000126778 s, 4.0 MB/s 2023-11-27T15:17:43.702 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-27T15:17:43.764 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_2 2023-11-27T15:17:43.825 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-27T15:17:43.826 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:43.826 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 387278 Links: 1 2023-11-27T15:17:43.826 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:43.826 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:43.826 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2023-11-27 15:17:28.308922886 +0000 2023-11-27T15:17:43.826 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2023-11-27 15:07:09.580400247 +0000 2023-11-27T15:17:43.826 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2023-11-27 15:07:09.580400247 +0000 2023-11-27T15:17:43.826 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2023-11-27T15:17:43.826 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-27T15:17:43.898 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2023-11-27T15:17:43.898 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2023-11-27T15:17:43.898 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000134849 s, 3.8 MB/s 2023-11-27T15:17:43.900 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-27T15:17:43.962 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_3 2023-11-27T15:17:44.023 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-27T15:17:44.023 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:44.023 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 387321 Links: 1 2023-11-27T15:17:44.023 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:44.023 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:44.024 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2023-11-27 15:17:28.308922886 +0000 2023-11-27T15:17:44.024 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2023-11-27 15:07:10.073391923 +0000 2023-11-27T15:17:44.024 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2023-11-27 15:07:10.073391923 +0000 2023-11-27T15:17:44.024 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2023-11-27T15:17:44.024 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-27T15:17:44.094 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2023-11-27T15:17:44.095 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2023-11-27T15:17:44.095 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000133752 s, 3.8 MB/s 2023-11-27T15:17:44.096 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-27T15:17:44.157 DEBUG:teuthology.orchestra.run.smithi029:> stat /dev/vg_nvme/lv_4 2023-11-27T15:17:44.219 INFO:teuthology.orchestra.run.smithi029.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-27T15:17:44.219 INFO:teuthology.orchestra.run.smithi029.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:44.219 INFO:teuthology.orchestra.run.smithi029.stdout:Device: 6h/6d Inode: 391504 Links: 1 2023-11-27T15:17:44.220 INFO:teuthology.orchestra.run.smithi029.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:44.220 INFO:teuthology.orchestra.run.smithi029.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:44.220 INFO:teuthology.orchestra.run.smithi029.stdout:Access: 2023-11-27 15:17:28.308922886 +0000 2023-11-27T15:17:44.220 INFO:teuthology.orchestra.run.smithi029.stdout:Modify: 2023-11-27 15:07:10.577383414 +0000 2023-11-27T15:17:44.220 INFO:teuthology.orchestra.run.smithi029.stdout:Change: 2023-11-27 15:07:10.577383414 +0000 2023-11-27T15:17:44.220 INFO:teuthology.orchestra.run.smithi029.stdout: Birth: - 2023-11-27T15:17:44.221 DEBUG:teuthology.orchestra.run.smithi029:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-27T15:17:44.290 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records in 2023-11-27T15:17:44.290 INFO:teuthology.orchestra.run.smithi029.stderr:1+0 records out 2023-11-27T15:17:44.291 INFO:teuthology.orchestra.run.smithi029.stderr:512 bytes copied, 0.000130585 s, 3.9 MB/s 2023-11-27T15:17:44.292 DEBUG:teuthology.orchestra.run.smithi029:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-27T15:17:44.354 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2023-11-27T15:17:44.354 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2023-11-27T15:17:44.374 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-27T15:17:44.375 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2023-11-27T15:17:44.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:44 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:44.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:44 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:17:44.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:44 smithi029 ceph-mon[131166]: Deploying daemon mgr.z on smithi060 2023-11-27T15:17:44.433 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-27T15:17:44.433 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:44.433 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 391401 Links: 1 2023-11-27T15:17:44.434 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:44.434 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:44.434 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-11-27 15:17:44.129444805 +0000 2023-11-27T15:17:44.434 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-11-27 15:06:53.249248366 +0000 2023-11-27T15:17:44.434 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-11-27 15:06:53.249248366 +0000 2023-11-27T15:17:44.434 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2023-11-27T15:17:44.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-27T15:17:44.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:44 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:17:44.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:44 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:17:44.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:44 smithi060 ceph-mon[131518]: Deploying daemon mgr.z on smithi060 2023-11-27T15:17:44.475 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:44 smithi060 systemd[1]: Starting Ceph mgr.z for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:17:44.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-11-27T15:17:44.503 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-11-27T15:17:44.503 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.00011266 s, 4.5 MB/s 2023-11-27T15:17:44.504 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-27T15:17:44.563 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2023-11-27T15:17:44.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:44 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:44.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:44 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:17:44.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:44 smithi008 ceph-mon[128788]: Deploying daemon mgr.z on smithi060 2023-11-27T15:17:44.621 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-27T15:17:44.621 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:44.622 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 394619 Links: 1 2023-11-27T15:17:44.622 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:44.622 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:44.622 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-11-27 15:17:44.129444805 +0000 2023-11-27T15:17:44.622 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-11-27 15:06:53.752243614 +0000 2023-11-27T15:17:44.622 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-11-27 15:06:53.752243614 +0000 2023-11-27T15:17:44.622 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2023-11-27T15:17:44.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-27T15:17:44.689 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-11-27T15:17:44.689 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-11-27T15:17:44.689 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000104335 s, 4.9 MB/s 2023-11-27T15:17:44.690 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-27T15:17:44.749 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2023-11-27T15:17:44.773 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:44 smithi060 podman[133352]: 2023-11-27T15:17:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-27T15:17:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:44.792 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 395293 Links: 1 2023-11-27T15:17:44.792 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:44.793 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:44.793 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-11-27 15:17:44.129444805 +0000 2023-11-27T15:17:44.793 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-11-27 15:06:54.263238786 +0000 2023-11-27T15:17:44.793 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-11-27 15:06:54.263238786 +0000 2023-11-27T15:17:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2023-11-27T15:17:44.793 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-27T15:17:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-11-27T15:17:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-11-27T15:17:44.860 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000101746 s, 5.0 MB/s 2023-11-27T15:17:44.861 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-27T15:17:44.919 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2023-11-27T15:17:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-27T15:17:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:17:44.977 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 6h/6d Inode: 393499 Links: 1 2023-11-27T15:17:44.977 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:17:44.977 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:17:44.977 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2023-11-27 15:17:44.130444796 +0000 2023-11-27T15:17:44.977 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2023-11-27 15:06:54.790233807 +0000 2023-11-27T15:17:44.977 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2023-11-27 15:06:54.790233807 +0000 2023-11-27T15:17:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2023-11-27T15:17:44.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-27T15:17:45.028 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:44 smithi060 bash[133352]: 62605840e05a0365be4e76dec6f46d4c1453507207e21a94c48c49deb268f238 2023-11-27T15:17:45.028 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:44 smithi060 ceph-mgr[133535]: -- 172.21.15.60:0/4185598517 <== mon.1 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560d886ab520 con 0x560d88680800 2023-11-27T15:17:45.028 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:44 smithi060 systemd[1]: Started Ceph mgr.z for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:17:45.028 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:44 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:44.927+0000 7fa420490200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:17:45.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2023-11-27T15:17:45.043 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2023-11-27T15:17:45.043 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000111937 s, 4.6 MB/s 2023-11-27T15:17:45.044 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-27T15:17:45.105 INFO:tasks.cephadm:Deploying osd.0 on smithi008 with /dev/vg_nvme/lv_4... 2023-11-27T15:17:45.105 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-27T15:17:45.325 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:45.024+0000 7fa420490200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:17:45.325 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:45.135+0000 7fa420490200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:17:45.325 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:45.245+0000 7fa420490200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:17:45.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-mon[131518]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:45.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:45.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:45.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:45.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:17:45.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:17:45.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:17:45.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:45 smithi008 ceph-mon[128788]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:45.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:45.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:17:45.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:17:45.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:17:45.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:45.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:45 smithi029 ceph-mon[131166]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:45.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:45.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:45.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:45.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:17:45.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:17:45.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:17:45.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:45.824 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:45.450+0000 7fa420490200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:17:45.967 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:46.106 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:45.844+0000 7fa420490200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:17:46.106 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:45.952+0000 7fa420490200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:17:46.386 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:46 smithi029 ceph-mon[131166]: Deploying daemon mgr.y on smithi029 2023-11-27T15:17:46.572 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:46 smithi008 ceph-mon[128788]: Deploying daemon mgr.y on smithi029 2023-11-27T15:17:46.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:46 smithi060 ceph-mon[131518]: Deploying daemon mgr.y on smithi029 2023-11-27T15:17:46.574 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:46 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:46.101+0000 7fa420490200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:17:46.575 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:46 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:46.216+0000 7fa420490200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:17:46.648 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:46 smithi029 systemd[1]: Starting Ceph mgr.y for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:17:47.074 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:46 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:46.724+0000 7fa420490200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:17:47.075 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:46 smithi029 podman[132603]: 2023-11-27T15:17:47.075 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:47 smithi029 bash[132603]: ef6a443154b6d88437c838945d6613fc0651c286a94238eedf8610bd2ab04bdc 2023-11-27T15:17:47.075 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-mgr[132643]: -- 172.21.15.29:0/52361492 <== mon.2 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55767235b520 con 0x557672330800 2023-11-27T15:17:47.371 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-mon[131166]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:47.371 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.372 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.372 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.372 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.372 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:47 smithi029 systemd[1]: Started Ceph mgr.y for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:17:47.372 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:47.163+0000 7f778b36f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:17:47.372 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:47.258+0000 7f778b36f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:17:47.372 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:47.367+0000 7f778b36f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:17:47.574 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:47 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:47.419+0000 7fa420490200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:17:47.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:47 smithi060 ceph-mon[131518]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:47.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:47 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:47 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:47 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:47 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:47 smithi008 ceph-mon[128788]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:47.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:47 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:47 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:47 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:47 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:47.647 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:47.482+0000 7f778b36f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:17:47.901 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-27T15:17:47.901 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-27T15:17:47.901 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2023-11-27T15:17:47.901 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2023-11-27T15:17:47.901 INFO:teuthology.orchestra.run.smithi008.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.040705 s, 258 MB/s 2023-11-27T15:17:47.901 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2023-11-27T15:17:48.074 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:47 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:47.617+0000 7fa420490200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:17:48.104 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:47.694+0000 7f778b36f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:17:48.343 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:48 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:48.343 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:48.350 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi008:vg_nvme/lv_4 2023-11-27T15:17:48.370 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:48 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:48.370 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:48.371 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:48.100+0000 7f778b36f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:17:48.371 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:48.210+0000 7f778b36f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:17:48.371 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:48.366+0000 7f778b36f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:17:48.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:48 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:17:48.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:48.649 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:48.483+0000 7f778b36f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:17:48.983 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:48 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:48.624+0000 7fa420490200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:17:48.983 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:48 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:48.738+0000 7fa420490200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:17:48.983 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:48 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:48.978+0000 7fa420490200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:17:49.212 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:49.316 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:49.011+0000 7f778b36f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:17:49.575 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:49.332+0000 7fa420490200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:17:49.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:49.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:49.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:49.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:17:49.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:17:49.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:49.604 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:49.604 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.605 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.605 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.605 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.605 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.605 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:49.605 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:49.605 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.605 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:17:49.605 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:17:49.605 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:49.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:49.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:49.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:49.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:49.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:17:49.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:17:49.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:50.074 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:49.584+0000 7fa420490200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:17:50.074 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:49.764+0000 7fa420490200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:17:50.147 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:49.711+0000 7f778b36f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:17:50.147 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:49 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:49.901+0000 7f778b36f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:17:50.574 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:50.091+0000 7fa420490200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:17:50.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:50 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:17:50.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:50 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:17:50.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:50 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:17:50.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:50 smithi060 ceph-mon[131518]: Reconfiguring mgr.x (unknown last config time)... 2023-11-27T15:17:50.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:50 smithi060 ceph-mon[131518]: Reconfiguring daemon mgr.x on smithi008 2023-11-27T15:17:50.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:50 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:50.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:50 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:17:50.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:50 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:50.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:50 smithi008 ceph-mon[128788]: Reconfiguring mgr.x (unknown last config time)... 2023-11-27T15:17:50.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:50 smithi008 ceph-mon[128788]: Reconfiguring daemon mgr.x on smithi008 2023-11-27T15:17:50.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:50 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:50.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:50 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:17:50.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:50 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:50.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:50 smithi029 ceph-mon[131166]: Reconfiguring mgr.x (unknown last config time)... 2023-11-27T15:17:50.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:50 smithi029 ceph-mon[131166]: Reconfiguring daemon mgr.x on smithi008 2023-11-27T15:17:51.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.119+0000 7f2bf14bc700 1 -- 172.21.15.8:0/1920652890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bec074b20 msgr2=0x7f2bec072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:51.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.119+0000 7f2bf14bc700 1 --2- 172.21.15.8:0/1920652890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bec074b20 0x7f2bec072f80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009990 tx=0x7f2bdc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:51.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.119+0000 7f2bf14bc700 1 -- 172.21.15.8:0/1920652890 shutdown_connections 2023-11-27T15:17:51.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.119+0000 7f2bf14bc700 1 --2- 172.21.15.8:0/1920652890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec10c8d0 0x7f2bec10ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:51.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.119+0000 7f2bf14bc700 1 --2- 172.21.15.8:0/1920652890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bec074b20 0x7f2bec072f80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:51.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.119+0000 7f2bf14bc700 1 --2- 172.21.15.8:0/1920652890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2bec073550 0x7f2bec0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:51.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.119+0000 7f2bf14bc700 1 -- 172.21.15.8:0/1920652890 >> 172.21.15.8:0/1920652890 conn(0x7f2bec0fb900 msgr2=0x7f2bec0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:51.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.119+0000 7f2bf14bc700 1 -- 172.21.15.8:0/1920652890 shutdown_connections 2023-11-27T15:17:51.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.119+0000 7f2bf14bc700 1 -- 172.21.15.8:0/1920652890 wait complete. 2023-11-27T15:17:51.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.120+0000 7f2bf14bc700 1 Processor -- start 2023-11-27T15:17:51.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.120+0000 7f2bf14bc700 1 -- start start 2023-11-27T15:17:51.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bf14bc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2bec073550 0x7f2bec19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:51.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bf14bc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec074b20 0x7f2bec19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:51.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bf14bc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bec10c8d0 0x7f2bec1a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:51.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bf14bc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1a41a0 con 0x7f2bec073550 2023-11-27T15:17:51.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bf14bc700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1a42e0 con 0x7f2bec10c8d0 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bf14bc700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1a4470 con 0x7f2bec074b20 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bea7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec074b20 0x7f2bec19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bea7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec074b20 0x7f2bec19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:35316/0 (socket says 172.21.15.8:35316) 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2beaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2bec073550 0x7f2bec19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bea7fc700 1 -- 172.21.15.8:0/2662106154 learned_addr learned my addr 172.21.15.8:0/2662106154 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2beb7fe700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bec10c8d0 0x7f2bec1a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bea7fc700 1 -- 172.21.15.8:0/2662106154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bec10c8d0 msgr2=0x7f2bec1a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bea7fc700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bec10c8d0 0x7f2bec1a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bea7fc700 1 -- 172.21.15.8:0/2662106154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2bec073550 msgr2=0x7f2bec19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bea7fc700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2bec073550 0x7f2bec19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:51.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.121+0000 7f2bea7fc700 1 -- 172.21.15.8:0/2662106154 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be0009520 con 0x7f2bec074b20 2023-11-27T15:17:51.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.122+0000 7f2bea7fc700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec074b20 0x7f2bec19cb50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2be000eb60 tx=0x7f2be000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:51.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.122+0000 7f2bd3fff700 1 -- 172.21.15.8:0/2662106154 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be000cbd0 con 0x7f2bec074b20 2023-11-27T15:17:51.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.122+0000 7f2bd3fff700 1 -- 172.21.15.8:0/2662106154 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be00044e0 con 0x7f2bec074b20 2023-11-27T15:17:51.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.122+0000 7f2bf14bc700 1 -- 172.21.15.8:0/2662106154 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc0095f0 con 0x7f2bec074b20 2023-11-27T15:17:51.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.122+0000 7f2bd3fff700 1 -- 172.21.15.8:0/2662106154 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0003df0 con 0x7f2bec074b20 2023-11-27T15:17:51.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.122+0000 7f2bf14bc700 1 -- 172.21.15.8:0/2662106154 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec1122a0 con 0x7f2bec074b20 2023-11-27T15:17:51.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.123+0000 7f2bd3fff700 1 -- 172.21.15.8:0/2662106154 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f2be0004020 con 0x7f2bec074b20 2023-11-27T15:17:51.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.123+0000 7f2bf14bc700 1 -- 172.21.15.8:0/2662106154 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec102420 con 0x7f2bec074b20 2023-11-27T15:17:51.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.124+0000 7f2bd3fff700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2bd40389e0 0x7f2bd403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:51.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.124+0000 7f2bd3fff700 1 -- 172.21.15.8:0/2662106154 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2be0014070 con 0x7f2bec074b20 2023-11-27T15:17:51.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.124+0000 7f2beaffd700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2bd40389e0 0x7f2bd403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:51.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.124+0000 7f2beaffd700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2bd40389e0 0x7f2bd403ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00b2e0 tx=0x7f2bdc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:51.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.129+0000 7f2bd3fff700 1 -- 172.21.15.8:0/2662106154 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2be0020b80 con 0x7f2bec074b20 2023-11-27T15:17:51.147 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:50 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:50.862+0000 7f778b36f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:17:51.148 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:50 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:50.972+0000 7f778b36f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:17:51.320 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:50.917+0000 7fa420490200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:17:51.320 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:51.074+0000 7fa420490200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:17:51.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:51.352+0000 7f2bf14bc700 1 -- 172.21.15.8:0/2662106154 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bec060f30 con 0x7f2bd40389e0 2023-11-27T15:17:51.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:51 smithi008 ceph-mon[128788]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:51.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:51.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:51.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:51.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:51.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:51.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:51.540 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:51.188+0000 7f778b36f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:17:51.540 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-mon[131166]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:51.540 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:51.540 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:51.540 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:51.540 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:51.541 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:51.541 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:51.574 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:51.315+0000 7fa420490200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:17:51.574 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:51.453+0000 7fa420490200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:17:51.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-mon[131518]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:51.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:51.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:51.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:51.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:51.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:51.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:17:51.792 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:51.535+0000 7f778b36f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:17:52.074 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:51.721+0000 7fa420490200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:17:52.148 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:51.787+0000 7f778b36f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:17:52.148 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:51 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:51.968+0000 7f778b36f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:17:52.587 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:52 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.587 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:52 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:17:52.587 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:52 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.587 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:52 smithi008 ceph-mon[128788]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:52.587 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:52.587 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:52.587 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:52.647 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:52 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:52.291+0000 7f778b36f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:17:52.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:52 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:52 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:17:52.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:52 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:52 smithi029 ceph-mon[131166]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:52.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:52.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:52.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:52.763 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:52 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.763 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:52 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:17:52.763 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:52 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.763 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:52 smithi060 ceph-mon[131518]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:52.764 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:52.764 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:52.764 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:53.075 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:17:52 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z[133449]: 2023-11-27T15:17:52.759+0000 7fa420490200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:17:53.398 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:53.113+0000 7f778b36f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:17:53.398 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:53.270+0000 7f778b36f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:17:53.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-mon[131166]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:53.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-mon[131166]: Standby manager daemon z started 2023-11-27T15:17:53.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-mon[131166]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:17:53.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-mon[131166]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:53.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-mon[131166]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:17:53.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-mon[131166]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:53.594 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:53 smithi008 ceph-mon[128788]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:53.595 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:53 smithi008 ceph-mon[128788]: Standby manager daemon z started 2023-11-27T15:17:53.595 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:53 smithi008 ceph-mon[128788]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:17:53.595 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:53 smithi008 ceph-mon[128788]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:53.595 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:53 smithi008 ceph-mon[128788]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:17:53.595 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:53 smithi008 ceph-mon[128788]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:53.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:53.718+0000 7f2bd3fff700 1 -- 172.21.15.8:0/2662106154 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mgrmap(e 14) v1 ==== 90598+0+0 (secure 0 0 0) 0x7f2be0018950 con 0x7f2bec074b20 2023-11-27T15:17:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:53 smithi060 ceph-mon[131518]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:53 smithi060 ceph-mon[131518]: Standby manager daemon z started 2023-11-27T15:17:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:53 smithi060 ceph-mon[131518]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:17:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:53 smithi060 ceph-mon[131518]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:53 smithi060 ceph-mon[131518]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:17:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:53 smithi060 ceph-mon[131518]: from='mgr.? 172.21.15.60:0/2039195485' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:53.897 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:53.510+0000 7f778b36f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:17:53.897 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:53.646+0000 7f778b36f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:17:54.397 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:53 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:53.915+0000 7f778b36f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:17:55.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:54 smithi060 ceph-mon[131518]: mgrmap e14: x(active, since 75s), standbys: z 2023-11-27T15:17:55.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:17:55.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:54 smithi060 ceph-mon[131518]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:55.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:54 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1638377414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f832011f-25d1-4f98-adf6-6c2225afa50c"}]: dispatch 2023-11-27T15:17:55.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:54 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1638377414' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f832011f-25d1-4f98-adf6-6c2225afa50c"}]': finished 2023-11-27T15:17:55.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:54 smithi060 ceph-mon[131518]: osdmap e5: 1 total, 0 up, 1 in 2023-11-27T15:17:55.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:17:55.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:54 smithi008 ceph-mon[128788]: mgrmap e14: x(active, since 75s), standbys: z 2023-11-27T15:17:55.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:17:55.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:54 smithi008 ceph-mon[128788]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:55.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:54 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1638377414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f832011f-25d1-4f98-adf6-6c2225afa50c"}]: dispatch 2023-11-27T15:17:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:54 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1638377414' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f832011f-25d1-4f98-adf6-6c2225afa50c"}]': finished 2023-11-27T15:17:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:54 smithi008 ceph-mon[128788]: osdmap e5: 1 total, 0 up, 1 in 2023-11-27T15:17:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:17:55.147 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:17:54 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y[132619]: 2023-11-27T15:17:54.952+0000 7f778b36f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:17:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:54 smithi029 ceph-mon[131166]: mgrmap e14: x(active, since 75s), standbys: z 2023-11-27T15:17:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:17:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:54 smithi029 ceph-mon[131166]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:54 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1638377414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f832011f-25d1-4f98-adf6-6c2225afa50c"}]: dispatch 2023-11-27T15:17:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:54 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1638377414' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f832011f-25d1-4f98-adf6-6c2225afa50c"}]': finished 2023-11-27T15:17:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:54 smithi029 ceph-mon[131166]: osdmap e5: 1 total, 0 up, 1 in 2023-11-27T15:17:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:17:55.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:17:55.731+0000 7f2bd3fff700 1 -- 172.21.15.8:0/2662106154 <== mon.1 v2:172.21.15.60:3300/0 8 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f2be0020390 con 0x7f2bec074b20 2023-11-27T15:17:56.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:55 smithi060 ceph-mon[131518]: Standby manager daemon y started 2023-11-27T15:17:56.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:55 smithi060 ceph-mon[131518]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:17:56.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:55 smithi060 ceph-mon[131518]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:56.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:55 smithi060 ceph-mon[131518]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:17:56.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:55 smithi060 ceph-mon[131518]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:56.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:55 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2777764026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:17:56.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:55 smithi008 ceph-mon[128788]: Standby manager daemon y started 2023-11-27T15:17:56.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:55 smithi008 ceph-mon[128788]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:17:56.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:55 smithi008 ceph-mon[128788]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:56.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:55 smithi008 ceph-mon[128788]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:17:56.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:55 smithi008 ceph-mon[128788]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:56.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:55 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2777764026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:17:56.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:55 smithi029 ceph-mon[131166]: Standby manager daemon y started 2023-11-27T15:17:56.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:55 smithi029 ceph-mon[131166]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:17:56.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:55 smithi029 ceph-mon[131166]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:17:56.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:55 smithi029 ceph-mon[131166]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:17:56.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:55 smithi029 ceph-mon[131166]: from='mgr.? 172.21.15.29:0/596286062' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:17:56.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:55 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2777764026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:17:57.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:56 smithi060 ceph-mon[131518]: mgrmap e15: x(active, since 77s), standbys: z, y 2023-11-27T15:17:57.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:56 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:17:57.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:56 smithi060 ceph-mon[131518]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:57.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:56 smithi008 ceph-mon[128788]: mgrmap e15: x(active, since 77s), standbys: z, y 2023-11-27T15:17:57.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:56 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:17:57.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:56 smithi008 ceph-mon[128788]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:57.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:56 smithi029 ceph-mon[131166]: mgrmap e15: x(active, since 77s), standbys: z, y 2023-11-27T15:17:57.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:56 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:17:57.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:56 smithi029 ceph-mon[131166]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:58.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:17:58 smithi008 ceph-mon[128788]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:59.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:17:58 smithi060 ceph-mon[131518]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:17:59.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:17:58 smithi029 ceph-mon[131166]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:01.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:01 smithi060 ceph-mon[131518]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:01.612 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:01 smithi029 ceph-mon[131166]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:01 smithi008 ceph-mon[128788]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:02.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:02 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-27T15:18:02.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:02 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:02.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:02 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-27T15:18:02.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:02 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:02.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:02 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-27T15:18:02.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:02 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:03.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:03 smithi060 ceph-mon[131518]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:03.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:03 smithi060 ceph-mon[131518]: Deploying daemon osd.0 on smithi008 2023-11-27T15:18:03.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:03 smithi008 ceph-mon[128788]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:03.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:03 smithi008 ceph-mon[128788]: Deploying daemon osd.0 on smithi008 2023-11-27T15:18:03.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:03 smithi029 ceph-mon[131166]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:03.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:03 smithi029 ceph-mon[131166]: Deploying daemon osd.0 on smithi008 2023-11-27T15:18:05.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:05 smithi008 ceph-mon[128788]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:05 smithi029 ceph-mon[131166]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:05 smithi060 ceph-mon[131518]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:07.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:07 smithi008 ceph-mon[128788]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:07.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:07 smithi029 ceph-mon[131166]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:07 smithi060 ceph-mon[131518]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:09.822 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:09 smithi008 ceph-mon[128788]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:09.822 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:09.822 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:09.822 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:09 smithi060 ceph-mon[131518]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:09.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:09.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:09 smithi029 ceph-mon[131166]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:10 smithi060 ceph-mon[131518]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:10 smithi008 ceph-mon[128788]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:10.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:10 smithi029 ceph-mon[131166]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:12.301 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:12.301 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:12.302 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:12.302 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:12.302 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:12 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:18:12.302 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:12.302 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:12 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:18:12.302 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:12 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:18:12.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:12.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:12.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:12.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:12.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:12 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:18:12.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:12.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:12 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:18:12.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:12 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:18:12.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.366+0000 7f2bd3fff700 1 -- 172.21.15.8:0/2662106154 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2bec060f30 con 0x7f2bd40389e0 2023-11-27T15:18:12.371 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 0 on host 'smithi008' 2023-11-27T15:18:12.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.368+0000 7f2bf14bc700 1 -- 172.21.15.8:0/2662106154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2bd40389e0 msgr2=0x7f2bd403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:12.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.368+0000 7f2bf14bc700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2bd40389e0 0x7f2bd403ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00b2e0 tx=0x7f2bdc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:18:12.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.368+0000 7f2bf14bc700 1 -- 172.21.15.8:0/2662106154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec074b20 msgr2=0x7f2bec19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:12.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.368+0000 7f2bf14bc700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec074b20 0x7f2bec19cb50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2be000eb60 tx=0x7f2be000ee70 comp rx=0 tx=0).stop 2023-11-27T15:18:12.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.369+0000 7f2bf14bc700 1 -- 172.21.15.8:0/2662106154 shutdown_connections 2023-11-27T15:18:12.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.369+0000 7f2bf14bc700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2bec10c8d0 0x7f2bec1a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:12.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.369+0000 7f2bf14bc700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2bec073550 0x7f2bec19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:12.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.369+0000 7f2bf14bc700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2bd40389e0 0x7f2bd403ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:12.373 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.369+0000 7f2bf14bc700 1 --2- 172.21.15.8:0/2662106154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2bec074b20 0x7f2bec19cb50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:12.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.369+0000 7f2bf14bc700 1 -- 172.21.15.8:0/2662106154 >> 172.21.15.8:0/2662106154 conn(0x7f2bec0fb900 msgr2=0x7f2bec0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:12.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.369+0000 7f2bf14bc700 1 -- 172.21.15.8:0/2662106154 shutdown_connections 2023-11-27T15:18:12.374 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:18:12.369+0000 7f2bf14bc700 1 -- 172.21.15.8:0/2662106154 wait complete. 2023-11-27T15:18:12.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:12.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:12.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:12.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:12.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:12 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:18:12.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:12.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:12 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:18:12.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:12 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:18:12.827 DEBUG:teuthology.orchestra.run.smithi008:osd.0> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.0.service 2023-11-27T15:18:12.833 INFO:tasks.cephadm:Deploying osd.1 on smithi029 with /dev/vg_nvme/lv_4... 2023-11-27T15:18:12.834 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-27T15:18:12.911 INFO:journalctl@ceph.osd.0.smithi008.stdout:-- Logs begin at Mon 2023-11-27 14:56:10 UTC. -- 2023-11-27T15:18:13.495 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:18:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:13 smithi008 ceph-mon[128788]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:13 smithi008 ceph-mon[128788]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:18:13.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:13 smithi029 ceph-mon[131166]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:13.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:13.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:13.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:13.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:13 smithi029 ceph-mon[131166]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:18:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:13 smithi060 ceph-mon[131518]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:13.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:13.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:13.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:13.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:13 smithi060 ceph-mon[131518]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:18:14.512 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:14 smithi029 ceph-mon[131166]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-27T15:18:14.512 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:14 smithi029 ceph-mon[131166]: osdmap e6: 1 total, 0 up, 1 in 2023-11-27T15:18:14.512 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:14.512 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:14 smithi029 ceph-mon[131166]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:18:14.602 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:14 smithi008 ceph-mon[128788]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-27T15:18:14.602 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:14 smithi008 ceph-mon[128788]: osdmap e6: 1 total, 0 up, 1 in 2023-11-27T15:18:14.602 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:14.602 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:14 smithi008 ceph-mon[128788]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:18:14.602 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:18:14 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-0[141167]: 2023-11-27T15:18:14.343+0000 7f3ddde50700 -1 osd.0 0 waiting for initial osdmap 2023-11-27T15:18:14.602 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:18:14 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-0[141167]: 2023-11-27T15:18:14.347+0000 7f3dd6c3f700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:18:14.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:14 smithi060 ceph-mon[131518]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-27T15:18:14.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:14 smithi060 ceph-mon[131518]: osdmap e6: 1 total, 0 up, 1 in 2023-11-27T15:18:14.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:14.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:14 smithi060 ceph-mon[131518]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:18:14.998 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-27T15:18:14.999 INFO:teuthology.orchestra.run.smithi029.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-27T15:18:14.999 INFO:teuthology.orchestra.run.smithi029.stderr: stderr: 10+0 records in 2023-11-27T15:18:14.999 INFO:teuthology.orchestra.run.smithi029.stderr:10+0 records out 2023-11-27T15:18:14.999 INFO:teuthology.orchestra.run.smithi029.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00811832 s, 1.3 GB/s 2023-11-27T15:18:14.999 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping successful for: 2023-11-27T15:18:15.340 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi029:vg_nvme/lv_4 2023-11-27T15:18:15.598 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:15 smithi008 ceph-mon[128788]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:15.599 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:15 smithi008 ceph-mon[128788]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:18:15.599 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:15 smithi008 ceph-mon[128788]: osdmap e7: 1 total, 0 up, 1 in 2023-11-27T15:18:15.599 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:15.599 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:15 smithi029 ceph-mon[131166]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:15 smithi029 ceph-mon[131166]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:18:15.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:15 smithi029 ceph-mon[131166]: osdmap e7: 1 total, 0 up, 1 in 2023-11-27T15:18:15.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:15.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:15.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:15 smithi060 ceph-mon[131518]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:15.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:15 smithi060 ceph-mon[131518]: from='osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:18:15.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:15 smithi060 ceph-mon[131518]: osdmap e7: 1 total, 0 up, 1 in 2023-11-27T15:18:15.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:15.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:15.993 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:18:16.470 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:18:16.470 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:18:16.470 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:16.470 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] boot 2023-11-27T15:18:16.470 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: osdmap e8: 1 total, 1 up, 1 in 2023-11-27T15:18:16.471 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:16.471 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.471 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.471 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.471 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.471 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:16.471 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:16.471 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] boot 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: osdmap e8: 1 total, 1 up, 1 in 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:16.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: osd.0 [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] boot 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: osdmap e8: 1 total, 1 up, 1 in 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:16.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:16.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.191+0000 7f1b36a29700 1 -- 172.21.15.29:0/2936340202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b300ff1e0 msgr2=0x7f1b300ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.191+0000 7f1b36a29700 1 --2- 172.21.15.29:0/2936340202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b300ff1e0 0x7f1b300ff5f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009990 tx=0x7f1b20009ca0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.191+0000 7f1b36a29700 1 -- 172.21.15.29:0/2936340202 shutdown_connections 2023-11-27T15:18:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.191+0000 7f1b36a29700 1 --2- 172.21.15.29:0/2936340202 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b30100d70 0x7f1b30107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.197 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.191+0000 7f1b36a29700 1 --2- 172.21.15.29:0/2936340202 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b300ff1e0 0x7f1b300ff5f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.191+0000 7f1b36a29700 1 --2- 172.21.15.29:0/2936340202 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b301003e0 0x7f1b30100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.191+0000 7f1b36a29700 1 -- 172.21.15.29:0/2936340202 >> 172.21.15.29:0/2936340202 conn(0x7f1b300fa7f0 msgr2=0x7f1b300fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.191+0000 7f1b36a29700 1 -- 172.21.15.29:0/2936340202 shutdown_connections 2023-11-27T15:18:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.191+0000 7f1b36a29700 1 -- 172.21.15.29:0/2936340202 wait complete. 2023-11-27T15:18:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.192+0000 7f1b36a29700 1 Processor -- start 2023-11-27T15:18:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.192+0000 7f1b36a29700 1 -- start start 2023-11-27T15:18:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.192+0000 7f1b36a29700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b300ff1e0 0x7f1b30068770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.192+0000 7f1b36a29700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b301003e0 0x7f1b30068cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.192+0000 7f1b36a29700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b30100d70 0x7f1b300711f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.198 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.192+0000 7f1b36a29700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b30069410 con 0x7f1b300ff1e0 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.192+0000 7f1b36a29700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b30069580 con 0x7f1b301003e0 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.192+0000 7f1b36a29700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300717b0 con 0x7f1b30100d70 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b301003e0 0x7f1b30068cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2f7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b301003e0 0x7f1b30068cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:34994/0 (socket says 172.21.15.29:34994) 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2f7fe700 1 -- 172.21.15.29:0/728372948 learned_addr learned my addr 172.21.15.29:0/728372948 (peer_addr_for_me v2:172.21.15.29:0/0) 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2ffff700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b300ff1e0 0x7f1b30068770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b34fc6700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b30100d70 0x7f1b300711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2f7fe700 1 -- 172.21.15.29:0/728372948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b30100d70 msgr2=0x7f1b300711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2f7fe700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b30100d70 0x7f1b300711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2f7fe700 1 -- 172.21.15.29:0/728372948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b300ff1e0 msgr2=0x7f1b30068770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2f7fe700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b300ff1e0 0x7f1b30068770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:17.199 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2f7fe700 1 -- 172.21.15.29:0/728372948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b200095f0 con 0x7f1b301003e0 2023-11-27T15:18:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b34fc6700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b30100d70 0x7f1b300711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:18:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2f7fe700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b301003e0 0x7f1b30068cb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800c920 tx=0x7f1b1800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2d7fa700 1 -- 172.21.15.29:0/728372948 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18007b00 con 0x7f1b301003e0 2023-11-27T15:18:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2d7fa700 1 -- 172.21.15.29:0/728372948 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b18004cf0 con 0x7f1b301003e0 2023-11-27T15:18:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b36a29700 1 -- 172.21.15.29:0/728372948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3010abb0 con 0x7f1b301003e0 2023-11-27T15:18:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b2d7fa700 1 -- 172.21.15.29:0/728372948 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b18005610 con 0x7f1b301003e0 2023-11-27T15:18:17.200 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.193+0000 7f1b36a29700 1 -- 172.21.15.29:0/728372948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3010aa20 con 0x7f1b301003e0 2023-11-27T15:18:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.194+0000 7f1b36a29700 1 -- 172.21.15.29:0/728372948 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b30071f40 con 0x7f1b301003e0 2023-11-27T15:18:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.195+0000 7f1b2d7fa700 1 -- 172.21.15.29:0/728372948 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f1b18004750 con 0x7f1b301003e0 2023-11-27T15:18:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.196+0000 7f1b2d7fa700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b1c0a1140 0x7f1b1c0a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.196+0000 7f1b2d7fa700 1 -- 172.21.15.29:0/728372948 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f1b180cad90 con 0x7f1b301003e0 2023-11-27T15:18:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.196+0000 7f1b2ffff700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b1c0a1140 0x7f1b1c0a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:17.201 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.197+0000 7f1b2ffff700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b1c0a1140 0x7f1b1c0a35f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000b5c0 tx=0x7f1b20005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:17.204 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.200+0000 7f1b2d7fa700 1 -- 172.21.15.29:0/728372948 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1b180ce030 con 0x7f1b301003e0 2023-11-27T15:18:17.406 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:17.403+0000 7f1b36a29700 1 -- 172.21.15.29:0/728372948 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b30060f30 con 0x7f1b1c0a1140 2023-11-27T15:18:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:17 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi008 2023-11-27T15:18:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:17 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi008 to 17276M 2023-11-27T15:18:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:17 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:17 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:18:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:17 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:17 smithi008 ceph-mon[128788]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:17.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:17 smithi008 ceph-mon[128788]: osdmap e9: 1 total, 1 up, 1 in 2023-11-27T15:18:17.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:17 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi008 2023-11-27T15:18:17.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:17 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi008 to 17276M 2023-11-27T15:18:17.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:17 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:17 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:18:17.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:17 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:17 smithi029 ceph-mon[131166]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:17.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:17 smithi029 ceph-mon[131166]: osdmap e9: 1 total, 1 up, 1 in 2023-11-27T15:18:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:17 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi008 2023-11-27T15:18:17.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:17 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi008 to 17276M 2023-11-27T15:18:17.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:17 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:17 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:18:17.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:17 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:17 smithi060 ceph-mon[131518]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:18:17.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:17 smithi060 ceph-mon[131518]: osdmap e9: 1 total, 1 up, 1 in 2023-11-27T15:18:18.587 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:18 smithi029 ceph-mon[131166]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:18.587 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:18 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:18.587 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:18 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:18.587 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:18 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:18 smithi008 ceph-mon[128788]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:18 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:18 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:18 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:18.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:18 smithi060 ceph-mon[131518]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:18.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:18 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:18.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:18 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:18.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:18 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:19 smithi008 ceph-mon[128788]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:19 smithi029 ceph-mon[131166]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:19.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:19 smithi060 ceph-mon[131518]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:20.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:20 smithi008 ceph-mon[128788]: from='client.? 172.21.15.29:0/3830276895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b3d6a3d-e153-4cc3-9551-81eb07f635c4"}]: dispatch 2023-11-27T15:18:20.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:20 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b3d6a3d-e153-4cc3-9551-81eb07f635c4"}]: dispatch 2023-11-27T15:18:20.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:20 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b3d6a3d-e153-4cc3-9551-81eb07f635c4"}]': finished 2023-11-27T15:18:20.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:20 smithi008 ceph-mon[128788]: osdmap e10: 2 total, 1 up, 2 in 2023-11-27T15:18:20.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:20 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:20.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:20 smithi029 ceph-mon[131166]: from='client.? 172.21.15.29:0/3830276895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b3d6a3d-e153-4cc3-9551-81eb07f635c4"}]: dispatch 2023-11-27T15:18:20.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:20 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b3d6a3d-e153-4cc3-9551-81eb07f635c4"}]: dispatch 2023-11-27T15:18:20.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:20 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b3d6a3d-e153-4cc3-9551-81eb07f635c4"}]': finished 2023-11-27T15:18:20.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:20 smithi029 ceph-mon[131166]: osdmap e10: 2 total, 1 up, 2 in 2023-11-27T15:18:20.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:20 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:20.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:20 smithi060 ceph-mon[131518]: from='client.? 172.21.15.29:0/3830276895' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b3d6a3d-e153-4cc3-9551-81eb07f635c4"}]: dispatch 2023-11-27T15:18:20.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:20 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6b3d6a3d-e153-4cc3-9551-81eb07f635c4"}]: dispatch 2023-11-27T15:18:20.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:20 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6b3d6a3d-e153-4cc3-9551-81eb07f635c4"}]': finished 2023-11-27T15:18:20.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:20 smithi060 ceph-mon[131518]: osdmap e10: 2 total, 1 up, 2 in 2023-11-27T15:18:20.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:20 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:21.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:21 smithi008 ceph-mon[128788]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:21.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:21 smithi008 ceph-mon[128788]: from='client.? 172.21.15.29:0/801297989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:21.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:21 smithi029 ceph-mon[131166]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:21.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:21 smithi029 ceph-mon[131166]: from='client.? 172.21.15.29:0/801297989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:21 smithi060 ceph-mon[131518]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:21 smithi060 ceph-mon[131518]: from='client.? 172.21.15.29:0/801297989' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:23.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:23 smithi008 ceph-mon[128788]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:23.645 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:23 smithi029 ceph-mon[131166]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:23 smithi060 ceph-mon[131518]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:25.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:25 smithi029 ceph-mon[131166]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:25.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:25 smithi060 ceph-mon[131518]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:25 smithi008 ceph-mon[128788]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:27.478 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:27 smithi029 ceph-mon[131166]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:27.478 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:27 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-27T15:18:27.478 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:27 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:27.478 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:27 smithi029 ceph-mon[131166]: Deploying daemon osd.1 on smithi029 2023-11-27T15:18:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:27 smithi060 ceph-mon[131518]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:27.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:27 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-27T15:18:27.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:27 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:27.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:27 smithi060 ceph-mon[131518]: Deploying daemon osd.1 on smithi029 2023-11-27T15:18:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:27 smithi008 ceph-mon[128788]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:27 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-27T15:18:27.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:27 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:27.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:27 smithi008 ceph-mon[128788]: Deploying daemon osd.1 on smithi029 2023-11-27T15:18:28.991 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:28 smithi029 ceph-mon[131166]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:29.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:28 smithi060 ceph-mon[131518]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:29.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:28 smithi008 ceph-mon[128788]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:31.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:31 smithi008 ceph-mon[128788]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:31.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:31 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:31.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:31 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:31.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:31 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:31.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:31 smithi029 ceph-mon[131166]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:31.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:31 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:31.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:31 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:31.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:31 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:31 smithi060 ceph-mon[131518]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:31.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:31 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:31.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:31 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:31.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:31 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:33.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:33 smithi008 ceph-mon[128788]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:33.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:33.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:33.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:33.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:33.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:33 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:33.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:33 smithi029 ceph-mon[131166]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:33.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:33 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:33.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:33 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:33.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:33 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:33.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:33 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:33.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:33 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:33.723 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.718+0000 7f1b2d7fa700 1 -- 172.21.15.29:0/728372948 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1b30060f30 con 0x7f1b1c0a1140 2023-11-27T15:18:33.724 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 1 on host 'smithi029' 2023-11-27T15:18:33.726 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 -- 172.21.15.29:0/728372948 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b1c0a1140 msgr2=0x7f1b1c0a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:33.726 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b1c0a1140 0x7f1b1c0a35f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000b5c0 tx=0x7f1b20005fb0 comp rx=0 tx=0).stop 2023-11-27T15:18:33.726 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 -- 172.21.15.29:0/728372948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b301003e0 msgr2=0x7f1b30068cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:33.726 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b301003e0 0x7f1b30068cb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800c920 tx=0x7f1b1800cce0 comp rx=0 tx=0).stop 2023-11-27T15:18:33.726 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 -- 172.21.15.29:0/728372948 shutdown_connections 2023-11-27T15:18:33.726 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b1c0a1140 0x7f1b1c0a35f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:33.726 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b30100d70 0x7f1b300711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:33.726 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b300ff1e0 0x7f1b30068770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:33.726 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 --2- 172.21.15.29:0/728372948 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b301003e0 0x7f1b30068cb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:33.727 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 -- 172.21.15.29:0/728372948 >> 172.21.15.29:0/728372948 conn(0x7f1b300fa7f0 msgr2=0x7f1b300fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:33.727 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 -- 172.21.15.29:0/728372948 shutdown_connections 2023-11-27T15:18:33.727 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:18:33.720+0000 7f1b36a29700 1 -- 172.21.15.29:0/728372948 wait complete. 2023-11-27T15:18:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:33 smithi060 ceph-mon[131518]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:33.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:33.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:33.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:33.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:33 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:34.092 DEBUG:teuthology.orchestra.run.smithi029:osd.1> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.1.service 2023-11-27T15:18:34.094 INFO:tasks.cephadm:Deploying osd.2 on smithi060 with /dev/vg_nvme/lv_4... 2023-11-27T15:18:34.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-27T15:18:34.125 INFO:journalctl@ceph.osd.1.smithi029.stdout:-- Logs begin at Mon 2023-11-27 14:52:42 UTC. -- 2023-11-27T15:18:34.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:34 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:18:34.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:34 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:18:34.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:34 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:18:34.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:34 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:34.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:34 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:34.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:34 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:34.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:34 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:18:34.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:34 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:18:34.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:34 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:18:34.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:34 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:34.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:34 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:34.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:34 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:34.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:34 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:18:34.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:34 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:18:34.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:34 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:18:34.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:34 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:34.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:34 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:34.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:34 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:34.756 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:18:34.898 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:18:34 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-1[136259]: 2023-11-27T15:18:34.659+0000 7f7a692d4640 -1 osd.1 0 log_to_monitors true 2023-11-27T15:18:35.466 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:35 smithi029 ceph-mon[131166]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:35.466 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:35 smithi029 ceph-mon[131166]: from='osd.1 [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:18:35.466 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:35 smithi029 ceph-mon[131166]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:18:35.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:35 smithi060 ceph-mon[131518]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:35.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:35 smithi060 ceph-mon[131518]: from='osd.1 [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:18:35.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:35 smithi060 ceph-mon[131518]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:18:35.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:35 smithi008 ceph-mon[128788]: pgmap v58: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:35.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:35 smithi008 ceph-mon[128788]: from='osd.1 [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:18:35.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:35 smithi008 ceph-mon[128788]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:18:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-27T15:18:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-27T15:18:36.497 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2023-11-27T15:18:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2023-11-27T15:18:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00808453 s, 1.3 GB/s 2023-11-27T15:18:36.497 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2023-11-27T15:18:36.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:36 smithi008 ceph-mon[128788]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-27T15:18:36.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:36 smithi008 ceph-mon[128788]: osdmap e11: 2 total, 1 up, 2 in 2023-11-27T15:18:36.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:36 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:36.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:36 smithi008 ceph-mon[128788]: from='osd.1 [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:18:36.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:36 smithi008 ceph-mon[128788]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:18:36.647 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:18:36 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-1[136259]: 2023-11-27T15:18:36.340+0000 7f7a5e94b700 -1 osd.1 0 waiting for initial osdmap 2023-11-27T15:18:36.648 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:18:36 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-1[136259]: 2023-11-27T15:18:36.344+0000 7f7a5973e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:18:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:36 smithi029 ceph-mon[131166]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-27T15:18:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:36 smithi029 ceph-mon[131166]: osdmap e11: 2 total, 1 up, 2 in 2023-11-27T15:18:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:36 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:36 smithi029 ceph-mon[131166]: from='osd.1 [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:18:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:36 smithi029 ceph-mon[131166]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:18:36.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:36 smithi060 ceph-mon[131518]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-27T15:18:36.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:36 smithi060 ceph-mon[131518]: osdmap e11: 2 total, 1 up, 2 in 2023-11-27T15:18:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:36 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:36 smithi060 ceph-mon[131518]: from='osd.1 [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:18:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:36 smithi060 ceph-mon[131518]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:18:37.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_4 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: osdmap e12: 2 total, 1 up, 2 in 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi029 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi029 to 17173M 2023-11-27T15:18:37.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:37.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:37.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:18:37.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:18:37.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:37 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:18:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:18:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: osdmap e12: 2 total, 1 up, 2 in 2023-11-27T15:18:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi029 2023-11-27T15:18:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi029 to 17173M 2023-11-27T15:18:37.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:37.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:37.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:18:37.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:18:37.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:37 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: osdmap e12: 2 total, 1 up, 2 in 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi029 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi029 to 17173M 2023-11-27T15:18:37.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:37.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:37.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:18:37.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:37.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:18:37.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:37 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:18:37.766 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:18:38.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:38 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:18:38.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:38 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:18:38.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:38 smithi008 ceph-mon[128788]: osd.1 [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] boot 2023-11-27T15:18:38.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:38 smithi008 ceph-mon[128788]: osdmap e13: 2 total, 2 up, 2 in 2023-11-27T15:18:38.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:38.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:38 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:18:38.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:38 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:18:38.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:38 smithi029 ceph-mon[131166]: osd.1 [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] boot 2023-11-27T15:18:38.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:38 smithi029 ceph-mon[131166]: osdmap e13: 2 total, 2 up, 2 in 2023-11-27T15:18:38.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:38 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:18:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:38 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:18:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:38 smithi060 ceph-mon[131518]: osd.1 [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] boot 2023-11-27T15:18:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:38 smithi060 ceph-mon[131518]: osdmap e13: 2 total, 2 up, 2 in 2023-11-27T15:18:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:18:39.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.147+0000 7f2501c53700 1 -- 172.21.15.60:0/1081482130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc101ea0 msgr2=0x7f24fc108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:39.153 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.147+0000 7f2501c53700 1 --2- 172.21.15.60:0/1081482130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc101ea0 0x7f24fc108760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f24f400b0a0 tx=0x7f24f400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.147+0000 7f2501c53700 1 -- 172.21.15.60:0/1081482130 shutdown_connections 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.147+0000 7f2501c53700 1 --2- 172.21.15.60:0/1081482130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc101ea0 0x7f24fc108760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.147+0000 7f2501c53700 1 --2- 172.21.15.60:0/1081482130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24fc100310 0x7f24fc100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.147+0000 7f2501c53700 1 --2- 172.21.15.60:0/1081482130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24fc101510 0x7f24fc101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.147+0000 7f2501c53700 1 -- 172.21.15.60:0/1081482130 >> 172.21.15.60:0/1081482130 conn(0x7f24fc0fb960 msgr2=0x7f24fc0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.147+0000 7f2501c53700 1 -- 172.21.15.60:0/1081482130 shutdown_connections 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.147+0000 7f2501c53700 1 -- 172.21.15.60:0/1081482130 wait complete. 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f2501c53700 1 Processor -- start 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f2501c53700 1 -- start start 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f2501c53700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24fc100310 0x7f24fc1961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f2501c53700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc101510 0x7f24fc196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:39.154 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f2501c53700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24fc101ea0 0x7f24fc19d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f2501c53700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc19dd60 con 0x7f24fc101ea0 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f2501c53700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc19dea0 con 0x7f24fc100310 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f2501c53700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc19e030 con 0x7f24fc101510 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f24faffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc101510 0x7f24fc196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f24faffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc101510 0x7f24fc196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34794/0 (socket says 172.21.15.60:34794) 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f24faffd700 1 -- 172.21.15.60:0/2340865336 learned_addr learned my addr 172.21.15.60:0/2340865336 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.148+0000 7f24fb7fe700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24fc100310 0x7f24fc1961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f24faffd700 1 -- 172.21.15.60:0/2340865336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24fc100310 msgr2=0x7f24fc1961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f24faffd700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24fc100310 0x7f24fc1961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f24faffd700 1 -- 172.21.15.60:0/2340865336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24fc101ea0 msgr2=0x7f24fc19d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f24faffd700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24fc101ea0 0x7f24fc19d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f24faffd700 1 -- 172.21.15.60:0/2340865336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f4009d00 con 0x7f24fc101510 2023-11-27T15:18:39.155 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f24fb7fe700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24fc100310 0x7f24fc1961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:18:39.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f24faffd700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc101510 0x7f24fc196710 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00b560 tx=0x7f24ec00b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:39.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f24f8ff9700 1 -- 172.21.15.60:0/2340865336 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec011830 con 0x7f24fc101510 2023-11-27T15:18:39.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f24f8ff9700 1 -- 172.21.15.60:0/2340865336 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24ec011e50 con 0x7f24fc101510 2023-11-27T15:18:39.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f2501c53700 1 -- 172.21.15.60:0/2340865336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc10bd40 con 0x7f24fc101510 2023-11-27T15:18:39.156 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f24f8ff9700 1 -- 172.21.15.60:0/2340865336 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00f2e0 con 0x7f24fc101510 2023-11-27T15:18:39.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.149+0000 7f2501c53700 1 -- 172.21.15.60:0/2340865336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24fc10bbb0 con 0x7f24fc101510 2023-11-27T15:18:39.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.150+0000 7f2501c53700 1 -- 172.21.15.60:0/2340865336 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24fc107870 con 0x7f24fc101510 2023-11-27T15:18:39.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.151+0000 7f24f8ff9700 1 -- 172.21.15.60:0/2340865336 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f24ec011990 con 0x7f24fc101510 2023-11-27T15:18:39.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.152+0000 7f24f8ff9700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24e40a1140 0x7f24e40a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:39.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.152+0000 7f24f8ff9700 1 -- 172.21.15.60:0/2340865336 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f24ec0cb0c0 con 0x7f24fc101510 2023-11-27T15:18:39.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.153+0000 7f24fb7fe700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24e40a1140 0x7f24e40a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:39.157 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.153+0000 7f24fb7fe700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24e40a1140 0x7f24e40a35f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f24fc101370 tx=0x7f24f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:39.160 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.156+0000 7f24f8ff9700 1 -- 172.21.15.60:0/2340865336 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f24ec096060 con 0x7f24fc101510 2023-11-27T15:18:39.367 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:39.363+0000 7f2501c53700 1 -- 172.21.15.60:0/2340865336 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f24fc061c90 con 0x7f24e40a1140 2023-11-27T15:18:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:39 smithi008 ceph-mon[128788]: pgmap v63: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:39 smithi008 ceph-mon[128788]: osdmap e14: 2 total, 2 up, 2 in 2023-11-27T15:18:39.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:39 smithi029 ceph-mon[131166]: pgmap v63: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:39.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:39 smithi029 ceph-mon[131166]: osdmap e14: 2 total, 2 up, 2 in 2023-11-27T15:18:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:39 smithi060 ceph-mon[131518]: pgmap v63: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:18:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:39 smithi060 ceph-mon[131518]: osdmap e14: 2 total, 2 up, 2 in 2023-11-27T15:18:40.612 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:40 smithi060 ceph-mon[131518]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:40.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:40 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:40.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:40 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:40.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:40 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:40.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:40 smithi008 ceph-mon[128788]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:40.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:40 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:40.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:40 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:40.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:40 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:40.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:40 smithi029 ceph-mon[131166]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:40.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:40 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:40.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:40 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:40.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:40 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:41.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:41 smithi060 ceph-mon[131518]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:41.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:41 smithi008 ceph-mon[128788]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:41.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:41 smithi029 ceph-mon[131166]: pgmap v65: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:42.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:42 smithi029 ceph-mon[131166]: from='client.? 172.21.15.60:0/2258713358' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b"}]: dispatch 2023-11-27T15:18:42.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:42 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b"}]: dispatch 2023-11-27T15:18:42.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:42 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b"}]': finished 2023-11-27T15:18:42.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:42 smithi029 ceph-mon[131166]: osdmap e15: 3 total, 2 up, 3 in 2023-11-27T15:18:42.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:42 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:18:42.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:42 smithi060 ceph-mon[131518]: from='client.? 172.21.15.60:0/2258713358' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b"}]: dispatch 2023-11-27T15:18:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:42 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b"}]: dispatch 2023-11-27T15:18:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:42 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b"}]': finished 2023-11-27T15:18:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:42 smithi060 ceph-mon[131518]: osdmap e15: 3 total, 2 up, 3 in 2023-11-27T15:18:42.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:42 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:18:42.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:42 smithi008 ceph-mon[128788]: from='client.? 172.21.15.60:0/2258713358' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b"}]: dispatch 2023-11-27T15:18:42.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:42 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b"}]: dispatch 2023-11-27T15:18:42.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:42 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b"}]': finished 2023-11-27T15:18:42.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:42 smithi008 ceph-mon[128788]: osdmap e15: 3 total, 2 up, 3 in 2023-11-27T15:18:42.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:42 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:18:43.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:43 smithi029 ceph-mon[131166]: pgmap v67: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:43.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:43 smithi029 ceph-mon[131166]: from='client.? 172.21.15.60:0/1598499476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:43 smithi060 ceph-mon[131518]: pgmap v67: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:43 smithi060 ceph-mon[131518]: from='client.? 172.21.15.60:0/1598499476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:43.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:43 smithi008 ceph-mon[128788]: pgmap v67: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:43.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:43 smithi008 ceph-mon[128788]: from='client.? 172.21.15.60:0/1598499476' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:45.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:45 smithi029 ceph-mon[131166]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:45 smithi060 ceph-mon[131518]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:45.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:45 smithi008 ceph-mon[128788]: pgmap v68: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:47.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:47 smithi029 ceph-mon[131166]: pgmap v69: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:47.666 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:47 smithi060 ceph-mon[131518]: pgmap v69: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:47.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:47 smithi008 ceph-mon[128788]: pgmap v69: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:48.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:48 smithi060 ceph-mon[131518]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:49.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:48 smithi008 ceph-mon[128788]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:49.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:48 smithi029 ceph-mon[131166]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:50.028 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-27T15:18:50.028 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:50.028 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:49 smithi060 ceph-mon[131518]: Deploying daemon osd.2 on smithi060 2023-11-27T15:18:50.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-27T15:18:50.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:50.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:49 smithi008 ceph-mon[128788]: Deploying daemon osd.2 on smithi060 2023-11-27T15:18:50.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-27T15:18:50.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:50.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:49 smithi029 ceph-mon[131166]: Deploying daemon osd.2 on smithi060 2023-11-27T15:18:51.048 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:50 smithi060 ceph-mon[131518]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:51.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:50 smithi008 ceph-mon[128788]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:51.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:50 smithi029 ceph-mon[131166]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:53.589 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:53 smithi060 ceph-mon[131518]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:53.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:53 smithi008 ceph-mon[128788]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:53.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:53 smithi029 ceph-mon[131166]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:55.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:54 smithi008 ceph-mon[128788]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:55.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:54 smithi029 ceph-mon[131166]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:55.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:55.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:55.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:55.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:54 smithi060 ceph-mon[131518]: pgmap v73: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:57.139 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 2 on host 'smithi060' 2023-11-27T15:18:57.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.133+0000 7f24f8ff9700 1 -- 172.21.15.60:0/2340865336 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f24fc061c90 con 0x7f24e40a1140 2023-11-27T15:18:57.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.135+0000 7f2501c53700 1 -- 172.21.15.60:0/2340865336 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24e40a1140 msgr2=0x7f24e40a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:57.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.135+0000 7f2501c53700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24e40a1140 0x7f24e40a35f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f24fc101370 tx=0x7f24f0005f90 comp rx=0 tx=0).stop 2023-11-27T15:18:57.140 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.135+0000 7f2501c53700 1 -- 172.21.15.60:0/2340865336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc101510 msgr2=0x7f24fc196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:57.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.135+0000 7f2501c53700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc101510 0x7f24fc196710 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00b560 tx=0x7f24ec00b920 comp rx=0 tx=0).stop 2023-11-27T15:18:57.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.135+0000 7f2501c53700 1 -- 172.21.15.60:0/2340865336 shutdown_connections 2023-11-27T15:18:57.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.135+0000 7f2501c53700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24fc101ea0 0x7f24fc19d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:57.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.135+0000 7f2501c53700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f24fc100310 0x7f24fc1961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:57.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.135+0000 7f2501c53700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24e40a1140 0x7f24e40a35f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:57.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.135+0000 7f2501c53700 1 --2- 172.21.15.60:0/2340865336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f24fc101510 0x7f24fc196710 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:57.141 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.135+0000 7f2501c53700 1 -- 172.21.15.60:0/2340865336 >> 172.21.15.60:0/2340865336 conn(0x7f24fc0fb960 msgr2=0x7f24fc1049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:57.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.136+0000 7f2501c53700 1 -- 172.21.15.60:0/2340865336 shutdown_connections 2023-11-27T15:18:57.142 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:18:57.136+0000 7f2501c53700 1 -- 172.21.15.60:0/2340865336 wait complete. 2023-11-27T15:18:57.527 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.527 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:57.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:57.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:18:57.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:18:57.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:18:57.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:57.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:57.564 DEBUG:teuthology.orchestra.run.smithi060:osd.2> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.2.service 2023-11-27T15:18:57.567 INFO:tasks.cephadm:Deploying osd.3 on smithi008 with /dev/vg_nvme/lv_3... 2023-11-27T15:18:57.567 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-27T15:18:57.600 INFO:journalctl@ceph.osd.2.smithi060.stdout:-- Logs begin at Mon 2023-11-27 14:51:59 UTC. -- 2023-11-27T15:18:57.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:57.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:57.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:18:57.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:18:57.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:18:57.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:57.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:18:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:18:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:18:57.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:57.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:18:57.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:58.074 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:18:57 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-2[137357]: 2023-11-27T15:18:57.805+0000 7fc82dccb640 -1 osd.2 0 log_to_monitors true 2023-11-27T15:18:58.439 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:18:58.979 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:58 smithi008 ceph-mon[128788]: from='osd.2 [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:18:58.980 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:58 smithi008 ceph-mon[128788]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:18:58.980 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:58 smithi008 ceph-mon[128788]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:59.013 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:58 smithi060 ceph-mon[131518]: from='osd.2 [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:18:59.013 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:58 smithi060 ceph-mon[131518]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:18:59.013 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:58 smithi060 ceph-mon[131518]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:59.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:58 smithi029 ceph-mon[131166]: from='osd.2 [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:18:59.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:58 smithi029 ceph-mon[131166]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:18:59.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:58 smithi029 ceph-mon[131166]: pgmap v75: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:18:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:59 smithi008 ceph-mon[128788]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-27T15:18:59.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:59 smithi008 ceph-mon[128788]: osdmap e16: 3 total, 2 up, 3 in 2023-11-27T15:18:59.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:59 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:18:59.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:59 smithi008 ceph-mon[128788]: from='osd.2 [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:18:59.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:18:59 smithi008 ceph-mon[128788]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:19:00.005 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:18:59 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-2[137357]: 2023-11-27T15:18:59.743+0000 7fc823342700 -1 osd.2 0 waiting for initial osdmap 2023-11-27T15:19:00.005 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:18:59 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-2[137357]: 2023-11-27T15:18:59.747+0000 7fc81d934700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:19:00.005 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:59 smithi060 ceph-mon[131518]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-27T15:19:00.005 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:59 smithi060 ceph-mon[131518]: osdmap e16: 3 total, 2 up, 3 in 2023-11-27T15:19:00.005 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:59 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:00.006 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:59 smithi060 ceph-mon[131518]: from='osd.2 [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:19:00.006 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:18:59 smithi060 ceph-mon[131518]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:19:00.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:59 smithi029 ceph-mon[131166]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-27T15:19:00.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:59 smithi029 ceph-mon[131166]: osdmap e16: 3 total, 2 up, 3 in 2023-11-27T15:19:00.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:59 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:00.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:59 smithi029 ceph-mon[131166]: from='osd.2 [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:19:00.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:18:59 smithi029 ceph-mon[131166]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:19:00.311 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-27T15:19:00.311 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-27T15:19:00.311 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2023-11-27T15:19:00.311 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2023-11-27T15:19:00.312 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0423875 s, 247 MB/s 2023-11-27T15:19:00.312 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2023-11-27T15:19:00.824 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi008:vg_nvme/lv_3 2023-11-27T15:19:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:19:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: osdmap e17: 3 total, 2 up, 3 in 2023-11-27T15:19:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi060 2023-11-27T15:19:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:01.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi060 to 17173M 2023-11-27T15:19:01.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:01.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:01.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:19:01.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:19:01.077 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:19:01.077 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:00 smithi060 ceph-mon[131518]: pgmap v78: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:19:01.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:19:01.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: osdmap e17: 3 total, 2 up, 3 in 2023-11-27T15:19:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi060 2023-11-27T15:19:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi060 to 17173M 2023-11-27T15:19:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:01.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:01.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:19:01.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:19:01.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:19:01.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:00 smithi008 ceph-mon[128788]: pgmap v78: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:19:01.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:19:01.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: osdmap e17: 3 total, 2 up, 3 in 2023-11-27T15:19:01.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:01.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:01.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi060 2023-11-27T15:19:01.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:01.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi060 to 17173M 2023-11-27T15:19:01.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:01.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:01.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:19:01.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:01.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:19:01.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:19:01.150 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:00 smithi029 ceph-mon[131166]: pgmap v78: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:19:01.726 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:19:02.030 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:01 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:19:02.030 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:01 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:19:02.030 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:01 smithi008 ceph-mon[128788]: osd.2 [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] boot 2023-11-27T15:19:02.031 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:01 smithi008 ceph-mon[128788]: osdmap e18: 3 total, 3 up, 3 in 2023-11-27T15:19:02.031 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:02.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:01 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:19:02.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:01 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:19:02.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:01 smithi060 ceph-mon[131518]: osd.2 [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] boot 2023-11-27T15:19:02.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:01 smithi060 ceph-mon[131518]: osdmap e18: 3 total, 3 up, 3 in 2023-11-27T15:19:02.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:02.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:01 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:19:02.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:01 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:19:02.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:01 smithi029 ceph-mon[131166]: osd.2 [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] boot 2023-11-27T15:19:02.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:01 smithi029 ceph-mon[131166]: osdmap e18: 3 total, 3 up, 3 in 2023-11-27T15:19:02.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:19:03.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:02 smithi029 ceph-mon[131166]: osdmap e19: 3 total, 3 up, 3 in 2023-11-27T15:19:03.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.250+0000 7faf1aff3700 1 -- 172.21.15.8:0/3811574929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf141002f0 msgr2=0x7faf14100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:03.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.250+0000 7faf1aff3700 1 --2- 172.21.15.8:0/3811574929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf141002f0 0x7faf14100700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faf08009990 tx=0x7faf08009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.250+0000 7faf1aff3700 1 -- 172.21.15.8:0/3811574929 shutdown_connections 2023-11-27T15:19:03.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.250+0000 7faf1aff3700 1 --2- 172.21.15.8:0/3811574929 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf14101e80 0x7faf14108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.250+0000 7faf1aff3700 1 --2- 172.21.15.8:0/3811574929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf141002f0 0x7faf14100700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.250+0000 7faf1aff3700 1 --2- 172.21.15.8:0/3811574929 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf141014f0 0x7faf14101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.250+0000 7faf1aff3700 1 -- 172.21.15.8:0/3811574929 >> 172.21.15.8:0/3811574929 conn(0x7faf140fb900 msgr2=0x7faf140fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:03.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.250+0000 7faf1aff3700 1 -- 172.21.15.8:0/3811574929 shutdown_connections 2023-11-27T15:19:03.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.250+0000 7faf1aff3700 1 -- 172.21.15.8:0/3811574929 wait complete. 2023-11-27T15:19:03.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.252+0000 7faf1aff3700 1 Processor -- start 2023-11-27T15:19:03.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.252+0000 7faf1aff3700 1 -- start start 2023-11-27T15:19:03.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.252+0000 7faf1aff3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf141002f0 0x7faf141982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:03.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.252+0000 7faf1aff3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf141014f0 0x7faf141987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:03.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.252+0000 7faf1aff3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf14101e80 0x7faf1419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:03.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.252+0000 7faf1aff3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1419fe30 con 0x7faf141002f0 2023-11-27T15:19:03.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.252+0000 7faf1aff3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1419ff70 con 0x7faf14101e80 2023-11-27T15:19:03.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.252+0000 7faf1aff3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf141a0100 con 0x7faf141014f0 2023-11-27T15:19:03.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf18d8f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf141002f0 0x7faf141982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:03.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf18d8f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf141002f0 0x7faf141982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39808/0 (socket says 172.21.15.8:39808) 2023-11-27T15:19:03.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf18d8f700 1 -- 172.21.15.8:0/3691699179 learned_addr learned my addr 172.21.15.8:0/3691699179 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:19:03.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf19590700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf14101e80 0x7faf1419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:03.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf13fff700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf141014f0 0x7faf141987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:03.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf18d8f700 1 -- 172.21.15.8:0/3691699179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf141014f0 msgr2=0x7faf141987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:03.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf18d8f700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf141014f0 0x7faf141987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf18d8f700 1 -- 172.21.15.8:0/3691699179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf14101e80 msgr2=0x7faf1419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:03.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf18d8f700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf14101e80 0x7faf1419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:03.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf18d8f700 1 -- 172.21.15.8:0/3691699179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf080095f0 con 0x7faf141002f0 2023-11-27T15:19:03.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.253+0000 7faf18d8f700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf141002f0 0x7faf141982a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faf08005740 tx=0x7faf08005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:03.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.254+0000 7faf11ffb700 1 -- 172.21.15.8:0/3691699179 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0801d070 con 0x7faf141002f0 2023-11-27T15:19:03.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.254+0000 7faf11ffb700 1 -- 172.21.15.8:0/3691699179 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf08021410 con 0x7faf141002f0 2023-11-27T15:19:03.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.254+0000 7faf1aff3700 1 -- 172.21.15.8:0/3691699179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1410bcb0 con 0x7faf141002f0 2023-11-27T15:19:03.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.254+0000 7faf11ffb700 1 -- 172.21.15.8:0/3691699179 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08017650 con 0x7faf141002f0 2023-11-27T15:19:03.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.254+0000 7faf1aff3700 1 -- 172.21.15.8:0/3691699179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1410bb20 con 0x7faf141002f0 2023-11-27T15:19:03.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.255+0000 7faf1aff3700 1 -- 172.21.15.8:0/3691699179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf14192a10 con 0x7faf141002f0 2023-11-27T15:19:03.261 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.257+0000 7faf11ffb700 1 -- 172.21.15.8:0/3691699179 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7faf08021580 con 0x7faf141002f0 2023-11-27T15:19:03.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.258+0000 7faf11ffb700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faf040a1140 0x7faf040a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:03.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.258+0000 7faf11ffb700 1 -- 172.21.15.8:0/3691699179 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3226+0+0 (secure 0 0 0) 0x7faf080ccf60 con 0x7faf141002f0 2023-11-27T15:19:03.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.259+0000 7faf13fff700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faf040a1140 0x7faf040a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:03.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.259+0000 7faf13fff700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faf040a1140 0x7faf040a35f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faf00009c60 tx=0x7faf00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:03.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.261+0000 7faf11ffb700 1 -- 172.21.15.8:0/3691699179 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7faf08097b60 con 0x7faf141002f0 2023-11-27T15:19:03.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:02 smithi060 ceph-mon[131518]: osdmap e19: 3 total, 3 up, 3 in 2023-11-27T15:19:03.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:02 smithi008 ceph-mon[128788]: osdmap e19: 3 total, 3 up, 3 in 2023-11-27T15:19:03.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:03.479+0000 7faf1aff3700 1 -- 172.21.15.8:0/3691699179 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7faf14061c60 con 0x7faf040a1140 2023-11-27T15:19:04.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:03 smithi029 ceph-mon[131166]: pgmap v81: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:04.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:19:04.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-27T15:19:04.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:03 smithi029 ceph-mon[131166]: osdmap e20: 3 total, 3 up, 3 in 2023-11-27T15:19:04.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:19:04.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:03 smithi029 ceph-mon[131166]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:04.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:04.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:04.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:04.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:03 smithi060 ceph-mon[131518]: pgmap v81: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:04.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:19:04.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-27T15:19:04.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:03 smithi060 ceph-mon[131518]: osdmap e20: 3 total, 3 up, 3 in 2023-11-27T15:19:04.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:19:04.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:03 smithi060 ceph-mon[131518]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:04.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:04.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:04.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:04.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:03 smithi008 ceph-mon[128788]: pgmap v81: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:04.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:19:04.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-27T15:19:04.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:03 smithi008 ceph-mon[128788]: osdmap e20: 3 total, 3 up, 3 in 2023-11-27T15:19:04.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:19:04.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:03 smithi008 ceph-mon[128788]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:04.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:04.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:04.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:04.863 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:19:04 smithi008 sudo[144334]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:19:04.864 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:19:04 smithi008 sudo[144334]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:04.864 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:19:04 smithi008 sudo[144334]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:04.864 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:19:04 smithi008 sudo[144334]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:04.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:04.928+0000 7faf11ffb700 1 -- 172.21.15.8:0/3691699179 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7faf08097f30 con 0x7faf141002f0 2023-11-27T15:19:05.303 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:04 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:19:05.303 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:04 smithi008 ceph-mon[128788]: osdmap e21: 3 total, 3 up, 3 in 2023-11-27T15:19:05.303 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:04 smithi008 ceph-mon[128788]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:05.303 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:19:05 smithi008 sudo[144386]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:19:05.304 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:19:05 smithi008 sudo[144386]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:05.304 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:19:05 smithi008 sudo[144386]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:05.304 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:19:05 smithi008 sudo[144386]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:05.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:04 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:19:05.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:04 smithi060 ceph-mon[131518]: osdmap e21: 3 total, 3 up, 3 in 2023-11-27T15:19:05.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:04 smithi060 ceph-mon[131518]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:05.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:04 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:19:05.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:04 smithi029 ceph-mon[131166]: osdmap e21: 3 total, 3 up, 3 in 2023-11-27T15:19:05.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:04 smithi029 ceph-mon[131166]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:05.898 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:19:05 smithi029 sudo[138577]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:19:05.898 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:19:05 smithi029 sudo[138577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:05.898 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:19:05 smithi029 sudo[138577]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:05.898 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:19:05 smithi029 sudo[138577]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:06.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:05 smithi060 ceph-mon[131518]: osdmap e22: 3 total, 3 up, 3 in 2023-11-27T15:19:06.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:05 smithi060 ceph-mon[131518]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-27T15:19:06.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:06.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:05 smithi008 ceph-mon[128788]: osdmap e22: 3 total, 3 up, 3 in 2023-11-27T15:19:06.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:05 smithi008 ceph-mon[128788]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-27T15:19:06.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:06.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:05 smithi029 ceph-mon[131166]: osdmap e22: 3 total, 3 up, 3 in 2023-11-27T15:19:06.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:05 smithi029 ceph-mon[131166]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-27T15:19:06.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:06.398 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:19:06 smithi029 sudo[138580]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:19:06.398 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:19:06 smithi029 sudo[138580]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:06.398 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:19:06 smithi029 sudo[138580]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:06.398 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:19:06 smithi029 sudo[138580]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:06.825 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:19:06 smithi060 sudo[139600]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:19:06.825 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:19:06 smithi060 sudo[139600]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:06.825 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:19:06 smithi060 sudo[139600]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:06.825 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:19:06 smithi060 sudo[139600]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:07.325 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:19:07 smithi060 sudo[139603]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:19:07.325 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:19:07 smithi060 sudo[139603]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:07.325 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:19:07 smithi060 sudo[139603]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:07.325 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:19:07 smithi060 sudo[139603]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:07 smithi008 sudo[144520]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:19:07.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:07 smithi008 sudo[144520]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:07.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:07 smithi008 sudo[144520]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:07.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:07 smithi008 sudo[144520]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:08.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:08.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 ceph-mon[131518]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:08.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2037632240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b26fe4c-ca46-4542-9906-25396da80c99"}]: dispatch 2023-11-27T15:19:08.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b26fe4c-ca46-4542-9906-25396da80c99"}]: dispatch 2023-11-27T15:19:08.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b26fe4c-ca46-4542-9906-25396da80c99"}]': finished 2023-11-27T15:19:08.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 ceph-mon[131518]: osdmap e23: 4 total, 3 up, 4 in 2023-11-27T15:19:08.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:08.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:08.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 ceph-mon[131518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:19:08.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 sudo[144708]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:19:08.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 sudo[144708]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:08.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 sudo[144708]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:08.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 sudo[144708]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:08.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:08.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 ceph-mon[128788]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:08.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2037632240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b26fe4c-ca46-4542-9906-25396da80c99"}]: dispatch 2023-11-27T15:19:08.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b26fe4c-ca46-4542-9906-25396da80c99"}]: dispatch 2023-11-27T15:19:08.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b26fe4c-ca46-4542-9906-25396da80c99"}]': finished 2023-11-27T15:19:08.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 ceph-mon[128788]: osdmap e23: 4 total, 3 up, 4 in 2023-11-27T15:19:08.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:08.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:08.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:08 smithi008 ceph-mon[128788]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:19:08.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:08 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:08.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:08 smithi029 ceph-mon[131166]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:08.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:08 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2037632240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b26fe4c-ca46-4542-9906-25396da80c99"}]: dispatch 2023-11-27T15:19:08.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:08 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7b26fe4c-ca46-4542-9906-25396da80c99"}]: dispatch 2023-11-27T15:19:08.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:08 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7b26fe4c-ca46-4542-9906-25396da80c99"}]': finished 2023-11-27T15:19:08.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:08 smithi029 ceph-mon[131166]: osdmap e23: 4 total, 3 up, 4 in 2023-11-27T15:19:08.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:08 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:08.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:08 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:08.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:08 smithi029 ceph-mon[131166]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:19:09.058 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 sudo[139606]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:19:09.058 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 sudo[139606]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:09.058 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 sudo[139606]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:09.058 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:08 smithi060 sudo[139606]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:09.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 sudo[139609]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:19:09.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 sudo[139609]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:09.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 sudo[139609]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:09.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 sudo[139609]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 sudo[138583]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:19:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 sudo[138583]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 sudo[138583]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 sudo[138583]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:10.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1425395248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:10.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:19:10.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:19:10.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:19:10.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:19:10.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:19:10.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:19:10.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:19:10.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:19:10.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 18 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:10.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:19:10.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:19:10.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:19:10.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:19:10.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1425395248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:10.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:19:10.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:19:10.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:19:10.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:19:10.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:19:10.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:19:10.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:19:10.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:19:10.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 18 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:10.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:19:10.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:19:10.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:19:10.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:19:10.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 sudo[138586]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:19:10.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 sudo[138586]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:19:10.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 sudo[138586]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:19:10.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 sudo[138586]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:19:10.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1425395248' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:10.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:19:10.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:19:10.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:19:10.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:19:10.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:19:10.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:19:10.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:19:10.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:19:10.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 18 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:10.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:19:10.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:19:10.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:19:10.399 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:19:11.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:10 smithi060 ceph-mon[131518]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:19:11.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:10 smithi060 ceph-mon[131518]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:19:11.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:10 smithi060 ceph-mon[131518]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:10 smithi008 ceph-mon[128788]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:19:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:10 smithi008 ceph-mon[128788]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:19:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:10 smithi008 ceph-mon[128788]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:11.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:10 smithi029 ceph-mon[131166]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:19:11.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:10 smithi029 ceph-mon[131166]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:19:11.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:10 smithi029 ceph-mon[131166]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:13.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:13 smithi008 ceph-mon[128788]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:13.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:13 smithi029 ceph-mon[131166]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:13 smithi060 ceph-mon[131518]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:15.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:15 smithi008 ceph-mon[128788]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-11-27T15:19:15.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-27T15:19:15.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:15.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:15 smithi008 ceph-mon[128788]: Deploying daemon osd.3 on smithi008 2023-11-27T15:19:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:15 smithi029 ceph-mon[131166]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-11-27T15:19:15.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-27T15:19:15.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:15.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:15 smithi029 ceph-mon[131166]: Deploying daemon osd.3 on smithi008 2023-11-27T15:19:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:15 smithi060 ceph-mon[131518]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-11-27T15:19:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-27T15:19:15.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:15.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:15 smithi060 ceph-mon[131518]: Deploying daemon osd.3 on smithi008 2023-11-27T15:19:17.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:17 smithi008 ceph-mon[128788]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-11-27T15:19:17.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:17 smithi029 ceph-mon[131166]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-11-27T15:19:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:17 smithi060 ceph-mon[131518]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-11-27T15:19:19.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:19 smithi060 ceph-mon[131518]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-11-27T15:19:19.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:19 smithi008 ceph-mon[128788]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-11-27T15:19:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:19 smithi029 ceph-mon[131166]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-11-27T15:19:21.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:21 smithi008 ceph-mon[128788]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-27T15:19:21.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:21 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:21.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:21 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:21.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:21 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:21.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:21 smithi029 ceph-mon[131166]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-27T15:19:21.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:21 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:21.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:21 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:21.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:21 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:21 smithi060 ceph-mon[131518]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-27T15:19:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:21 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:21 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:21 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:23.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:23 smithi008 ceph-mon[128788]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:19:23.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:23 smithi029 ceph-mon[131166]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:19:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:23 smithi060 ceph-mon[131518]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:19:24.551 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 3 on host 'smithi008' 2023-11-27T15:19:24.551 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.546+0000 7faf11ffb700 1 -- 172.21.15.8:0/3691699179 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faf14061c60 con 0x7faf040a1140 2023-11-27T15:19:24.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 -- 172.21.15.8:0/3691699179 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faf040a1140 msgr2=0x7faf040a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:24.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faf040a1140 0x7faf040a35f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faf00009c60 tx=0x7faf00009260 comp rx=0 tx=0).stop 2023-11-27T15:19:24.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 -- 172.21.15.8:0/3691699179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf141002f0 msgr2=0x7faf141982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:24.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf141002f0 0x7faf141982a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faf08005740 tx=0x7faf08005770 comp rx=0 tx=0).stop 2023-11-27T15:19:24.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 -- 172.21.15.8:0/3691699179 shutdown_connections 2023-11-27T15:19:24.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faf14101e80 0x7faf1419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:24.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf141002f0 0x7faf141982a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:24.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faf040a1140 0x7faf040a35f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:24.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 --2- 172.21.15.8:0/3691699179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faf141014f0 0x7faf141987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:24.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 -- 172.21.15.8:0/3691699179 >> 172.21.15.8:0/3691699179 conn(0x7faf140fb900 msgr2=0x7faf14106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:24.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 -- 172.21.15.8:0/3691699179 shutdown_connections 2023-11-27T15:19:24.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:19:24.550+0000 7faf1aff3700 1 -- 172.21.15.8:0/3691699179 wait complete. 2023-11-27T15:19:24.973 DEBUG:teuthology.orchestra.run.smithi008:osd.3> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.3.service 2023-11-27T15:19:24.977 INFO:tasks.cephadm:Deploying osd.4 on smithi029 with /dev/vg_nvme/lv_3... 2023-11-27T15:19:24.978 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-27T15:19:25.072 INFO:journalctl@ceph.osd.3.smithi008.stdout:-- Logs begin at Mon 2023-11-27 14:56:10 UTC. -- 2023-11-27T15:19:25.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:25.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:24 smithi008 ceph-mon[128788]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-27T15:19:25.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:25.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:24 smithi029 ceph-mon[131166]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-27T15:19:25.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:25.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:25.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:19:25.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:19:25.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:19:25.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:19:25.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:25.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:25.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:24 smithi060 ceph-mon[131518]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-27T15:19:25.651 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:19:26.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:26 smithi060 ceph-mon[131518]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-27T15:19:26.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:26 smithi060 ceph-mon[131518]: osdmap e24: 4 total, 3 up, 4 in 2023-11-27T15:19:26.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:26 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:26.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:26 smithi060 ceph-mon[131518]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:19:26.828 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:26 smithi008 ceph-mon[128788]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-27T15:19:26.828 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:26 smithi008 ceph-mon[128788]: osdmap e24: 4 total, 3 up, 4 in 2023-11-27T15:19:26.828 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:26 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:26.828 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:26 smithi008 ceph-mon[128788]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:19:26.828 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:19:26 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-3[146762]: 2023-11-27T15:19:26.513+0000 7f5118c07700 -1 osd.3 0 waiting for initial osdmap 2023-11-27T15:19:26.828 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:19:26 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-3[146762]: 2023-11-27T15:19:26.521+0000 7f51129f8700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:19:26.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:26 smithi029 ceph-mon[131166]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-27T15:19:26.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:26 smithi029 ceph-mon[131166]: osdmap e24: 4 total, 3 up, 4 in 2023-11-27T15:19:26.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:26 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:26.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:26 smithi029 ceph-mon[131166]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:19:27.136 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-27T15:19:27.136 INFO:teuthology.orchestra.run.smithi029.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-27T15:19:27.136 INFO:teuthology.orchestra.run.smithi029.stderr: stderr: 10+0 records in 2023-11-27T15:19:27.136 INFO:teuthology.orchestra.run.smithi029.stderr:10+0 records out 2023-11-27T15:19:27.137 INFO:teuthology.orchestra.run.smithi029.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00817442 s, 1.3 GB/s 2023-11-27T15:19:27.137 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping successful for: 2023-11-27T15:19:27.611 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi029:vg_nvme/lv_3 2023-11-27T15:19:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:27 smithi060 ceph-mon[131518]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:27 smithi060 ceph-mon[131518]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:19:27.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:27 smithi060 ceph-mon[131518]: osdmap e25: 4 total, 3 up, 4 in 2023-11-27T15:19:27.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:27 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:27.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:27 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:27 smithi008 ceph-mon[128788]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:27 smithi008 ceph-mon[128788]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:19:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:27 smithi008 ceph-mon[128788]: osdmap e25: 4 total, 3 up, 4 in 2023-11-27T15:19:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:27 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:27.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:27 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:27 smithi029 ceph-mon[131166]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 20 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:19:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:27 smithi029 ceph-mon[131166]: from='osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:19:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:27 smithi029 ceph-mon[131166]: osdmap e25: 4 total, 3 up, 4 in 2023-11-27T15:19:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:27 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:27.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:27 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:28.267 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:19:28.789 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:19:28.789 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:19:28.789 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] boot 2023-11-27T15:19:28.789 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: osdmap e26: 4 total, 4 up, 4 in 2023-11-27T15:19:28.789 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:28.789 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.789 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.790 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:28.790 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:28.790 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.790 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:28.790 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:28.790 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.790 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:28 smithi029 ceph-mon[131166]: osdmap e27: 4 total, 4 up, 4 in 2023-11-27T15:19:28.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] boot 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: osdmap e26: 4 total, 4 up, 4 in 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:28 smithi060 ceph-mon[131518]: osdmap e27: 4 total, 4 up, 4 in 2023-11-27T15:19:28.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: osd.3 [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] boot 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: osdmap e26: 4 total, 4 up, 4 in 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:28.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:28 smithi008 ceph-mon[128788]: osdmap e27: 4 total, 4 up, 4 in 2023-11-27T15:19:29.448 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.444+0000 7f6ff0470700 1 -- 172.21.15.29:0/3360641559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fe81003e0 msgr2=0x7f6fe8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:29.449 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.444+0000 7f6ff0470700 1 --2- 172.21.15.29:0/3360641559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fe81003e0 0x7f6fe8100830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009990 tx=0x7f6fd8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:29.449 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.444+0000 7f6ff0470700 1 -- 172.21.15.29:0/3360641559 shutdown_connections 2023-11-27T15:19:29.449 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.444+0000 7f6ff0470700 1 --2- 172.21.15.29:0/3360641559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe8100d70 0x7f6fe8107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:29.449 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.444+0000 7f6ff0470700 1 --2- 172.21.15.29:0/3360641559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fe80ff1e0 0x7f6fe80ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:29.449 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.444+0000 7f6ff0470700 1 --2- 172.21.15.29:0/3360641559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fe81003e0 0x7f6fe8100830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:29.449 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.444+0000 7f6ff0470700 1 -- 172.21.15.29:0/3360641559 >> 172.21.15.29:0/3360641559 conn(0x7f6fe80fa7f0 msgr2=0x7f6fe80fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:29.449 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.444+0000 7f6ff0470700 1 -- 172.21.15.29:0/3360641559 shutdown_connections 2023-11-27T15:19:29.449 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.444+0000 7f6ff0470700 1 -- 172.21.15.29:0/3360641559 wait complete. 2023-11-27T15:19:29.451 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6ff0470700 1 Processor -- start 2023-11-27T15:19:29.452 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6ff0470700 1 -- start start 2023-11-27T15:19:29.452 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6ff0470700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fe80ff1e0 0x7f6fe8069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:29.452 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6ff0470700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe81003e0 0x7f6fe819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:29.452 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6ff0470700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fe8100d70 0x7f6fe819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:29.452 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6ff0470700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe819dca0 con 0x7f6fe8100d70 2023-11-27T15:19:29.452 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6ff0470700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe819de10 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.452 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6ff0470700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe819dfd0 con 0x7f6fe81003e0 2023-11-27T15:19:29.452 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6fee20c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fe80ff1e0 0x7f6fe8069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:29.452 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6fee20c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fe80ff1e0 0x7f6fe8069130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35070/0 (socket says 172.21.15.29:35070) 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6fee20c700 1 -- 172.21.15.29:0/1462385924 learned_addr learned my addr 172.21.15.29:0/1462385924 (peer_addr_for_me v2:172.21.15.29:0/0) 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6feea0d700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fe8100d70 0x7f6fe819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6feda0b700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe81003e0 0x7f6fe819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6fee20c700 1 -- 172.21.15.29:0/1462385924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe81003e0 msgr2=0x7f6fe819aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6fee20c700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe81003e0 0x7f6fe819aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6fee20c700 1 -- 172.21.15.29:0/1462385924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fe8100d70 msgr2=0x7f6fe819d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6fee20c700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fe8100d70 0x7f6fe819d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.445+0000 7f6fee20c700 1 -- 172.21.15.29:0/1462385924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd80095f0 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.446+0000 7f6feda0b700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe81003e0 0x7f6fe819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.446+0000 7f6fee20c700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fe80ff1e0 0x7f6fe8069130 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400b500 tx=0x7f6fe400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:29.453 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.446+0000 7f6fdf7fe700 1 -- 172.21.15.29:0/1462385924 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4011830 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.454 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.446+0000 7f6fdf7fe700 1 -- 172.21.15.29:0/1462385924 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fe4011e50 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.454 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.446+0000 7f6ff0470700 1 -- 172.21.15.29:0/1462385924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe810abd0 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.454 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.446+0000 7f6fdf7fe700 1 -- 172.21.15.29:0/1462385924 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400f2e0 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.454 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.446+0000 7f6ff0470700 1 -- 172.21.15.29:0/1462385924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe810aa40 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.454 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.447+0000 7f6ff0470700 1 -- 172.21.15.29:0/1462385924 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe8106880 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.455 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.448+0000 7f6fdf7fe700 1 -- 172.21.15.29:0/1462385924 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6fe4010580 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.455 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.449+0000 7f6fdf7fe700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fd40a1250 0x7f6fd40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:29.455 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.449+0000 7f6fdf7fe700 1 -- 172.21.15.29:0/1462385924 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f6fe40cb7d0 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.455 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.449+0000 7f6feda0b700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fd40a1250 0x7f6fd40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:29.455 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.449+0000 7f6feda0b700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fd40a1250 0x7f6fd40a3700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800b5c0 tx=0x7f6fd8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:29.456 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.452+0000 7f6fdf7fe700 1 -- 172.21.15.29:0/1462385924 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6fe4096080 con 0x7f6fe80ff1e0 2023-11-27T15:19:29.662 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:29.658+0000 7f6ff0470700 1 -- 172.21.15.29:0/1462385924 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fe8061b90 con 0x7f6fd40a1250 2023-11-27T15:19:29.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:29 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi008 2023-11-27T15:19:29.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:29 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi008 to 8638M 2023-11-27T15:19:29.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:29 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:19:29.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:29 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:19:29.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:29 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:19:29.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:29 smithi060 ceph-mon[131518]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:29.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:29 smithi060 ceph-mon[131518]: osdmap e28: 4 total, 4 up, 4 in 2023-11-27T15:19:29.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:29 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi008 2023-11-27T15:19:29.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:29 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi008 to 8638M 2023-11-27T15:19:29.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:29 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:19:29.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:29 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:19:29.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:29 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:19:29.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:29 smithi008 ceph-mon[128788]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:29.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:29 smithi008 ceph-mon[128788]: osdmap e28: 4 total, 4 up, 4 in 2023-11-27T15:19:29.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:29 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi008 2023-11-27T15:19:29.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:29 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi008 to 8638M 2023-11-27T15:19:29.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:29 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:19:29.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:29 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:19:29.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:29 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:19:29.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:29 smithi029 ceph-mon[131166]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:29.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:29 smithi029 ceph-mon[131166]: osdmap e28: 4 total, 4 up, 4 in 2023-11-27T15:19:30.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:30 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:30.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:30 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:30.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:30 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:30.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:30 smithi060 ceph-mon[131518]: osdmap e29: 4 total, 4 up, 4 in 2023-11-27T15:19:30.861 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:30 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:30.862 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:30 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:30.862 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:30 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:30.862 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:30 smithi029 ceph-mon[131166]: osdmap e29: 4 total, 4 up, 4 in 2023-11-27T15:19:30.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:30 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:30.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:30 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:30.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:30 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:30.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:30 smithi008 ceph-mon[128788]: osdmap e29: 4 total, 4 up, 4 in 2023-11-27T15:19:31.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:31 smithi029 ceph-mon[131166]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:31.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:31 smithi029 ceph-mon[131166]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:31.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:31 smithi029 ceph-mon[131166]: osdmap e30: 4 total, 4 up, 4 in 2023-11-27T15:19:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:31 smithi060 ceph-mon[131518]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:31.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:31 smithi060 ceph-mon[131518]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:31.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:31 smithi060 ceph-mon[131518]: osdmap e30: 4 total, 4 up, 4 in 2023-11-27T15:19:31.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:31 smithi008 ceph-mon[128788]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:31.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:31 smithi008 ceph-mon[128788]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:31.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:31 smithi008 ceph-mon[128788]: osdmap e30: 4 total, 4 up, 4 in 2023-11-27T15:19:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:32 smithi060 ceph-mon[131518]: from='client.? 172.21.15.29:0/4156274025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ced007-63e0-4416-a2b1-bebc48d33c03"}]: dispatch 2023-11-27T15:19:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:32 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ced007-63e0-4416-a2b1-bebc48d33c03"}]: dispatch 2023-11-27T15:19:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:32 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77ced007-63e0-4416-a2b1-bebc48d33c03"}]': finished 2023-11-27T15:19:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:32 smithi060 ceph-mon[131518]: osdmap e31: 5 total, 4 up, 5 in 2023-11-27T15:19:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:32 smithi008 ceph-mon[128788]: from='client.? 172.21.15.29:0/4156274025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ced007-63e0-4416-a2b1-bebc48d33c03"}]: dispatch 2023-11-27T15:19:32.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:32 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ced007-63e0-4416-a2b1-bebc48d33c03"}]: dispatch 2023-11-27T15:19:32.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:32 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77ced007-63e0-4416-a2b1-bebc48d33c03"}]': finished 2023-11-27T15:19:32.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:32 smithi008 ceph-mon[128788]: osdmap e31: 5 total, 4 up, 5 in 2023-11-27T15:19:32.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:32 smithi029 ceph-mon[131166]: from='client.? 172.21.15.29:0/4156274025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ced007-63e0-4416-a2b1-bebc48d33c03"}]: dispatch 2023-11-27T15:19:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:32 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77ced007-63e0-4416-a2b1-bebc48d33c03"}]: dispatch 2023-11-27T15:19:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:32 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77ced007-63e0-4416-a2b1-bebc48d33c03"}]': finished 2023-11-27T15:19:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:32 smithi029 ceph-mon[131166]: osdmap e31: 5 total, 4 up, 5 in 2023-11-27T15:19:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:33 smithi060 ceph-mon[131518]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:33.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:33 smithi060 ceph-mon[131518]: from='client.? 172.21.15.29:0/1617976590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:33.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:33 smithi008 ceph-mon[128788]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:33.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:33 smithi008 ceph-mon[128788]: from='client.? 172.21.15.29:0/1617976590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:33 smithi029 ceph-mon[131166]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:33.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:33 smithi029 ceph-mon[131166]: from='client.? 172.21.15.29:0/1617976590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:35 smithi060 ceph-mon[131518]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-11-27T15:19:35.850 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:35 smithi029 ceph-mon[131166]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-11-27T15:19:35.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:35 smithi008 ceph-mon[128788]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-11-27T15:19:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:37 smithi060 ceph-mon[131518]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-11-27T15:19:37.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:37 smithi008 ceph-mon[128788]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-11-27T15:19:37.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:37 smithi029 ceph-mon[131166]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2023-11-27T15:19:39.428 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:39 smithi029 ceph-mon[131166]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:19:39.428 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-27T15:19:39.428 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:39.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:39 smithi060 ceph-mon[131518]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:19:39.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-27T15:19:39.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:39 smithi008 ceph-mon[128788]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:19:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-27T15:19:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:40.527 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:40 smithi029 ceph-mon[131166]: Deploying daemon osd.4 on smithi029 2023-11-27T15:19:40.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:40 smithi060 ceph-mon[131518]: Deploying daemon osd.4 on smithi029 2023-11-27T15:19:40.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:40 smithi008 ceph-mon[128788]: Deploying daemon osd.4 on smithi029 2023-11-27T15:19:41.562 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:41 smithi029 ceph-mon[131166]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-11-27T15:19:41.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:41 smithi060 ceph-mon[131518]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-11-27T15:19:41.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:41 smithi008 ceph-mon[128788]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-11-27T15:19:43.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:43 smithi008 ceph-mon[128788]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-11-27T15:19:43.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:43 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:43.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:43 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:43.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:43 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:43.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:43 smithi029 ceph-mon[131166]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-11-27T15:19:43.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:43 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:43.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:43 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:43.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:43 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:43 smithi060 ceph-mon[131518]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2023-11-27T15:19:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:43 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:43 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:43.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:43 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:45 smithi008 ceph-mon[128788]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-27T15:19:45.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:45 smithi029 ceph-mon[131166]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-27T15:19:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:45 smithi060 ceph-mon[131518]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-27T15:19:45.876 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.872+0000 7f6fdf7fe700 1 -- 172.21.15.29:0/1462385924 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6fe8061b90 con 0x7f6fd40a1250 2023-11-27T15:19:45.876 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 4 on host 'smithi029' 2023-11-27T15:19:45.878 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 -- 172.21.15.29:0/1462385924 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fd40a1250 msgr2=0x7f6fd40a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.878 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fd40a1250 0x7f6fd40a3700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800b5c0 tx=0x7f6fd8005fd0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.878 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 -- 172.21.15.29:0/1462385924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fe80ff1e0 msgr2=0x7f6fe8069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:45.878 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fe80ff1e0 0x7f6fe8069130 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400b500 tx=0x7f6fe400b8c0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.878 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 -- 172.21.15.29:0/1462385924 shutdown_connections 2023-11-27T15:19:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fe8100d70 0x7f6fe819d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fe80ff1e0 0x7f6fe8069130 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fd40a1250 0x7f6fd40a3700 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 --2- 172.21.15.29:0/1462385924 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fe81003e0 0x7f6fe819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 -- 172.21.15.29:0/1462385924 >> 172.21.15.29:0/1462385924 conn(0x7f6fe80fa7f0 msgr2=0x7f6fe81039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 -- 172.21.15.29:0/1462385924 shutdown_connections 2023-11-27T15:19:45.879 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:19:45.874+0000 7f6ff0470700 1 -- 172.21.15.29:0/1462385924 wait complete. 2023-11-27T15:19:46.252 DEBUG:teuthology.orchestra.run.smithi029:osd.4> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.4.service 2023-11-27T15:19:46.254 INFO:tasks.cephadm:Deploying osd.5 on smithi060 with /dev/vg_nvme/lv_3... 2023-11-27T15:19:46.255 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-27T15:19:46.290 INFO:journalctl@ceph.osd.4.smithi029.stdout:-- Logs begin at Mon 2023-11-27 14:52:42 UTC. -- 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:46.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:46.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:19:46.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:46.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:46.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:46.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:19:46.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:46.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:46.932 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:19:47.075 INFO:journalctl@ceph.osd.4.smithi029.stdout:Nov 27 15:19:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-4[141760]: 2023-11-27T15:19:47.000+0000 7f3b2e6f6640 -1 osd.4 0 log_to_monitors true 2023-11-27T15:19:47.667 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:47 smithi029 ceph-mon[131166]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:47.667 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:47 smithi029 ceph-mon[131166]: from='osd.4 [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:19:47.667 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:47 smithi029 ceph-mon[131166]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:19:47.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:47 smithi060 ceph-mon[131518]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:47.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:47 smithi060 ceph-mon[131518]: from='osd.4 [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:19:47.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:47 smithi060 ceph-mon[131518]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:19:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:47 smithi008 ceph-mon[128788]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:47.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:47 smithi008 ceph-mon[128788]: from='osd.4 [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:19:47.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:47 smithi008 ceph-mon[128788]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:19:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-27T15:19:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-27T15:19:48.512 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2023-11-27T15:19:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2023-11-27T15:19:48.512 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00808543 s, 1.3 GB/s 2023-11-27T15:19:48.512 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2023-11-27T15:19:48.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_3 2023-11-27T15:19:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-27T15:19:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: osdmap e32: 5 total, 4 up, 5 in 2023-11-27T15:19:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='osd.4 [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:19:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:19:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi029 2023-11-27T15:19:49.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:49.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:49.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:49.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:49.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-27T15:19:49.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: osdmap e32: 5 total, 4 up, 5 in 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='osd.4 [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi029 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:49.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:49.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.147 INFO:journalctl@ceph.osd.4.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-4[141760]: 2023-11-27T15:19:48.845+0000 7f3b23d6d700 -1 osd.4 0 waiting for initial osdmap 2023-11-27T15:19:49.147 INFO:journalctl@ceph.osd.4.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-4[141760]: 2023-11-27T15:19:48.852+0000 7f3b1db5e700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:19:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-27T15:19:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: osdmap e32: 5 total, 4 up, 5 in 2023-11-27T15:19:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='osd.4 [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:19:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:19:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi029 2023-11-27T15:19:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:49.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:49.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:49.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:49.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:19:49.530 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:19:50.025 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:49 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi029 to 8586M 2023-11-27T15:19:50.025 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:49 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:19:50.025 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:49 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:19:50.025 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:49 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:19:50.025 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:49 smithi060 ceph-mon[131518]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:19:50.025 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:49 smithi060 ceph-mon[131518]: osdmap e33: 5 total, 4 up, 5 in 2023-11-27T15:19:50.025 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:50.026 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:50.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:49 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi029 to 8586M 2023-11-27T15:19:50.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:49 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:19:50.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:49 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:19:50.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:49 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:19:50.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:49 smithi008 ceph-mon[128788]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:19:50.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:49 smithi008 ceph-mon[128788]: osdmap e33: 5 total, 4 up, 5 in 2023-11-27T15:19:50.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:50.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:50.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:49 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi029 to 8586M 2023-11-27T15:19:50.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:49 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:19:50.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:49 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:19:50.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:49 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:19:50.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:49 smithi029 ceph-mon[131166]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:19:50.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:49 smithi029 ceph-mon[131166]: osdmap e33: 5 total, 4 up, 5 in 2023-11-27T15:19:50.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:50.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:50.719 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.715+0000 7f71e81ec700 1 -- 172.21.15.60:0/260382219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e01003e0 msgr2=0x7f71e0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:50.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.715+0000 7f71e81ec700 1 --2- 172.21.15.60:0/260382219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e01003e0 0x7f71e0100830 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f71dc009990 tx=0x7f71dc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.715+0000 7f71e81ec700 1 -- 172.21.15.60:0/260382219 shutdown_connections 2023-11-27T15:19:50.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.715+0000 7f71e81ec700 1 --2- 172.21.15.60:0/260382219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71e0100d70 0x7f71e0107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.715+0000 7f71e81ec700 1 --2- 172.21.15.60:0/260382219 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e00ff1e0 0x7f71e00ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.715+0000 7f71e81ec700 1 --2- 172.21.15.60:0/260382219 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e01003e0 0x7f71e0100830 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.715+0000 7f71e81ec700 1 -- 172.21.15.60:0/260382219 >> 172.21.15.60:0/260382219 conn(0x7f71e00fa7f0 msgr2=0x7f71e00fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:50.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.715+0000 7f71e81ec700 1 -- 172.21.15.60:0/260382219 shutdown_connections 2023-11-27T15:19:50.720 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.715+0000 7f71e81ec700 1 -- 172.21.15.60:0/260382219 wait complete. 2023-11-27T15:19:50.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e81ec700 1 Processor -- start 2023-11-27T15:19:50.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e81ec700 1 -- start start 2023-11-27T15:19:50.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e81ec700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e00ff1e0 0x7f71e0194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:50.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e81ec700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e01003e0 0x7f71e0195500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:50.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e81ec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71e0100d70 0x7f71e019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:50.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e81ec700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e019cb50 con 0x7f71e0100d70 2023-11-27T15:19:50.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e81ec700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e019cc90 con 0x7f71e01003e0 2023-11-27T15:19:50.723 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e81ec700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e019ce20 con 0x7f71e00ff1e0 2023-11-27T15:19:50.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e5f88700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e00ff1e0 0x7f71e0194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:50.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e5f88700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e00ff1e0 0x7f71e0194fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34880/0 (socket says 172.21.15.60:34880) 2023-11-27T15:19:50.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e5f88700 1 -- 172.21.15.60:0/669916 learned_addr learned my addr 172.21.15.60:0/669916 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-27T15:19:50.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e5787700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e01003e0 0x7f71e0195500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:50.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e5f88700 1 -- 172.21.15.60:0/669916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e01003e0 msgr2=0x7f71e0195500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:50.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e5f88700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e01003e0 0x7f71e0195500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e5f88700 1 -- 172.21.15.60:0/669916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71e0100d70 msgr2=0x7f71e019c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:50.724 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e5f88700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71e0100d70 0x7f71e019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.716+0000 7f71e5f88700 1 -- 172.21.15.60:0/669916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc0095f0 con 0x7f71e00ff1e0 2023-11-27T15:19:50.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.717+0000 7f71e5f88700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e00ff1e0 0x7f71e0194fc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f71d000d9b0 tx=0x7f71d000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:50.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.717+0000 7f71d6ffd700 1 -- 172.21.15.60:0/669916 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0004020 con 0x7f71e00ff1e0 2023-11-27T15:19:50.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.717+0000 7f71d6ffd700 1 -- 172.21.15.60:0/669916 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71d0009730 con 0x7f71e00ff1e0 2023-11-27T15:19:50.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.717+0000 7f71e81ec700 1 -- 172.21.15.60:0/669916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e010ac00 con 0x7f71e00ff1e0 2023-11-27T15:19:50.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.717+0000 7f71d6ffd700 1 -- 172.21.15.60:0/669916 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d0003a30 con 0x7f71e00ff1e0 2023-11-27T15:19:50.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.717+0000 7f71e81ec700 1 -- 172.21.15.60:0/669916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e010aa70 con 0x7f71e00ff1e0 2023-11-27T15:19:50.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.718+0000 7f71e81ec700 1 -- 172.21.15.60:0/669916 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e0066ea0 con 0x7f71e00ff1e0 2023-11-27T15:19:50.725 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.719+0000 7f71d6ffd700 1 -- 172.21.15.60:0/669916 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f71d0022020 con 0x7f71e00ff1e0 2023-11-27T15:19:50.726 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.719+0000 7f71d6ffd700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f71cc0a11d0 0x7f71cc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:50.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.719+0000 7f71d6ffd700 1 -- 172.21.15.60:0/669916 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f71d00cb030 con 0x7f71e00ff1e0 2023-11-27T15:19:50.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.720+0000 7f71e5787700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f71cc0a11d0 0x7f71cc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:50.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.720+0000 7f71e5787700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f71cc0a11d0 0x7f71cc0a3680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00b5c0 tx=0x7f71dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:50.727 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.723+0000 7f71d6ffd700 1 -- 172.21.15.60:0/669916 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f71d0095870 con 0x7f71e00ff1e0 2023-11-27T15:19:50.931 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:19:50.927+0000 7f71e81ec700 1 -- 172.21.15.60:0/669916 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f71e0069350 con 0x7f71cc0a11d0 2023-11-27T15:19:51.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:50 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:19:51.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:50 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:19:51.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:50 smithi008 ceph-mon[128788]: osd.4 [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] boot 2023-11-27T15:19:51.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:50 smithi008 ceph-mon[128788]: osdmap e34: 5 total, 5 up, 5 in 2023-11-27T15:19:51.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:51.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:50 smithi008 ceph-mon[128788]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:51.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:50 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:19:51.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:50 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:19:51.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:50 smithi029 ceph-mon[131166]: osd.4 [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] boot 2023-11-27T15:19:51.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:50 smithi029 ceph-mon[131166]: osdmap e34: 5 total, 5 up, 5 in 2023-11-27T15:19:51.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:51.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:50 smithi029 ceph-mon[131166]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:51.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:50 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:19:51.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:50 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:19:51.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:50 smithi060 ceph-mon[131518]: osd.4 [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] boot 2023-11-27T15:19:51.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:50 smithi060 ceph-mon[131518]: osdmap e34: 5 total, 5 up, 5 in 2023-11-27T15:19:51.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:19:51.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:50 smithi060 ceph-mon[131518]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:19:52.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:51 smithi008 ceph-mon[128788]: osdmap e35: 5 total, 5 up, 5 in 2023-11-27T15:19:52.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:51 smithi008 ceph-mon[128788]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:52.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:52.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:52.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:52.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:51 smithi060 ceph-mon[131518]: osdmap e35: 5 total, 5 up, 5 in 2023-11-27T15:19:52.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:51 smithi060 ceph-mon[131518]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:52.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:52.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:52.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:52.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:51 smithi029 ceph-mon[131166]: osdmap e35: 5 total, 5 up, 5 in 2023-11-27T15:19:52.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:51 smithi029 ceph-mon[131166]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:52.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:52.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:52.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:53.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:52 smithi060 ceph-mon[131518]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:53.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:52 smithi029 ceph-mon[131166]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:53.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:52 smithi008 ceph-mon[128788]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:54.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:54 smithi060 ceph-mon[131518]: from='client.? 172.21.15.60:0/3084332149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4c2fbe2-63ef-4d43-a04f-38bef2ac6839"}]: dispatch 2023-11-27T15:19:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:54 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4c2fbe2-63ef-4d43-a04f-38bef2ac6839"}]: dispatch 2023-11-27T15:19:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:54 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4c2fbe2-63ef-4d43-a04f-38bef2ac6839"}]': finished 2023-11-27T15:19:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:54 smithi060 ceph-mon[131518]: osdmap e36: 6 total, 5 up, 6 in 2023-11-27T15:19:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:19:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:54 smithi060 ceph-mon[131518]: from='client.? 172.21.15.60:0/1221362221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:54.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:54 smithi008 ceph-mon[128788]: from='client.? 172.21.15.60:0/3084332149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4c2fbe2-63ef-4d43-a04f-38bef2ac6839"}]: dispatch 2023-11-27T15:19:54.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:54 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4c2fbe2-63ef-4d43-a04f-38bef2ac6839"}]: dispatch 2023-11-27T15:19:54.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:54 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4c2fbe2-63ef-4d43-a04f-38bef2ac6839"}]': finished 2023-11-27T15:19:54.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:54 smithi008 ceph-mon[128788]: osdmap e36: 6 total, 5 up, 6 in 2023-11-27T15:19:54.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:19:54.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:54 smithi008 ceph-mon[128788]: from='client.? 172.21.15.60:0/1221362221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:54.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:54 smithi029 ceph-mon[131166]: from='client.? 172.21.15.60:0/3084332149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4c2fbe2-63ef-4d43-a04f-38bef2ac6839"}]: dispatch 2023-11-27T15:19:54.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:54 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4c2fbe2-63ef-4d43-a04f-38bef2ac6839"}]: dispatch 2023-11-27T15:19:54.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:54 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4c2fbe2-63ef-4d43-a04f-38bef2ac6839"}]': finished 2023-11-27T15:19:54.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:54 smithi029 ceph-mon[131166]: osdmap e36: 6 total, 5 up, 6 in 2023-11-27T15:19:54.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:19:54.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:54 smithi029 ceph-mon[131166]: from='client.? 172.21.15.60:0/1221362221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:55.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:55 smithi060 ceph-mon[131518]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:55.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:55 smithi008 ceph-mon[128788]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:55.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:55 smithi029 ceph-mon[131166]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:57.517 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:57 smithi060 ceph-mon[131518]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:57.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:57 smithi008 ceph-mon[128788]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:57.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:57 smithi029 ceph-mon[131166]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:59.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:19:59 smithi060 ceph-mon[131518]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:19:59 smithi008 ceph-mon[128788]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:19:59.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:19:59 smithi029 ceph-mon[131166]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:00.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:00 smithi060 ceph-mon[131518]: overall HEALTH_OK 2023-11-27T15:20:00.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-27T15:20:00.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:00.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:00 smithi008 ceph-mon[128788]: overall HEALTH_OK 2023-11-27T15:20:00.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-27T15:20:00.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:00.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:00 smithi029 ceph-mon[131166]: overall HEALTH_OK 2023-11-27T15:20:00.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-27T15:20:00.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:01.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:01 smithi060 ceph-mon[131518]: Deploying daemon osd.5 on smithi060 2023-11-27T15:20:01.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:01 smithi060 ceph-mon[131518]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:01 smithi008 ceph-mon[128788]: Deploying daemon osd.5 on smithi060 2023-11-27T15:20:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:01 smithi008 ceph-mon[128788]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:01.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:01 smithi029 ceph-mon[131166]: Deploying daemon osd.5 on smithi060 2023-11-27T15:20:01.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:01 smithi029 ceph-mon[131166]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:03.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:03 smithi008 ceph-mon[128788]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:03.614 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:03 smithi060 ceph-mon[131518]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:03.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:03 smithi029 ceph-mon[131166]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:05.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:05 smithi008 ceph-mon[128788]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:05.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:05.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:05 smithi029 ceph-mon[131166]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:05.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:05.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:05.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:05.657 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:05 smithi060 ceph-mon[131518]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:05.657 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:05.657 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:05.657 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:07.513 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:07 smithi060 ceph-mon[131518]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:07.513 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:07.513 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:07.513 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:07.513 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:07.513 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:07.547 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.543+0000 7f71d6ffd700 1 -- 172.21.15.60:0/669916 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f71e0069350 con 0x7f71cc0a11d0 2023-11-27T15:20:07.547 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 5 on host 'smithi060' 2023-11-27T15:20:07.549 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.545+0000 7f71e81ec700 1 -- 172.21.15.60:0/669916 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f71cc0a11d0 msgr2=0x7f71cc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.545+0000 7f71e81ec700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f71cc0a11d0 0x7f71cc0a3680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00b5c0 tx=0x7f71dc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.545+0000 7f71e81ec700 1 -- 172.21.15.60:0/669916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e00ff1e0 msgr2=0x7f71e0194fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.545+0000 7f71e81ec700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e00ff1e0 0x7f71e0194fc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f71d000d9b0 tx=0x7f71d000dcc0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.545+0000 7f71e81ec700 1 -- 172.21.15.60:0/669916 shutdown_connections 2023-11-27T15:20:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.545+0000 7f71e81ec700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f71cc0a11d0 0x7f71cc0a3680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.545+0000 7f71e81ec700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f71e00ff1e0 0x7f71e0194fc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.545+0000 7f71e81ec700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f71e0100d70 0x7f71e019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.545+0000 7f71e81ec700 1 --2- 172.21.15.60:0/669916 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f71e01003e0 0x7f71e0195500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.545+0000 7f71e81ec700 1 -- 172.21.15.60:0/669916 >> 172.21.15.60:0/669916 conn(0x7f71e00fa7f0 msgr2=0x7f71e0103920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:07.550 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.546+0000 7f71e81ec700 1 -- 172.21.15.60:0/669916 shutdown_connections 2023-11-27T15:20:07.551 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:20:07.546+0000 7f71e81ec700 1 -- 172.21.15.60:0/669916 wait complete. 2023-11-27T15:20:07.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:07 smithi008 ceph-mon[128788]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:07.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:07.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:07.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:07.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:07.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:07.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:07 smithi029 ceph-mon[131166]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:07.913 DEBUG:teuthology.orchestra.run.smithi060:osd.5> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.5.service 2023-11-27T15:20:07.915 INFO:tasks.cephadm:Deploying osd.6 on smithi008 with /dev/vg_nvme/lv_2... 2023-11-27T15:20:07.915 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-27T15:20:07.945 INFO:journalctl@ceph.osd.5.smithi060.stdout:-- Logs begin at Mon 2023-11-27 14:51:59 UTC. -- 2023-11-27T15:20:08.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:08 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:20:08.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:08 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:20:08.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:08 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:20:08.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:08 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:08.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:08 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:08.740 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:08 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:08.740 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:08 smithi060 ceph-mon[131518]: from='osd.5 [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:20:08.740 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:08 smithi060 ceph-mon[131518]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:20:08.740 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:20:08 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:20:08.409+0000 7f6622077640 -1 osd.5 0 log_to_monitors true 2023-11-27T15:20:08.789 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:20:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:08 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:20:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:08 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:20:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:08 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:20:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:08.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:08.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:08 smithi008 ceph-mon[128788]: from='osd.5 [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:20:08.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:08 smithi008 ceph-mon[128788]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:20:08.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:08 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:20:08.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:08 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:20:08.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:08 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:20:08.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:08 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:08.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:08 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:08.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:08 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:08.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:08 smithi029 ceph-mon[131166]: from='osd.5 [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:20:08.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:08 smithi029 ceph-mon[131166]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:20:09.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:09 smithi060 ceph-mon[131518]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:09.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:09 smithi060 ceph-mon[131518]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-27T15:20:09.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:09 smithi060 ceph-mon[131518]: osdmap e37: 6 total, 5 up, 6 in 2023-11-27T15:20:09.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:09.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:09 smithi060 ceph-mon[131518]: from='osd.5 [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:20:09.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:09 smithi060 ceph-mon[131518]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:20:09.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:09 smithi008 ceph-mon[128788]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:09.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:09 smithi008 ceph-mon[128788]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-27T15:20:09.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:09 smithi008 ceph-mon[128788]: osdmap e37: 6 total, 5 up, 6 in 2023-11-27T15:20:09.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:09.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:09 smithi008 ceph-mon[128788]: from='osd.5 [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:20:09.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:09 smithi008 ceph-mon[128788]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:20:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:09 smithi029 ceph-mon[131166]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:09 smithi029 ceph-mon[131166]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-27T15:20:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:09 smithi029 ceph-mon[131166]: osdmap e37: 6 total, 5 up, 6 in 2023-11-27T15:20:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:09 smithi029 ceph-mon[131166]: from='osd.5 [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:20:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:09 smithi029 ceph-mon[131166]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:20:10.574 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:20:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:20:10.180+0000 7f66176ee700 -1 osd.5 0 waiting for initial osdmap 2023-11-27T15:20:10.575 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:20:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:20:10.189+0000 7f66104dd700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:20:10.896 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-27T15:20:10.896 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-27T15:20:10.896 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2023-11-27T15:20:10.896 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2023-11-27T15:20:10.896 INFO:teuthology.orchestra.run.smithi008.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0330252 s, 318 MB/s 2023-11-27T15:20:10.896 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2023-11-27T15:20:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi060 2023-11-27T15:20:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi060 to 8586M 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: osdmap e38: 6 total, 5 up, 6 in 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:11.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:20:11.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:11 smithi008 ceph-mon[128788]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:11.479 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi008:vg_nvme/lv_2 2023-11-27T15:20:11.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi060 2023-11-27T15:20:11.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:11.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:11.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi060 to 8586M 2023-11-27T15:20:11.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:11.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:11.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:20:11.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: osdmap e38: 6 total, 5 up, 6 in 2023-11-27T15:20:11.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:11.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:11.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:20:11.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.578 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:11 smithi060 ceph-mon[131518]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:11.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi060 2023-11-27T15:20:11.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:11.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:11.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi060 to 8586M 2023-11-27T15:20:11.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:11.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:11.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:20:11.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: osdmap e38: 6 total, 5 up, 6 in 2023-11-27T15:20:11.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:11.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:11.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:11.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:20:11.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:20:11.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:11 smithi029 ceph-mon[131166]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:20:12.343 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:20:12.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:12 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:20:12.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:12 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:20:12.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:12 smithi060 ceph-mon[131518]: osd.5 [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] boot 2023-11-27T15:20:12.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:12 smithi060 ceph-mon[131518]: osdmap e39: 6 total, 6 up, 6 in 2023-11-27T15:20:12.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:12.609 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:12 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:20:12.609 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:12 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:20:12.610 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:12 smithi008 ceph-mon[128788]: osd.5 [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] boot 2023-11-27T15:20:12.610 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:12 smithi008 ceph-mon[128788]: osdmap e39: 6 total, 6 up, 6 in 2023-11-27T15:20:12.610 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:12.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:12 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:20:12.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:12 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:20:12.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:12 smithi029 ceph-mon[131166]: osd.5 [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] boot 2023-11-27T15:20:12.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:12 smithi029 ceph-mon[131166]: osdmap e39: 6 total, 6 up, 6 in 2023-11-27T15:20:12.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:20:13.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:13 smithi060 ceph-mon[131518]: osdmap e40: 6 total, 6 up, 6 in 2023-11-27T15:20:13.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:13 smithi060 ceph-mon[131518]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:13.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:13 smithi008 ceph-mon[128788]: osdmap e40: 6 total, 6 up, 6 in 2023-11-27T15:20:13.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:13 smithi008 ceph-mon[128788]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:13.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:13 smithi029 ceph-mon[131166]: osdmap e40: 6 total, 6 up, 6 in 2023-11-27T15:20:13.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:13 smithi029 ceph-mon[131166]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:13.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.874+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2827066474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b7c074ac0 msgr2=0x7f9b7c072f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:13.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.874+0000 7f9b80b1c700 1 --2- 172.21.15.8:0/2827066474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b7c074ac0 0x7f9b7c072f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009990 tx=0x7f9b6c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.875+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2827066474 shutdown_connections 2023-11-27T15:20:13.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.875+0000 7f9b80b1c700 1 --2- 172.21.15.8:0/2827066474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b7c108190 0x7f9b7c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.875+0000 7f9b80b1c700 1 --2- 172.21.15.8:0/2827066474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b7c074ac0 0x7f9b7c072f20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.875+0000 7f9b80b1c700 1 --2- 172.21.15.8:0/2827066474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b7c0734f0 0x7f9b7c073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.875+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2827066474 >> 172.21.15.8:0/2827066474 conn(0x7f9b7c0fb900 msgr2=0x7f9b7c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:13.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.875+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2827066474 shutdown_connections 2023-11-27T15:20:13.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.875+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2827066474 wait complete. 2023-11-27T15:20:13.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.876+0000 7f9b80b1c700 1 Processor -- start 2023-11-27T15:20:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.876+0000 7f9b80b1c700 1 -- start start 2023-11-27T15:20:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.876+0000 7f9b80b1c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b7c0734f0 0x7f9b7c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.876+0000 7f9b80b1c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b7c074ac0 0x7f9b7c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.876+0000 7f9b80b1c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b7c108190 0x7f9b7c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.876+0000 7f9b80b1c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c19fd80 con 0x7f9b7c074ac0 2023-11-27T15:20:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.876+0000 7f9b80b1c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c19fec0 con 0x7f9b7c0734f0 2023-11-27T15:20:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.876+0000 7f9b80b1c700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c1a0050 con 0x7f9b7c108190 2023-11-27T15:20:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b79d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b7c074ac0 0x7f9b7c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b79d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b7c074ac0 0x7f9b7c198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38316/0 (socket says 172.21.15.8:38316) 2023-11-27T15:20:13.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b79d9b700 1 -- 172.21.15.8:0/2952944676 learned_addr learned my addr 172.21.15.8:0/2952944676 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b7a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b7c0734f0 0x7f9b7c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b7ad9d700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b7c108190 0x7f9b7c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b79d9b700 1 -- 172.21.15.8:0/2952944676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b7c108190 msgr2=0x7f9b7c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b79d9b700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b7c108190 0x7f9b7c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b79d9b700 1 -- 172.21.15.8:0/2952944676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b7c0734f0 msgr2=0x7f9b7c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b79d9b700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b7c0734f0 0x7f9b7c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b79d9b700 1 -- 172.21.15.8:0/2952944676 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c0095f0 con 0x7f9b7c074ac0 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b79d9b700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b7c074ac0 0x7f9b7c198730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000e9d0 tx=0x7f9b7000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.877+0000 7f9b6b7fe700 1 -- 172.21.15.8:0/2952944676 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70009790 con 0x7f9b7c074ac0 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.878+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2952944676 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7c10de50 con 0x7f9b7c074ac0 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.878+0000 7f9b6b7fe700 1 -- 172.21.15.8:0/2952944676 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b7000ee70 con 0x7f9b7c074ac0 2023-11-27T15:20:13.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.878+0000 7f9b6b7fe700 1 -- 172.21.15.8:0/2952944676 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70010ea0 con 0x7f9b7c074ac0 2023-11-27T15:20:13.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.878+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2952944676 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7c10dcc0 con 0x7f9b7c074ac0 2023-11-27T15:20:13.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.879+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2952944676 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7c1068d0 con 0x7f9b7c074ac0 2023-11-27T15:20:13.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.881+0000 7f9b6b7fe700 1 -- 172.21.15.8:0/2952944676 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9b7000cbd0 con 0x7f9b7c074ac0 2023-11-27T15:20:13.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.882+0000 7f9b6b7fe700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9b640a1150 0x7f9b640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:13.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.882+0000 7f9b6b7fe700 1 -- 172.21.15.8:0/2952944676 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f9b70014070 con 0x7f9b7c074ac0 2023-11-27T15:20:13.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.882+0000 7f9b7a59c700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9b640a1150 0x7f9b640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:13.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.883+0000 7f9b7a59c700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9b640a1150 0x7f9b640a3600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00b2e0 tx=0x7f9b6c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:13.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:13.885+0000 7f9b6b7fe700 1 -- 172.21.15.8:0/2952944676 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9b700966c0 con 0x7f9b7c074ac0 2023-11-27T15:20:14.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:14.100+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2952944676 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b7c061a10 con 0x7f9b640a1150 2023-11-27T15:20:14.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:14 smithi060 ceph-mon[131518]: osdmap e41: 6 total, 6 up, 6 in 2023-11-27T15:20:14.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:20:14.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:20:14.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:14 smithi008 ceph-mon[128788]: osdmap e41: 6 total, 6 up, 6 in 2023-11-27T15:20:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:20:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:20:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:14.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:14 smithi029 ceph-mon[131166]: osdmap e41: 6 total, 6 up, 6 in 2023-11-27T15:20:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:20:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:20:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:15.484 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:15 smithi008 ceph-mon[128788]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:20:15.484 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:15 smithi008 ceph-mon[128788]: osdmap e42: 6 total, 6 up, 6 in 2023-11-27T15:20:15.484 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:15 smithi008 ceph-mon[128788]: pgmap v140: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 1/6 objects misplaced (16.667%); 64 KiB/s, 0 objects/s recovering 2023-11-27T15:20:15.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:15 smithi060 ceph-mon[131518]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:20:15.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:15 smithi060 ceph-mon[131518]: osdmap e42: 6 total, 6 up, 6 in 2023-11-27T15:20:15.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:15 smithi060 ceph-mon[131518]: pgmap v140: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 1/6 objects misplaced (16.667%); 64 KiB/s, 0 objects/s recovering 2023-11-27T15:20:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:15 smithi029 ceph-mon[131166]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:20:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:15 smithi029 ceph-mon[131166]: osdmap e42: 6 total, 6 up, 6 in 2023-11-27T15:20:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:15 smithi029 ceph-mon[131166]: pgmap v140: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail; 1/6 objects misplaced (16.667%); 64 KiB/s, 0 objects/s recovering 2023-11-27T15:20:16.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:16 smithi060 ceph-mon[131518]: osdmap e43: 6 total, 6 up, 6 in 2023-11-27T15:20:16.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:16 smithi008 ceph-mon[128788]: osdmap e43: 6 total, 6 up, 6 in 2023-11-27T15:20:16.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:16 smithi029 ceph-mon[131166]: osdmap e43: 6 total, 6 up, 6 in 2023-11-27T15:20:17.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:17 smithi060 ceph-mon[131518]: pgmap v142: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 1/6 objects misplaced (16.667%); 62 KiB/s, 0 objects/s recovering 2023-11-27T15:20:17.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:17 smithi008 ceph-mon[128788]: pgmap v142: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 1/6 objects misplaced (16.667%); 62 KiB/s, 0 objects/s recovering 2023-11-27T15:20:17.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:17 smithi029 ceph-mon[131166]: pgmap v142: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 1/6 objects misplaced (16.667%); 62 KiB/s, 0 objects/s recovering 2023-11-27T15:20:18.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:18 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3586258333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "309035f0-69df-47d3-b854-c1b480c200c0"}]: dispatch 2023-11-27T15:20:18.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:18 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3586258333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "309035f0-69df-47d3-b854-c1b480c200c0"}]': finished 2023-11-27T15:20:18.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:18 smithi060 ceph-mon[131518]: osdmap e44: 7 total, 6 up, 7 in 2023-11-27T15:20:18.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:18 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:18.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:18 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3514810094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:20:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:18 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3586258333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "309035f0-69df-47d3-b854-c1b480c200c0"}]: dispatch 2023-11-27T15:20:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:18 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3586258333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "309035f0-69df-47d3-b854-c1b480c200c0"}]': finished 2023-11-27T15:20:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:18 smithi008 ceph-mon[128788]: osdmap e44: 7 total, 6 up, 7 in 2023-11-27T15:20:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:18 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:18 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3514810094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:20:18.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:18 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3586258333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "309035f0-69df-47d3-b854-c1b480c200c0"}]: dispatch 2023-11-27T15:20:18.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:18 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3586258333' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "309035f0-69df-47d3-b854-c1b480c200c0"}]': finished 2023-11-27T15:20:18.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:18 smithi029 ceph-mon[131166]: osdmap e44: 7 total, 6 up, 7 in 2023-11-27T15:20:18.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:18 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:18.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:18 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3514810094' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:20:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:19 smithi060 ceph-mon[131518]: pgmap v144: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 1/6 objects misplaced (16.667%); 50 KiB/s, 0 objects/s recovering 2023-11-27T15:20:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:19 smithi008 ceph-mon[128788]: pgmap v144: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 1/6 objects misplaced (16.667%); 50 KiB/s, 0 objects/s recovering 2023-11-27T15:20:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:19 smithi029 ceph-mon[131166]: pgmap v144: 1 pgs: 1 active+recovering+undersized+remapped; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 1/6 objects misplaced (16.667%); 50 KiB/s, 0 objects/s recovering 2023-11-27T15:20:21.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:21 smithi008 ceph-mon[128788]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-11-27T15:20:21.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:21 smithi029 ceph-mon[131166]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-11-27T15:20:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:21 smithi060 ceph-mon[131518]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-11-27T15:20:23.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:23 smithi008 ceph-mon[128788]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 40 KiB/s, 0 objects/s recovering 2023-11-27T15:20:23.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:23 smithi029 ceph-mon[131166]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 40 KiB/s, 0 objects/s recovering 2023-11-27T15:20:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:23 smithi060 ceph-mon[131518]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 40 KiB/s, 0 objects/s recovering 2023-11-27T15:20:25.350 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:25 smithi008 ceph-mon[128788]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 35 KiB/s, 0 objects/s recovering 2023-11-27T15:20:25.350 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-27T15:20:25.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:25.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:25 smithi029 ceph-mon[131166]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 35 KiB/s, 0 objects/s recovering 2023-11-27T15:20:25.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:25 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-27T15:20:25.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:25 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:25.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:25 smithi060 ceph-mon[131518]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 35 KiB/s, 0 objects/s recovering 2023-11-27T15:20:25.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-27T15:20:25.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:26.393 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:26 smithi008 ceph-mon[128788]: Deploying daemon osd.6 on smithi008 2023-11-27T15:20:26.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:26 smithi029 ceph-mon[131166]: Deploying daemon osd.6 on smithi008 2023-11-27T15:20:26.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:26 smithi060 ceph-mon[131518]: Deploying daemon osd.6 on smithi008 2023-11-27T15:20:27.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:27 smithi008 ceph-mon[128788]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 32 KiB/s, 0 objects/s recovering 2023-11-27T15:20:27.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:27 smithi029 ceph-mon[131166]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 32 KiB/s, 0 objects/s recovering 2023-11-27T15:20:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:27 smithi060 ceph-mon[131518]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 32 KiB/s, 0 objects/s recovering 2023-11-27T15:20:29.493 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:29 smithi008 ceph-mon[128788]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s, 0 objects/s recovering 2023-11-27T15:20:29.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:29 smithi060 ceph-mon[131518]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s, 0 objects/s recovering 2023-11-27T15:20:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:29 smithi029 ceph-mon[131166]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s, 0 objects/s recovering 2023-11-27T15:20:31.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:31 smithi008 ceph-mon[128788]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 27 KiB/s, 0 objects/s recovering 2023-11-27T15:20:31.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:31 smithi029 ceph-mon[131166]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 27 KiB/s, 0 objects/s recovering 2023-11-27T15:20:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:31 smithi060 ceph-mon[131518]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 27 KiB/s, 0 objects/s recovering 2023-11-27T15:20:33.124 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:33.124 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:33.125 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:33.125 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:32 smithi008 ceph-mon[128788]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:33.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:33.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:33.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:33.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:32 smithi029 ceph-mon[131166]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:33.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:33.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:33.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:33.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:32 smithi060 ceph-mon[131518]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:35.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:35 smithi008 ceph-mon[128788]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:35.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:35.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:35.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:35.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:35.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:35.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:35 smithi029 ceph-mon[131166]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:35.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:35.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:35.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:35.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:35.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:35.769 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 6 on host 'smithi008' 2023-11-27T15:20:35.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.764+0000 7f9b6b7fe700 1 -- 172.21.15.8:0/2952944676 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9b7c061a10 con 0x7f9b640a1150 2023-11-27T15:20:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2952944676 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9b640a1150 msgr2=0x7f9b640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9b640a1150 0x7f9b640a3600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00b2e0 tx=0x7f9b6c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2952944676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b7c074ac0 msgr2=0x7f9b7c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b7c074ac0 0x7f9b7c198730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000e9d0 tx=0x7f9b7000ece0 comp rx=0 tx=0).stop 2023-11-27T15:20:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2952944676 shutdown_connections 2023-11-27T15:20:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9b640a1150 0x7f9b640a3600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:35.771 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9b7c108190 0x7f9b7c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9b7c0734f0 0x7f9b7c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 --2- 172.21.15.8:0/2952944676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b7c074ac0 0x7f9b7c198730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2952944676 >> 172.21.15.8:0/2952944676 conn(0x7f9b7c0fb900 msgr2=0x7f9b7c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2952944676 shutdown_connections 2023-11-27T15:20:35.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:20:35.767+0000 7f9b80b1c700 1 -- 172.21.15.8:0/2952944676 wait complete. 2023-11-27T15:20:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:35 smithi060 ceph-mon[131518]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:36.279 DEBUG:teuthology.orchestra.run.smithi008:osd.6> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.6.service 2023-11-27T15:20:36.284 INFO:tasks.cephadm:Deploying osd.7 on smithi029 with /dev/vg_nvme/lv_2... 2023-11-27T15:20:36.284 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-27T15:20:36.359 INFO:journalctl@ceph.osd.6.smithi008.stdout:-- Logs begin at Mon 2023-11-27 14:56:10 UTC. -- 2023-11-27T15:20:36.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:36 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:20:36.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:36 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:20:36.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:36 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:20:36.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:36 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:36.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:36 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:36.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:36 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:36.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:36 smithi008 ceph-mon[128788]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-27T15:20:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:36 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:20:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:36 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:20:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:36 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:20:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:36 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:36 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:36 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:36.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:36 smithi029 ceph-mon[131166]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-27T15:20:36.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:36 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:20:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:36 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:20:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:36 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:20:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:36 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:36 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:36 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:36 smithi060 ceph-mon[131518]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-27T15:20:36.948 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:20:37.453 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:37 smithi029 ceph-mon[131166]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:37.453 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:37 smithi029 ceph-mon[131166]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-27T15:20:37.453 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:37 smithi029 ceph-mon[131166]: osdmap e45: 7 total, 6 up, 7 in 2023-11-27T15:20:37.454 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:37 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:37.454 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:37 smithi029 ceph-mon[131166]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:20:37.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:37 smithi008 ceph-mon[128788]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:37.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:37 smithi008 ceph-mon[128788]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-27T15:20:37.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:37 smithi008 ceph-mon[128788]: osdmap e45: 7 total, 6 up, 7 in 2023-11-27T15:20:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:37 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:37.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:37 smithi008 ceph-mon[128788]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:20:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:37 smithi060 ceph-mon[131518]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:37 smithi060 ceph-mon[131518]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-27T15:20:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:37 smithi060 ceph-mon[131518]: osdmap e45: 7 total, 6 up, 7 in 2023-11-27T15:20:37.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:37 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:37.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:37 smithi060 ceph-mon[131518]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:20:38.113 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:20:37 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:20:37.764+0000 7f73c1c32700 -1 osd.6 0 waiting for initial osdmap 2023-11-27T15:20:38.114 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:20:37 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:20:37.776+0000 7f73bba23700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:20:38.463 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-27T15:20:38.463 INFO:teuthology.orchestra.run.smithi029.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-27T15:20:38.463 INFO:teuthology.orchestra.run.smithi029.stderr: stderr: 10+0 records in 2023-11-27T15:20:38.463 INFO:teuthology.orchestra.run.smithi029.stderr:10+0 records out 2023-11-27T15:20:38.463 INFO:teuthology.orchestra.run.smithi029.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133791 s, 784 MB/s 2023-11-27T15:20:38.463 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping successful for: 2023-11-27T15:20:38.805 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi029:vg_nvme/lv_2 2023-11-27T15:20:39.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:38 smithi060 ceph-mon[131518]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:20:39.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:38 smithi060 ceph-mon[131518]: osdmap e46: 7 total, 6 up, 7 in 2023-11-27T15:20:39.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:39.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:39.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:38 smithi060 ceph-mon[131518]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:38 smithi008 ceph-mon[128788]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:20:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:38 smithi008 ceph-mon[128788]: osdmap e46: 7 total, 6 up, 7 in 2023-11-27T15:20:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:39.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:38 smithi008 ceph-mon[128788]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:39.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:38 smithi029 ceph-mon[131166]: from='osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:20:39.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:38 smithi029 ceph-mon[131166]: osdmap e46: 7 total, 6 up, 7 in 2023-11-27T15:20:39.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:39.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:39.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:38 smithi029 ceph-mon[131166]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:20:39.458 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:20:39.995 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:20:39.995 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:20:39.995 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] boot 2023-11-27T15:20:39.995 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: osdmap e47: 7 total, 7 up, 7 in 2023-11-27T15:20:39.995 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:39.995 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi008 2023-11-27T15:20:39.995 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi008 to 5758M 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:20:39.996 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:39 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:20:40.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:20:40.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:20:40.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] boot 2023-11-27T15:20:40.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: osdmap e47: 7 total, 7 up, 7 in 2023-11-27T15:20:40.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:40.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi008 2023-11-27T15:20:40.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:40.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:40.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:40.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:40.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:40.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi008 to 5758M 2023-11-27T15:20:40.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:40.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:40.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:40.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:20:40.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:40.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:20:40.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:39 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:20:40.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:20:40.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:20:40.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: osd.6 [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] boot 2023-11-27T15:20:40.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: osdmap e47: 7 total, 7 up, 7 in 2023-11-27T15:20:40.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:20:40.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi008 2023-11-27T15:20:40.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:40.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:40.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:40.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:40.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:20:40.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi008 to 5758M 2023-11-27T15:20:40.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:40.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:40.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:40.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:20:40.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:40.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:20:40.115 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:39 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:20:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.669+0000 7f2f7a782700 1 -- 172.21.15.29:0/3607797802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f74101e80 msgr2=0x7f2f74108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.669+0000 7f2f7a782700 1 --2- 172.21.15.29:0/3607797802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f74101e80 0x7f2f74108740 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00b0a0 tx=0x7f2f6c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.669+0000 7f2f7a782700 1 -- 172.21.15.29:0/3607797802 shutdown_connections 2023-11-27T15:20:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.669+0000 7f2f7a782700 1 --2- 172.21.15.29:0/3607797802 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f74101e80 0x7f2f74108740 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.669+0000 7f2f7a782700 1 --2- 172.21.15.29:0/3607797802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f741002f0 0x7f2f74100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.669+0000 7f2f7a782700 1 --2- 172.21.15.29:0/3607797802 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f741014f0 0x7f2f74101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.669+0000 7f2f7a782700 1 -- 172.21.15.29:0/3607797802 >> 172.21.15.29:0/3607797802 conn(0x7f2f740fb900 msgr2=0x7f2f740fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.676 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.669+0000 7f2f7a782700 1 -- 172.21.15.29:0/3607797802 shutdown_connections 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.669+0000 7f2f7a782700 1 -- 172.21.15.29:0/3607797802 wait complete. 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f7a782700 1 Processor -- start 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f7a782700 1 -- start start 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f7a782700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f741002f0 0x7f2f74196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f7a782700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f741014f0 0x7f2f741966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f7a782700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f74101e80 0x7f2f7419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f7a782700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7419dd00 con 0x7f2f74101e80 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f7a782700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7419de40 con 0x7f2f741014f0 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f7a782700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7419dfd0 con 0x7f2f741002f0 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f737fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f741014f0 0x7f2f741966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f737fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f741014f0 0x7f2f741966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35188/0 (socket says 172.21.15.29:35188) 2023-11-27T15:20:40.677 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.670+0000 7f2f737fe700 1 -- 172.21.15.29:0/2182419311 learned_addr learned my addr 172.21.15.29:0/2182419311 (peer_addr_for_me v2:172.21.15.29:0/0) 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f78d1f700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f74101e80 0x7f2f7419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f73fff700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f741002f0 0x7f2f74196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f737fe700 1 -- 172.21.15.29:0/2182419311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f741002f0 msgr2=0x7f2f74196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f737fe700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f741002f0 0x7f2f74196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f737fe700 1 -- 172.21.15.29:0/2182419311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f74101e80 msgr2=0x7f2f7419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f737fe700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f74101e80 0x7f2f7419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f737fe700 1 -- 172.21.15.29:0/2182419311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c009d00 con 0x7f2f741014f0 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f737fe700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f741014f0 0x7f2f741966b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00d980 tx=0x7f2f5c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f717fa700 1 -- 172.21.15.29:0/2182419311 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c004220 con 0x7f2f741014f0 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f717fa700 1 -- 172.21.15.29:0/2182419311 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f5c007640 con 0x7f2f741014f0 2023-11-27T15:20:40.678 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f7a782700 1 -- 172.21.15.29:0/2182419311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7410bce0 con 0x7f2f741014f0 2023-11-27T15:20:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f717fa700 1 -- 172.21.15.29:0/2182419311 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0056c0 con 0x7f2f741014f0 2023-11-27T15:20:40.679 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.671+0000 7f2f7a782700 1 -- 172.21.15.29:0/2182419311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7410bb50 con 0x7f2f741014f0 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.672+0000 7f2f7a782700 1 -- 172.21.15.29:0/2182419311 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f741077f0 con 0x7f2f741014f0 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.673+0000 7f2f717fa700 1 -- 172.21.15.29:0/2182419311 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2f5c004380 con 0x7f2f741014f0 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.674+0000 7f2f717fa700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2f600a11d0 0x7f2f600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.674+0000 7f2f717fa700 1 -- 172.21.15.29:0/2182419311 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f2f5c0cb9b0 con 0x7f2f741014f0 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.674+0000 7f2f73fff700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2f600a11d0 0x7f2f600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.675+0000 7f2f73fff700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2f600a11d0 0x7f2f600a3680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2f74101350 tx=0x7f2f64009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.682 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.678+0000 7f2f717fa700 1 -- 172.21.15.29:0/2182419311 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2f5c095d70 con 0x7f2f741014f0 2023-11-27T15:20:40.889 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:40.885+0000 7f2f7a782700 1 -- 172.21.15.29:0/2182419311 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f74061bf0 con 0x7f2f600a11d0 2023-11-27T15:20:41.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:40 smithi060 ceph-mon[131518]: osdmap e48: 7 total, 7 up, 7 in 2023-11-27T15:20:41.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:40 smithi060 ceph-mon[131518]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:41.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:40 smithi008 ceph-mon[128788]: osdmap e48: 7 total, 7 up, 7 in 2023-11-27T15:20:41.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:40 smithi008 ceph-mon[128788]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:41.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:40 smithi029 ceph-mon[131166]: osdmap e48: 7 total, 7 up, 7 in 2023-11-27T15:20:41.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:40 smithi029 ceph-mon[131166]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:42.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:41 smithi060 ceph-mon[131518]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:20:42.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:41 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:20:42.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:41 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:20:42.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:41 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:42.077 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:41 smithi029 ceph-mon[131166]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:20:42.077 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:41 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:20:42.078 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:41 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:20:42.078 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:41 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:42.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:41 smithi008 ceph-mon[128788]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:20:42.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:41 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:20:42.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:41 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:20:42.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:41 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:42.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:42 smithi029 ceph-mon[131166]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:43.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:42 smithi060 ceph-mon[131518]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:43.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:42 smithi008 ceph-mon[128788]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:44.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:44 smithi060 ceph-mon[131518]: from='client.? 172.21.15.29:0/3526012600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fcef10a-546a-4928-9341-7d31b79942be"}]: dispatch 2023-11-27T15:20:44.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:44 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fcef10a-546a-4928-9341-7d31b79942be"}]: dispatch 2023-11-27T15:20:44.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:44 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fcef10a-546a-4928-9341-7d31b79942be"}]': finished 2023-11-27T15:20:44.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:44 smithi060 ceph-mon[131518]: osdmap e49: 8 total, 7 up, 8 in 2023-11-27T15:20:44.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:44 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:20:44.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:44 smithi060 ceph-mon[131518]: from='client.? 172.21.15.29:0/879065133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:20:44.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:44 smithi008 ceph-mon[128788]: from='client.? 172.21.15.29:0/3526012600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fcef10a-546a-4928-9341-7d31b79942be"}]: dispatch 2023-11-27T15:20:44.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:44 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fcef10a-546a-4928-9341-7d31b79942be"}]: dispatch 2023-11-27T15:20:44.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:44 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fcef10a-546a-4928-9341-7d31b79942be"}]': finished 2023-11-27T15:20:44.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:44 smithi008 ceph-mon[128788]: osdmap e49: 8 total, 7 up, 8 in 2023-11-27T15:20:44.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:44 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:20:44.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:44 smithi008 ceph-mon[128788]: from='client.? 172.21.15.29:0/879065133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:20:44.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:44 smithi029 ceph-mon[131166]: from='client.? 172.21.15.29:0/3526012600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fcef10a-546a-4928-9341-7d31b79942be"}]: dispatch 2023-11-27T15:20:44.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:44 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fcef10a-546a-4928-9341-7d31b79942be"}]: dispatch 2023-11-27T15:20:44.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:44 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fcef10a-546a-4928-9341-7d31b79942be"}]': finished 2023-11-27T15:20:44.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:44 smithi029 ceph-mon[131166]: osdmap e49: 8 total, 7 up, 8 in 2023-11-27T15:20:44.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:44 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:20:44.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:44 smithi029 ceph-mon[131166]: from='client.? 172.21.15.29:0/879065133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:20:45.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:45 smithi060 ceph-mon[131518]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:45 smithi008 ceph-mon[128788]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:45.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:45 smithi029 ceph-mon[131166]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:47.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:47 smithi008 ceph-mon[128788]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:47.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:47 smithi029 ceph-mon[131166]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:47 smithi060 ceph-mon[131518]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:49.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:49 smithi029 ceph-mon[131166]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:49.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:49 smithi060 ceph-mon[131518]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:49.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:49 smithi008 ceph-mon[128788]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:50.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-27T15:20:50.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:50.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-27T15:20:50.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:50.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-27T15:20:50.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:51.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:51 smithi029 ceph-mon[131166]: Deploying daemon osd.7 on smithi029 2023-11-27T15:20:51.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:51 smithi029 ceph-mon[131166]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:51.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:51 smithi060 ceph-mon[131518]: Deploying daemon osd.7 on smithi029 2023-11-27T15:20:51.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:51 smithi060 ceph-mon[131518]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:51.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:51 smithi008 ceph-mon[128788]: Deploying daemon osd.7 on smithi029 2023-11-27T15:20:51.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:51 smithi008 ceph-mon[128788]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:53.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:53 smithi008 ceph-mon[128788]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:53.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:53 smithi029 ceph-mon[131166]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:53 smithi060 ceph-mon[131518]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:55.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:55 smithi029 ceph-mon[131166]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:55.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:55 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:55.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:55 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:55.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:55 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:55 smithi060 ceph-mon[131518]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:55 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:55 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:55.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:55 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:55 smithi008 ceph-mon[128788]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:55 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:55 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:55 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:57.555 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 7 on host 'smithi029' 2023-11-27T15:20:57.556 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.551+0000 7f2f717fa700 1 -- 172.21.15.29:0/2182419311 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2f74061bf0 con 0x7f2f600a11d0 2023-11-27T15:20:57.558 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.553+0000 7f2f7a782700 1 -- 172.21.15.29:0/2182419311 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2f600a11d0 msgr2=0x7f2f600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:57.558 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.553+0000 7f2f7a782700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2f600a11d0 0x7f2f600a3680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2f74101350 tx=0x7f2f64009190 comp rx=0 tx=0).stop 2023-11-27T15:20:57.558 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.554+0000 7f2f7a782700 1 -- 172.21.15.29:0/2182419311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f741014f0 msgr2=0x7f2f741966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:57.558 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.554+0000 7f2f7a782700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f741014f0 0x7f2f741966b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00d980 tx=0x7f2f5c00dc90 comp rx=0 tx=0).stop 2023-11-27T15:20:57.558 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.554+0000 7f2f7a782700 1 -- 172.21.15.29:0/2182419311 shutdown_connections 2023-11-27T15:20:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.554+0000 7f2f7a782700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2f600a11d0 0x7f2f600a3680 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.554+0000 7f2f7a782700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2f741002f0 0x7f2f74196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.554+0000 7f2f7a782700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f74101e80 0x7f2f7419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.554+0000 7f2f7a782700 1 --2- 172.21.15.29:0/2182419311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2f741014f0 0x7f2f741966b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.554+0000 7f2f7a782700 1 -- 172.21.15.29:0/2182419311 >> 172.21.15.29:0/2182419311 conn(0x7f2f740fb900 msgr2=0x7f2f74104930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.554+0000 7f2f7a782700 1 -- 172.21.15.29:0/2182419311 shutdown_connections 2023-11-27T15:20:57.559 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:20:57.554+0000 7f2f7a782700 1 -- 172.21.15.29:0/2182419311 wait complete. 2023-11-27T15:20:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:57 smithi060 ceph-mon[131518]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:57 smithi008 ceph-mon[128788]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:57.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:57.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:57.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:57.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:57 smithi029 ceph-mon[131166]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:57.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:57.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:57.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:57.953 DEBUG:teuthology.orchestra.run.smithi029:osd.7> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.7.service 2023-11-27T15:20:57.956 INFO:tasks.cephadm:Deploying osd.8 on smithi060 with /dev/vg_nvme/lv_2... 2023-11-27T15:20:57.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-27T15:20:57.989 INFO:journalctl@ceph.osd.7.smithi029.stdout:-- Logs begin at Mon 2023-11-27 14:52:42 UTC. -- 2023-11-27T15:20:58.619 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:20:58.676 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:20:58 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:20:58.420+0000 7fd6bffa7640 -1 osd.7 0 log_to_monitors true 2023-11-27T15:20:58.676 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:58 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:20:58.676 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:58 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:20:58.677 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:58 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:20:58.677 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:58 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:58.677 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:58 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:58.677 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:58 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:58.677 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:58 smithi029 ceph-mon[131166]: from='osd.7 [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:20:58.677 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:58 smithi029 ceph-mon[131166]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:20:58.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:58 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:20:58.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:58 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:20:58.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:58 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:20:58.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:58 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:58.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:58 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:58 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:58 smithi060 ceph-mon[131518]: from='osd.7 [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:20:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:58 smithi060 ceph-mon[131518]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:20:58.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:58 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:20:58.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:58 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:20:58.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:58 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:20:58.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:58 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:58.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:58 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:20:58.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:58 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:58.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:58 smithi008 ceph-mon[128788]: from='osd.7 [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:20:58.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:58 smithi008 ceph-mon[128788]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:20:59.791 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:59 smithi029 ceph-mon[131166]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:59.791 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:59 smithi029 ceph-mon[131166]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-27T15:20:59.791 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:59 smithi029 ceph-mon[131166]: osdmap e50: 8 total, 7 up, 8 in 2023-11-27T15:20:59.791 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:59 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:20:59.791 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:59 smithi029 ceph-mon[131166]: from='osd.7 [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:20:59.792 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:20:59 smithi029 ceph-mon[131166]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:20:59.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:59 smithi060 ceph-mon[131518]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:59.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:59 smithi060 ceph-mon[131518]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-27T15:20:59.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:59 smithi060 ceph-mon[131518]: osdmap e50: 8 total, 7 up, 8 in 2023-11-27T15:20:59.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:59 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:20:59.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:59 smithi060 ceph-mon[131518]: from='osd.7 [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:20:59.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:20:59 smithi060 ceph-mon[131518]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:20:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:59 smithi008 ceph-mon[128788]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:20:59.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:59 smithi008 ceph-mon[128788]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-27T15:20:59.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:59 smithi008 ceph-mon[128788]: osdmap e50: 8 total, 7 up, 8 in 2023-11-27T15:20:59.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:59 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:20:59.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:59 smithi008 ceph-mon[128788]: from='osd.7 [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:20:59.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:20:59 smithi008 ceph-mon[128788]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:21:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-27T15:21:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-27T15:21:00.084 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2023-11-27T15:21:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2023-11-27T15:21:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00796656 s, 1.3 GB/s 2023-11-27T15:21:00.084 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2023-11-27T15:21:00.480 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_2 2023-11-27T15:21:00.648 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:21:00 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:21:00.195+0000 7fd6b561e700 -1 osd.7 0 waiting for initial osdmap 2023-11-27T15:21:00.648 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:21:00 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:21:00.205+0000 7fd6ad40b700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:21:01.137 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:21:01.489 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:21:01.489 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: osdmap e51: 8 total, 7 up, 8 in 2023-11-27T15:21:01.489 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:21:01.489 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:21:01.489 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:21:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi029 2023-11-27T15:21:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi029 to 5724M 2023-11-27T15:21:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:01.491 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:01.491 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:21:01.491 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.491 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:21:01.491 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:01 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:21:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:21:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: osdmap e51: 8 total, 7 up, 8 in 2023-11-27T15:21:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:21:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:21:01.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:21:01.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi029 2023-11-27T15:21:01.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:01.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:01.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:01.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi029 to 5724M 2023-11-27T15:21:01.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:01.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:01.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:21:01.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:21:01.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:01 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:21:01.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:21:01.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: osdmap e51: 8 total, 7 up, 8 in 2023-11-27T15:21:01.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:21:01.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:21:01.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:21:01.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi029 2023-11-27T15:21:01.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:01.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:01.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:01.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi029 to 5724M 2023-11-27T15:21:01.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:01.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:01.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:21:01.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:01.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:21:01.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:01 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:21:02.281 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:02 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:21:02.281 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:02 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:21:02.281 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:02 smithi060 ceph-mon[131518]: osd.7 [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] boot 2023-11-27T15:21:02.282 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:02 smithi060 ceph-mon[131518]: osdmap e52: 8 total, 8 up, 8 in 2023-11-27T15:21:02.282 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:02 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:21:02.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 -- 172.21.15.60:0/3802694291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e041003e0 msgr2=0x7f5e04100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 --2- 172.21.15.60:0/3802694291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e041003e0 0x7f5e04100830 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009990 tx=0x7f5df8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 -- 172.21.15.60:0/3802694291 shutdown_connections 2023-11-27T15:21:02.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 --2- 172.21.15.60:0/3802694291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e04100d70 0x7f5e04107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.448 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 --2- 172.21.15.60:0/3802694291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e040ff1e0 0x7f5e040ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 --2- 172.21.15.60:0/3802694291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e041003e0 0x7f5e04100830 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 -- 172.21.15.60:0/3802694291 >> 172.21.15.60:0/3802694291 conn(0x7f5e040fa7f0 msgr2=0x7f5e040fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:02.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 -- 172.21.15.60:0/3802694291 shutdown_connections 2023-11-27T15:21:02.449 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 -- 172.21.15.60:0/3802694291 wait complete. 2023-11-27T15:21:02.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 Processor -- start 2023-11-27T15:21:02.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.444+0000 7f5e0a293700 1 -- start start 2023-11-27T15:21:02.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e0a293700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e040ff1e0 0x7f5e041960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.452 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e0a293700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e041003e0 0x7f5e04196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e0a293700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e04100d70 0x7f5e0419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e0a293700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0419dc70 con 0x7f5e040ff1e0 2023-11-27T15:21:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e0a293700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0419ddb0 con 0x7f5e04100d70 2023-11-27T15:21:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e0a293700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0419df40 con 0x7f5e041003e0 2023-11-27T15:21:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e037fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e041003e0 0x7f5e04196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e037fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e041003e0 0x7f5e04196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:34998/0 (socket says 172.21.15.60:34998) 2023-11-27T15:21:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e037fe700 1 -- 172.21.15.60:0/3749271646 learned_addr learned my addr 172.21.15.60:0/3749271646 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-27T15:21:02.453 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e03fff700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e040ff1e0 0x7f5e041960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e08830700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e04100d70 0x7f5e0419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e037fe700 1 -- 172.21.15.60:0/3749271646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e04100d70 msgr2=0x7f5e0419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e037fe700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e04100d70 0x7f5e0419d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e037fe700 1 -- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e040ff1e0 msgr2=0x7f5e041960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e037fe700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e040ff1e0 0x7f5e041960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e037fe700 1 -- 172.21.15.60:0/3749271646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df80095f0 con 0x7f5e041003e0 2023-11-27T15:21:02.454 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e03fff700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e040ff1e0 0x7f5e041960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:21:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e08830700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e04100d70 0x7f5e0419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:21:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e037fe700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e041003e0 0x7f5e04196620 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5df80059d0 tx=0x7f5df800f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e017fa700 1 -- 172.21.15.60:0/3749271646 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df801d070 con 0x7f5e041003e0 2023-11-27T15:21:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e017fa700 1 -- 172.21.15.60:0/3749271646 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5df800fc60 con 0x7f5e041003e0 2023-11-27T15:21:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e0a293700 1 -- 172.21.15.60:0/3749271646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0410bce0 con 0x7f5e041003e0 2023-11-27T15:21:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e017fa700 1 -- 172.21.15.60:0/3749271646 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df80176e0 con 0x7f5e041003e0 2023-11-27T15:21:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.445+0000 7f5e0a293700 1 -- 172.21.15.60:0/3749271646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0410bb50 con 0x7f5e041003e0 2023-11-27T15:21:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.446+0000 7f5e0a293700 1 -- 172.21.15.60:0/3749271646 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e04066ea0 con 0x7f5e041003e0 2023-11-27T15:21:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.448+0000 7f5e017fa700 1 -- 172.21.15.60:0/3749271646 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5df80229c0 con 0x7f5e041003e0 2023-11-27T15:21:02.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.448+0000 7f5e017fa700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5dec0a1150 0x7f5dec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.456 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.448+0000 7f5e017fa700 1 -- 172.21.15.60:0/3749271646 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f5df80cd760 con 0x7f5e041003e0 2023-11-27T15:21:02.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.449+0000 7f5e03fff700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5dec0a1150 0x7f5dec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.449+0000 7f5e03fff700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5dec0a1150 0x7f5dec0a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5df4005fd0 tx=0x7f5df4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:02.457 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.452+0000 7f5e017fa700 1 -- 172.21.15.60:0/3749271646 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5df8097a80 con 0x7f5e041003e0 2023-11-27T15:21:02.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:02 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:21:02.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:02 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:21:02.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:02 smithi008 ceph-mon[128788]: osd.7 [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] boot 2023-11-27T15:21:02.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:02 smithi008 ceph-mon[128788]: osdmap e52: 8 total, 8 up, 8 in 2023-11-27T15:21:02.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:02 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:21:02.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:02 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:21:02.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:02 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:21:02.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:02 smithi029 ceph-mon[131166]: osd.7 [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] boot 2023-11-27T15:21:02.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:02 smithi029 ceph-mon[131166]: osdmap e52: 8 total, 8 up, 8 in 2023-11-27T15:21:02.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:02 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:21:02.660 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:02.656+0000 7f5e0a293700 1 -- 172.21.15.60:0/3749271646 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e04060f30 con 0x7f5dec0a1150 2023-11-27T15:21:03.571 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:03 smithi060 ceph-mon[131518]: osdmap e53: 8 total, 8 up, 8 in 2023-11-27T15:21:03.571 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:03 smithi060 ceph-mon[131518]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:03.571 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:21:03.571 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:21:03.572 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:03.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:03 smithi008 ceph-mon[128788]: osdmap e53: 8 total, 8 up, 8 in 2023-11-27T15:21:03.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:03 smithi008 ceph-mon[128788]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:03.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:21:03.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:21:03.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:03.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:03 smithi029 ceph-mon[131166]: osdmap e53: 8 total, 8 up, 8 in 2023-11-27T15:21:03.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:03 smithi029 ceph-mon[131166]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:03.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:21:03.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:21:03.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:04.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:04 smithi060 ceph-mon[131518]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:04.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:04 smithi060 ceph-mon[131518]: osdmap e54: 8 total, 8 up, 8 in 2023-11-27T15:21:04.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:04 smithi008 ceph-mon[128788]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:04.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:04 smithi008 ceph-mon[128788]: osdmap e54: 8 total, 8 up, 8 in 2023-11-27T15:21:04.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:04 smithi029 ceph-mon[131166]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:04.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:04 smithi029 ceph-mon[131166]: osdmap e54: 8 total, 8 up, 8 in 2023-11-27T15:21:05.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:05 smithi060 ceph-mon[131518]: osdmap e55: 8 total, 8 up, 8 in 2023-11-27T15:21:05.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:05 smithi060 ceph-mon[131518]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:05.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:05 smithi060 ceph-mon[131518]: from='client.? 172.21.15.60:0/2828428210' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "addef24e-8aa7-4f3d-be94-9decd5f22f87"}]: dispatch 2023-11-27T15:21:05.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:05 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "addef24e-8aa7-4f3d-be94-9decd5f22f87"}]: dispatch 2023-11-27T15:21:05.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:05 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "addef24e-8aa7-4f3d-be94-9decd5f22f87"}]': finished 2023-11-27T15:21:05.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:05 smithi060 ceph-mon[131518]: osdmap e56: 9 total, 8 up, 9 in 2023-11-27T15:21:05.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:05.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:05 smithi008 ceph-mon[128788]: osdmap e55: 8 total, 8 up, 8 in 2023-11-27T15:21:05.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:05 smithi008 ceph-mon[128788]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:05.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:05 smithi008 ceph-mon[128788]: from='client.? 172.21.15.60:0/2828428210' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "addef24e-8aa7-4f3d-be94-9decd5f22f87"}]: dispatch 2023-11-27T15:21:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:05 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "addef24e-8aa7-4f3d-be94-9decd5f22f87"}]: dispatch 2023-11-27T15:21:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:05 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "addef24e-8aa7-4f3d-be94-9decd5f22f87"}]': finished 2023-11-27T15:21:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:05 smithi008 ceph-mon[128788]: osdmap e56: 9 total, 8 up, 9 in 2023-11-27T15:21:05.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:05.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:05 smithi029 ceph-mon[131166]: osdmap e55: 8 total, 8 up, 8 in 2023-11-27T15:21:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:05 smithi029 ceph-mon[131166]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:05 smithi029 ceph-mon[131166]: from='client.? 172.21.15.60:0/2828428210' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "addef24e-8aa7-4f3d-be94-9decd5f22f87"}]: dispatch 2023-11-27T15:21:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:05 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "addef24e-8aa7-4f3d-be94-9decd5f22f87"}]: dispatch 2023-11-27T15:21:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:05 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "addef24e-8aa7-4f3d-be94-9decd5f22f87"}]': finished 2023-11-27T15:21:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:05 smithi029 ceph-mon[131166]: osdmap e56: 9 total, 8 up, 9 in 2023-11-27T15:21:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:06.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:06 smithi060 ceph-mon[131518]: from='client.? 172.21.15.60:0/2355240584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:21:06.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:06 smithi008 ceph-mon[128788]: from='client.? 172.21.15.60:0/2355240584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:21:06.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:06 smithi029 ceph-mon[131166]: from='client.? 172.21.15.60:0/2355240584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:21:07.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:07 smithi060 ceph-mon[131518]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:07.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:07 smithi008 ceph-mon[128788]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:07.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:07 smithi029 ceph-mon[131166]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:09.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:09 smithi060 ceph-mon[131518]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:09.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:09 smithi008 ceph-mon[128788]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:09.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:09 smithi029 ceph-mon[131166]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:11.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:11 smithi008 ceph-mon[128788]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:11.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:11 smithi029 ceph-mon[131166]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:11.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:11 smithi060 ceph-mon[131518]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:12.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-27T15:21:12.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:12.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-27T15:21:12.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:12.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-27T15:21:12.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:13.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:13 smithi060 ceph-mon[131518]: Deploying daemon osd.8 on smithi060 2023-11-27T15:21:13.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:13 smithi060 ceph-mon[131518]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:13.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:13 smithi008 ceph-mon[128788]: Deploying daemon osd.8 on smithi060 2023-11-27T15:21:13.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:13 smithi008 ceph-mon[128788]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:13.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:13 smithi029 ceph-mon[131166]: Deploying daemon osd.8 on smithi060 2023-11-27T15:21:13.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:13 smithi029 ceph-mon[131166]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:15 smithi029 ceph-mon[131166]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:15.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:15 smithi060 ceph-mon[131518]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:15 smithi008 ceph-mon[128788]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:17.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:17 smithi029 ceph-mon[131166]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:17.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:17 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:17.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:17 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:17.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:17 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:17 smithi060 ceph-mon[131518]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:17 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:17.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:17 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:17.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:17 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:17 smithi008 ceph-mon[128788]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:17 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:17 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:17 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:19.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:19 smithi060 ceph-mon[131518]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:19 smithi008 ceph-mon[128788]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:19 smithi029 ceph-mon[131166]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:20.479 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 8 on host 'smithi060' 2023-11-27T15:21:20.479 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.474+0000 7f5e017fa700 1 -- 172.21.15.60:0/3749271646 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5e04060f30 con 0x7f5dec0a1150 2023-11-27T15:21:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 -- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5dec0a1150 msgr2=0x7f5dec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5dec0a1150 0x7f5dec0a3600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5df4005fd0 tx=0x7f5df4005dc0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 -- 172.21.15.60:0/3749271646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e041003e0 msgr2=0x7f5e04196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:20.480 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e041003e0 0x7f5e04196620 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5df80059d0 tx=0x7f5df800f7b0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 -- 172.21.15.60:0/3749271646 shutdown_connections 2023-11-27T15:21:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5e04100d70 0x7f5e0419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e040ff1e0 0x7f5e041960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5dec0a1150 0x7f5dec0a3600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 --2- 172.21.15.60:0/3749271646 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5e041003e0 0x7f5e04196620 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 -- 172.21.15.60:0/3749271646 >> 172.21.15.60:0/3749271646 conn(0x7f5e040fa7f0 msgr2=0x7f5e040fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 -- 172.21.15.60:0/3749271646 shutdown_connections 2023-11-27T15:21:20.481 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:21:20.476+0000 7f5e0a293700 1 -- 172.21.15.60:0/3749271646 wait complete. 2023-11-27T15:21:20.817 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.817 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:20.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:20.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:21:20.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:21:20.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:21:20.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:20.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:20 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:20.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:20.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:20.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:21:20.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:21:20.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:21:20.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:20.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:20 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:20.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:20.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:20.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:21:20.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:21:20.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:21:20.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:20.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:20.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:20 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:20.923 DEBUG:teuthology.orchestra.run.smithi060:osd.8> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.8.service 2023-11-27T15:21:20.926 INFO:tasks.cephadm:Deploying osd.9 on smithi008 with /dev/vg_nvme/lv_1... 2023-11-27T15:21:20.927 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-27T15:21:20.956 INFO:journalctl@ceph.osd.8.smithi060.stdout:-- Logs begin at Mon 2023-11-27 14:51:59 UTC. -- 2023-11-27T15:21:21.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:21:21 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:21:21.007+0000 7fafb2ae5640 -1 osd.8 0 log_to_monitors true 2023-11-27T15:21:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:21 smithi060 ceph-mon[131518]: from='osd.8 [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:21:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:21 smithi060 ceph-mon[131518]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:21:21.802 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:21 smithi029 ceph-mon[131166]: from='osd.8 [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:21:21.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:21 smithi029 ceph-mon[131166]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:21:22.073 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:21 smithi008 ceph-mon[128788]: from='osd.8 [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:21:22.073 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:21 smithi008 ceph-mon[128788]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:21:22.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:22 smithi008 ceph-mon[128788]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-27T15:21:22.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:22 smithi008 ceph-mon[128788]: osdmap e57: 9 total, 8 up, 9 in 2023-11-27T15:21:22.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:22 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:22.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:22 smithi008 ceph-mon[128788]: from='osd.8 [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:21:22.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:22 smithi008 ceph-mon[128788]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:21:22.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:22 smithi008 ceph-mon[128788]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:22.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:22 smithi029 ceph-mon[131166]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-27T15:21:22.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:22 smithi029 ceph-mon[131166]: osdmap e57: 9 total, 8 up, 9 in 2023-11-27T15:21:22.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:22 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:22.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:22 smithi029 ceph-mon[131166]: from='osd.8 [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:21:22.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:22 smithi029 ceph-mon[131166]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:21:22.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:22 smithi029 ceph-mon[131166]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:22.905 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:21:22 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:21:22.631+0000 7fafa895d700 -1 osd.8 0 waiting for initial osdmap 2023-11-27T15:21:22.905 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:21:22 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:21:22.642+0000 7fafa1f4d700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:21:22.905 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:22 smithi060 ceph-mon[131518]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-27T15:21:22.906 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:22 smithi060 ceph-mon[131518]: osdmap e57: 9 total, 8 up, 9 in 2023-11-27T15:21:22.906 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:22 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:22.906 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:22 smithi060 ceph-mon[131518]: from='osd.8 [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:21:22.906 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:22 smithi060 ceph-mon[131518]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:21:22.906 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:22 smithi060 ceph-mon[131518]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:21:23.775 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-27T15:21:23.775 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-27T15:21:23.775 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2023-11-27T15:21:23.776 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2023-11-27T15:21:23.776 INFO:teuthology.orchestra.run.smithi008.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0311457 s, 337 MB/s 2023-11-27T15:21:23.776 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 2023-11-27T15:21:23.776 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2023-11-27T15:21:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:21:23.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: osdmap e58: 9 total, 8 up, 9 in 2023-11-27T15:21:23.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:23.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:23.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: osd.8 [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] boot 2023-11-27T15:21:23.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: osdmap e59: 9 total, 9 up, 9 in 2023-11-27T15:21:23.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:23.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi060 2023-11-27T15:21:23.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:23.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:23.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:23.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:23.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:23.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi060 to 5724M 2023-11-27T15:21:23.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:23.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:23.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:23.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:21:23.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:23.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:21:23.900 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:23 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:21:23.944 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:21:23.944 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: osdmap e58: 9 total, 8 up, 9 in 2023-11-27T15:21:23.944 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:23.944 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:23.944 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: osd.8 [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] boot 2023-11-27T15:21:23.944 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: osdmap e59: 9 total, 9 up, 9 in 2023-11-27T15:21:23.944 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:23.944 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi060 2023-11-27T15:21:23.944 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:23.945 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:23.945 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:23.945 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:23.945 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:23.945 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi060 to 5724M 2023-11-27T15:21:23.945 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:23.945 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:23.945 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:23.945 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:21:23.945 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:23.946 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:21:23.946 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:23 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:21:24.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: osdmap e58: 9 total, 8 up, 9 in 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: osd.8 [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] boot 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: osdmap e59: 9 total, 9 up, 9 in 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi060 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:24.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:24.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:24.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi060 to 5724M 2023-11-27T15:21:24.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:24.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:24.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:24.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:21:24.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:24.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:21:24.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:23 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:21:24.287 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi008:vg_nvme/lv_1 2023-11-27T15:21:24.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:24 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:21:24.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:24 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:21:24.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:24 smithi029 ceph-mon[131166]: osdmap e60: 9 total, 9 up, 9 in 2023-11-27T15:21:24.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:24 smithi029 ceph-mon[131166]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:25.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:24 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:21:25.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:24 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:21:25.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:24 smithi060 ceph-mon[131518]: osdmap e60: 9 total, 9 up, 9 in 2023-11-27T15:21:25.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:24 smithi060 ceph-mon[131518]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:25.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:24 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:21:25.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:24 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:21:25.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:24 smithi008 ceph-mon[128788]: osdmap e60: 9 total, 9 up, 9 in 2023-11-27T15:21:25.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:24 smithi008 ceph-mon[128788]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:25.150 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:26.505 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:26 smithi008 ceph-mon[128788]: osdmap e61: 9 total, 9 up, 9 in 2023-11-27T15:21:26.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:26 smithi060 ceph-mon[131518]: osdmap e61: 9 total, 9 up, 9 in 2023-11-27T15:21:26.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:26 smithi029 ceph-mon[131166]: osdmap e61: 9 total, 9 up, 9 in 2023-11-27T15:21:26.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.823+0000 7fd1ea703700 1 -- 172.21.15.8:0/3684372184 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e41002f0 msgr2=0x7fd1e4100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:26.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.823+0000 7fd1ea703700 1 --2- 172.21.15.8:0/3684372184 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e41002f0 0x7fd1e4100700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009990 tx=0x7fd1d4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:26.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.824+0000 7fd1ea703700 1 -- 172.21.15.8:0/3684372184 shutdown_connections 2023-11-27T15:21:26.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.824+0000 7fd1ea703700 1 --2- 172.21.15.8:0/3684372184 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1e4101e80 0x7fd1e4108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:26.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.824+0000 7fd1ea703700 1 --2- 172.21.15.8:0/3684372184 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e41002f0 0x7fd1e4100700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:26.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.824+0000 7fd1ea703700 1 --2- 172.21.15.8:0/3684372184 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41014f0 0x7fd1e4101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:26.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.824+0000 7fd1ea703700 1 -- 172.21.15.8:0/3684372184 >> 172.21.15.8:0/3684372184 conn(0x7fd1e40fb900 msgr2=0x7fd1e40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:26.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.824+0000 7fd1ea703700 1 -- 172.21.15.8:0/3684372184 shutdown_connections 2023-11-27T15:21:26.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.824+0000 7fd1ea703700 1 -- 172.21.15.8:0/3684372184 wait complete. 2023-11-27T15:21:26.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1ea703700 1 Processor -- start 2023-11-27T15:21:26.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1ea703700 1 -- start start 2023-11-27T15:21:26.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1ea703700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41002f0 0x7fd1e4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:26.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1ea703700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1e41014f0 0x7fd1e41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:26.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1ea703700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e4101e80 0x7fd1e419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:26.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1ea703700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e419fe20 con 0x7fd1e4101e80 2023-11-27T15:21:26.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1ea703700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e419ff60 con 0x7fd1e41002f0 2023-11-27T15:21:26.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1ea703700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e41a00f0 con 0x7fd1e41014f0 2023-11-27T15:21:26.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41002f0 0x7fd1e4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:26.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1e3fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41002f0 0x7fd1e4198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:33014/0 (socket says 172.21.15.8:33014) 2023-11-27T15:21:26.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.825+0000 7fd1e3fff700 1 -- 172.21.15.8:0/2432874931 learned_addr learned my addr 172.21.15.8:0/2432874931 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:21:26.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.826+0000 7fd1e37fe700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1e41014f0 0x7fd1e41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:26.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.826+0000 7fd1e8ca0700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e4101e80 0x7fd1e419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:26.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.826+0000 7fd1e3fff700 1 -- 172.21.15.8:0/2432874931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1e41014f0 msgr2=0x7fd1e41987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:26.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.826+0000 7fd1e3fff700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1e41014f0 0x7fd1e41987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:26.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.826+0000 7fd1e3fff700 1 -- 172.21.15.8:0/2432874931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e4101e80 msgr2=0x7fd1e419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:26.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.826+0000 7fd1e3fff700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e4101e80 0x7fd1e419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:26.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.826+0000 7fd1e3fff700 1 -- 172.21.15.8:0/2432874931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d40095f0 con 0x7fd1e41002f0 2023-11-27T15:21:26.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.826+0000 7fd1e3fff700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41002f0 0x7fd1e4198290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4004710 tx=0x7fd1d4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:26.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.826+0000 7fd1e17fa700 1 -- 172.21.15.8:0/2432874931 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d401d070 con 0x7fd1e41002f0 2023-11-27T15:21:26.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.826+0000 7fd1ea703700 1 -- 172.21.15.8:0/2432874931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e410bc90 con 0x7fd1e41002f0 2023-11-27T15:21:26.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.827+0000 7fd1e17fa700 1 -- 172.21.15.8:0/2432874931 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1d4021410 con 0x7fd1e41002f0 2023-11-27T15:21:26.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.827+0000 7fd1ea703700 1 -- 172.21.15.8:0/2432874931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e410bb00 con 0x7fd1e41002f0 2023-11-27T15:21:26.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.827+0000 7fd1e17fa700 1 -- 172.21.15.8:0/2432874931 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4017650 con 0x7fd1e41002f0 2023-11-27T15:21:26.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.828+0000 7fd1ea703700 1 -- 172.21.15.8:0/2432874931 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e41929f0 con 0x7fd1e41002f0 2023-11-27T15:21:26.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.830+0000 7fd1e17fa700 1 -- 172.21.15.8:0/2432874931 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd1d4021580 con 0x7fd1e41002f0 2023-11-27T15:21:26.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.831+0000 7fd1e17fa700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd1cc0a11d0 0x7fd1cc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:26.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.831+0000 7fd1e17fa700 1 -- 172.21.15.8:0/2432874931 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5829+0+0 (secure 0 0 0) 0x7fd1d40cd520 con 0x7fd1e41002f0 2023-11-27T15:21:26.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.831+0000 7fd1e37fe700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd1cc0a11d0 0x7fd1cc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:26.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.832+0000 7fd1e37fe700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd1cc0a11d0 0x7fd1cc0a3680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009c60 tx=0x7fd1d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:26.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:26.833+0000 7fd1e17fa700 1 -- 172.21.15.8:0/2432874931 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd1d4097610 con 0x7fd1e41002f0 2023-11-27T15:21:27.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:27.050+0000 7fd1ea703700 1 -- 172.21.15.8:0/2432874931 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1e4061c60 con 0x7fd1cc0a11d0 2023-11-27T15:21:27.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:27 smithi060 ceph-mon[131518]: osdmap e62: 9 total, 9 up, 9 in 2023-11-27T15:21:27.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:27 smithi060 ceph-mon[131518]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:27.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:27 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:21:27.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:27 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:21:27.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:27 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:27.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:27 smithi008 ceph-mon[128788]: osdmap e62: 9 total, 9 up, 9 in 2023-11-27T15:21:27.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:27 smithi008 ceph-mon[128788]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:27.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:27 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:21:27.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:27 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:21:27.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:27 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:27.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:27 smithi029 ceph-mon[131166]: osdmap e62: 9 total, 9 up, 9 in 2023-11-27T15:21:27.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:27 smithi029 ceph-mon[131166]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:27.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:27 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:21:27.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:27 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:21:27.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:27 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:28.560 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:28 smithi008 ceph-mon[128788]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:28.561 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:28 smithi008 ceph-mon[128788]: osdmap e63: 9 total, 9 up, 9 in 2023-11-27T15:21:28.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:28 smithi060 ceph-mon[131518]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:28.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:28 smithi060 ceph-mon[131518]: osdmap e63: 9 total, 9 up, 9 in 2023-11-27T15:21:28.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:28 smithi029 ceph-mon[131166]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:28.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:28 smithi029 ceph-mon[131166]: osdmap e63: 9 total, 9 up, 9 in 2023-11-27T15:21:29.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:29 smithi060 ceph-mon[131518]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail; 140 KiB/s, 0 objects/s recovering 2023-11-27T15:21:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:29 smithi008 ceph-mon[128788]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail; 140 KiB/s, 0 objects/s recovering 2023-11-27T15:21:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:29 smithi029 ceph-mon[131166]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail; 140 KiB/s, 0 objects/s recovering 2023-11-27T15:21:30.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:30 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1257987663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0"}]: dispatch 2023-11-27T15:21:30.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:30 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1257987663' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0"}]': finished 2023-11-27T15:21:30.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:30 smithi060 ceph-mon[131518]: osdmap e64: 10 total, 9 up, 10 in 2023-11-27T15:21:30.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:30 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:30.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:30 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1257987663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0"}]: dispatch 2023-11-27T15:21:30.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:30 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1257987663' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0"}]': finished 2023-11-27T15:21:30.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:30 smithi008 ceph-mon[128788]: osdmap e64: 10 total, 9 up, 10 in 2023-11-27T15:21:30.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:30 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:30.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:30 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1257987663' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0"}]: dispatch 2023-11-27T15:21:30.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:30 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1257987663' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0"}]': finished 2023-11-27T15:21:30.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:30 smithi029 ceph-mon[131166]: osdmap e64: 10 total, 9 up, 10 in 2023-11-27T15:21:30.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:30 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:31.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:31 smithi060 ceph-mon[131518]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-11-27T15:21:31.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:31 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/885741461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:21:31.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:31 smithi008 ceph-mon[128788]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-11-27T15:21:31.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:31 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/885741461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:21:31.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:31 smithi029 ceph-mon[131166]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-11-27T15:21:31.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:31 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/885741461' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:21:33.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:33 smithi008 ceph-mon[128788]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-27T15:21:33.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:33 smithi029 ceph-mon[131166]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-27T15:21:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:33 smithi060 ceph-mon[131518]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-27T15:21:35.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:35 smithi008 ceph-mon[128788]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:21:35.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:35 smithi029 ceph-mon[131166]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:21:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:35 smithi060 ceph-mon[131518]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:21:37.642 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:37 smithi008 ceph-mon[128788]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:37.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:37 smithi029 ceph-mon[131166]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:37 smithi060 ceph-mon[131518]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:38.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-27T15:21:38.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:38.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-27T15:21:38.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:38.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-27T15:21:38.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:39 smithi008 ceph-mon[128788]: Deploying daemon osd.9 on smithi008 2023-11-27T15:21:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:39 smithi008 ceph-mon[128788]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:39.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:39 smithi029 ceph-mon[131166]: Deploying daemon osd.9 on smithi008 2023-11-27T15:21:39.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:39 smithi029 ceph-mon[131166]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:39 smithi060 ceph-mon[131518]: Deploying daemon osd.9 on smithi008 2023-11-27T15:21:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:39 smithi060 ceph-mon[131518]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:41.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:41 smithi029 ceph-mon[131166]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:41.720 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:41 smithi008 ceph-mon[128788]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:41 smithi060 ceph-mon[131518]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:43.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:43 smithi008 ceph-mon[128788]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:43.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:43 smithi029 ceph-mon[131166]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:43 smithi060 ceph-mon[131518]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:45 smithi060 ceph-mon[131518]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:45.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:45.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:45.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:45.829 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:45 smithi008 ceph-mon[128788]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:45.830 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:45.830 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:45.830 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:45 smithi029 ceph-mon[131166]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:47 smithi060 ceph-mon[131518]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:47 smithi008 ceph-mon[128788]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:47 smithi029 ceph-mon[131166]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:48.363 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.358+0000 7fd1e17fa700 1 -- 172.21.15.8:0/2432874931 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd1e4061c60 con 0x7fd1cc0a11d0 2023-11-27T15:21:48.364 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 9 on host 'smithi008' 2023-11-27T15:21:48.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 -- 172.21.15.8:0/2432874931 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd1cc0a11d0 msgr2=0x7fd1cc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:48.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd1cc0a11d0 0x7fd1cc0a3680 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009c60 tx=0x7fd1d8009260 comp rx=0 tx=0).stop 2023-11-27T15:21:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 -- 172.21.15.8:0/2432874931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41002f0 msgr2=0x7fd1e4198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41002f0 0x7fd1e4198290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4004710 tx=0x7fd1d4009580 comp rx=0 tx=0).stop 2023-11-27T15:21:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 -- 172.21.15.8:0/2432874931 shutdown_connections 2023-11-27T15:21:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd1e4101e80 0x7fd1e419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd1e41002f0 0x7fd1e4198290 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd1cc0a11d0 0x7fd1cc0a3680 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 --2- 172.21.15.8:0/2432874931 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd1e41014f0 0x7fd1e41987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 -- 172.21.15.8:0/2432874931 >> 172.21.15.8:0/2432874931 conn(0x7fd1e40fb900 msgr2=0x7fd1e4106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 -- 172.21.15.8:0/2432874931 shutdown_connections 2023-11-27T15:21:48.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:21:48.361+0000 7fd1ea703700 1 -- 172.21.15.8:0/2432874931 wait complete. 2023-11-27T15:21:48.889 DEBUG:teuthology.orchestra.run.smithi008:osd.9> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.9.service 2023-11-27T15:21:48.893 INFO:tasks.cephadm:Deploying osd.10 on smithi029 with /dev/vg_nvme/lv_1... 2023-11-27T15:21:48.893 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-27T15:21:48.967 INFO:journalctl@ceph.osd.9.smithi008.stdout:-- Logs begin at Mon 2023-11-27 14:56:10 UTC. -- 2023-11-27T15:21:49.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:21:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:21:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:21:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:49.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:48 smithi060 ceph-mon[131518]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:21:49.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:49.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:48 smithi008 ceph-mon[128788]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:21:49.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:49.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:21:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:21:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:21:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:48 smithi029 ceph-mon[131166]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:21:49.556 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:21:50.583 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:50 smithi029 ceph-mon[131166]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-27T15:21:50.584 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:50 smithi029 ceph-mon[131166]: osdmap e65: 10 total, 9 up, 10 in 2023-11-27T15:21:50.584 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:50.584 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:50 smithi029 ceph-mon[131166]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:21:50.647 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:50 smithi008 ceph-mon[128788]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-27T15:21:50.647 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:50 smithi008 ceph-mon[128788]: osdmap e65: 10 total, 9 up, 10 in 2023-11-27T15:21:50.648 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:50.648 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:50 smithi008 ceph-mon[128788]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:21:50.648 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:21:50 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:21:50.366+0000 7f6a2baab700 -1 osd.9 0 waiting for initial osdmap 2023-11-27T15:21:50.648 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:21:50 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:21:50.384+0000 7f6a24099700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:21:50.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:50 smithi060 ceph-mon[131518]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-27T15:21:50.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:50 smithi060 ceph-mon[131518]: osdmap e65: 10 total, 9 up, 10 in 2023-11-27T15:21:50.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:50.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:50 smithi060 ceph-mon[131518]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2023-11-27T15:21:51.044 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-27T15:21:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-27T15:21:51.045 INFO:teuthology.orchestra.run.smithi029.stderr: stderr: 10+0 records in 2023-11-27T15:21:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:10+0 records out 2023-11-27T15:21:51.045 INFO:teuthology.orchestra.run.smithi029.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00810752 s, 1.3 GB/s 2023-11-27T15:21:51.045 INFO:teuthology.orchestra.run.smithi029.stderr:--> Zapping successful for: 2023-11-27T15:21:51.415 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi029:vg_nvme/lv_1 2023-11-27T15:21:51.640 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:51 smithi008 ceph-mon[128788]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:51.641 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:51 smithi008 ceph-mon[128788]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:21:51.641 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:51 smithi008 ceph-mon[128788]: osdmap e66: 10 total, 9 up, 10 in 2023-11-27T15:21:51.641 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:51.641 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:51.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:51 smithi029 ceph-mon[131166]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:51.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:51 smithi029 ceph-mon[131166]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:21:51.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:51 smithi029 ceph-mon[131166]: osdmap e66: 10 total, 9 up, 10 in 2023-11-27T15:21:51.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:51.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:51 smithi060 ceph-mon[131518]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:21:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:51 smithi060 ceph-mon[131518]: from='osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2023-11-27T15:21:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:51 smithi060 ceph-mon[131518]: osdmap e66: 10 total, 9 up, 10 in 2023-11-27T15:21:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:51.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:52.070 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:21:52.561 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:21:52.561 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:21:52.561 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] boot 2023-11-27T15:21:52.561 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: osdmap e67: 10 total, 10 up, 10 in 2023-11-27T15:21:52.561 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:52.561 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.561 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.561 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.561 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.562 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.562 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.562 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.562 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:52.562 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:52.562 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:21:52.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:21:52.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] boot 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: osdmap e67: 10 total, 10 up, 10 in 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:52.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:21:52.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:21:52.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: osd.9 [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] boot 2023-11-27T15:21:52.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: osdmap e67: 10 total, 10 up, 10 in 2023-11-27T15:21:52.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:21:52.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:21:52.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:52.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:52.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:52.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:21:53.255 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.247+0000 7f4e06eef700 1 -- 172.21.15.29:0/212176001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e000ff1e0 msgr2=0x7f4e000ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:53.255 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.247+0000 7f4e06eef700 1 --2- 172.21.15.29:0/212176001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e000ff1e0 0x7f4e000ff5f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009990 tx=0x7f4df4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.255 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.247+0000 7f4e06eef700 1 -- 172.21.15.29:0/212176001 shutdown_connections 2023-11-27T15:21:53.255 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.247+0000 7f4e06eef700 1 --2- 172.21.15.29:0/212176001 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e00100d70 0x7f4e00107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.255 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.247+0000 7f4e06eef700 1 --2- 172.21.15.29:0/212176001 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e000ff1e0 0x7f4e000ff5f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.256 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.247+0000 7f4e06eef700 1 --2- 172.21.15.29:0/212176001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e001003e0 0x7f4e00100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.256 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.247+0000 7f4e06eef700 1 -- 172.21.15.29:0/212176001 >> 172.21.15.29:0/212176001 conn(0x7f4e000fa7f0 msgr2=0x7f4e000fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:53.256 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.247+0000 7f4e06eef700 1 -- 172.21.15.29:0/212176001 shutdown_connections 2023-11-27T15:21:53.256 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.247+0000 7f4e06eef700 1 -- 172.21.15.29:0/212176001 wait complete. 2023-11-27T15:21:53.256 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e06eef700 1 Processor -- start 2023-11-27T15:21:53.256 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e06eef700 1 -- start start 2023-11-27T15:21:53.256 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e06eef700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e000ff1e0 0x7f4e00069170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:53.257 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e06eef700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e001003e0 0x7f4e0019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:53.257 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e06eef700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e00100d70 0x7f4e0019d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:53.257 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e06eef700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0019dcb0 con 0x7f4e001003e0 2023-11-27T15:21:53.257 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e06eef700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0019de20 con 0x7f4e00100d70 2023-11-27T15:21:53.257 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e06eef700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0019dfe0 con 0x7f4e000ff1e0 2023-11-27T15:21:53.258 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e0548c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e00100d70 0x7f4e0019d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:53.258 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e0548c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e00100d70 0x7f4e0019d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.29:35326/0 (socket says 172.21.15.29:35326) 2023-11-27T15:21:53.258 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e0548c700 1 -- 172.21.15.29:0/3907630148 learned_addr learned my addr 172.21.15.29:0/3907630148 (peer_addr_for_me v2:172.21.15.29:0/0) 2023-11-27T15:21:53.258 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.248+0000 7f4e04c8b700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e000ff1e0 0x7f4e00069170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:53.258 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4e0548c700 1 -- 172.21.15.29:0/3907630148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e000ff1e0 msgr2=0x7f4e00069170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:53.258 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4e0548c700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e000ff1e0 0x7f4e00069170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.258 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4dfffff700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e001003e0 0x7f4e0019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:53.258 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4e0548c700 1 -- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e001003e0 msgr2=0x7f4e0019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:53.258 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4e0548c700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e001003e0 0x7f4e0019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4e0548c700 1 -- 172.21.15.29:0/3907630148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40095f0 con 0x7f4e00100d70 2023-11-27T15:21:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4e04c8b700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e000ff1e0 0x7f4e00069170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4dfffff700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e001003e0 0x7f4e0019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:21:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4e0548c700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e00100d70 0x7f4e0019d510 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b970 tx=0x7f4df800bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4dfdffb700 1 -- 172.21.15.29:0/3907630148 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800c9b0 con 0x7f4e00100d70 2023-11-27T15:21:53.259 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4e06eef700 1 -- 172.21.15.29:0/3907630148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0010ac30 con 0x7f4e00100d70 2023-11-27T15:21:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4dfdffb700 1 -- 172.21.15.29:0/3907630148 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4df8014440 con 0x7f4e00100d70 2023-11-27T15:21:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4dfdffb700 1 -- 172.21.15.29:0/3907630148 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df80075d0 con 0x7f4e00100d70 2023-11-27T15:21:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.249+0000 7f4e06eef700 1 -- 172.21.15.29:0/3907630148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0010aaa0 con 0x7f4e00100d70 2023-11-27T15:21:53.260 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.250+0000 7f4e06eef700 1 -- 172.21.15.29:0/3907630148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0019e750 con 0x7f4e00100d70 2023-11-27T15:21:53.261 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.252+0000 7f4dfdffb700 1 -- 172.21.15.29:0/3907630148 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4df80145b0 con 0x7f4e00100d70 2023-11-27T15:21:53.261 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.253+0000 7f4dfdffb700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4df00a1150 0x7f4df00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:53.261 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.253+0000 7f4dfdffb700 1 -- 172.21.15.29:0/3907630148 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f4df8006e40 con 0x7f4e00100d70 2023-11-27T15:21:53.261 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.253+0000 7f4e04c8b700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4df00a1150 0x7f4df00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:53.262 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.254+0000 7f4e04c8b700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4df00a1150 0x7f4df00a3600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4df400b5c0 tx=0x7f4df4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:53.262 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.256+0000 7f4dfdffb700 1 -- 172.21.15.29:0/3907630148 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4df8095c70 con 0x7f4e00100d70 2023-11-27T15:21:53.468 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:21:53.463+0000 7f4e06eef700 1 -- 172.21.15.29:0/3907630148 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e0019ed00 con 0x7f4df00a1150 2023-11-27T15:21:53.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:53 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi008 2023-11-27T15:21:53.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:53 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi008 to 4319M 2023-11-27T15:21:53.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:53 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:21:53.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:53 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:21:53.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:53 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:21:53.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:53 smithi029 ceph-mon[131166]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:53.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:53 smithi029 ceph-mon[131166]: osdmap e68: 10 total, 10 up, 10 in 2023-11-27T15:21:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:53 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi008 2023-11-27T15:21:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:53 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi008 to 4319M 2023-11-27T15:21:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:53 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:21:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:53 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:21:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:53 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:21:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:53 smithi060 ceph-mon[131518]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:53.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:53 smithi060 ceph-mon[131518]: osdmap e68: 10 total, 10 up, 10 in 2023-11-27T15:21:53.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:53 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi008 2023-11-27T15:21:53.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:53 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi008 to 4319M 2023-11-27T15:21:53.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:53 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:21:53.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:53 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:21:53.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:53 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:21:53.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:53 smithi008 ceph-mon[128788]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:53.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:53 smithi008 ceph-mon[128788]: osdmap e68: 10 total, 10 up, 10 in 2023-11-27T15:21:54.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:54 smithi029 ceph-mon[131166]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:54.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:21:54.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:21:54.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:54.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:54 smithi060 ceph-mon[131518]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:54.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:21:54.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:21:54.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:54.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:54 smithi008 ceph-mon[128788]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi029:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:54.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:21:54.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:21:54.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:55.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:55 smithi029 ceph-mon[131166]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:55.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:55 smithi060 ceph-mon[131518]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:55 smithi008 ceph-mon[128788]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:56.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:56 smithi029 ceph-mon[131166]: from='client.? 172.21.15.29:0/995885221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc133e8-62fc-4f5d-bde0-8454b3292bca"}]: dispatch 2023-11-27T15:21:56.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:56 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc133e8-62fc-4f5d-bde0-8454b3292bca"}]: dispatch 2023-11-27T15:21:56.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:56 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc133e8-62fc-4f5d-bde0-8454b3292bca"}]': finished 2023-11-27T15:21:56.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:56 smithi029 ceph-mon[131166]: osdmap e69: 11 total, 10 up, 11 in 2023-11-27T15:21:56.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:56 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:21:56.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:56 smithi060 ceph-mon[131518]: from='client.? 172.21.15.29:0/995885221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc133e8-62fc-4f5d-bde0-8454b3292bca"}]: dispatch 2023-11-27T15:21:56.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:56 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc133e8-62fc-4f5d-bde0-8454b3292bca"}]: dispatch 2023-11-27T15:21:56.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:56 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc133e8-62fc-4f5d-bde0-8454b3292bca"}]': finished 2023-11-27T15:21:56.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:56 smithi060 ceph-mon[131518]: osdmap e69: 11 total, 10 up, 11 in 2023-11-27T15:21:56.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:56 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:21:56.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:56 smithi008 ceph-mon[128788]: from='client.? 172.21.15.29:0/995885221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc133e8-62fc-4f5d-bde0-8454b3292bca"}]: dispatch 2023-11-27T15:21:56.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:56 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc133e8-62fc-4f5d-bde0-8454b3292bca"}]: dispatch 2023-11-27T15:21:56.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:56 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc133e8-62fc-4f5d-bde0-8454b3292bca"}]': finished 2023-11-27T15:21:56.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:56 smithi008 ceph-mon[128788]: osdmap e69: 11 total, 10 up, 11 in 2023-11-27T15:21:56.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:56 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:21:57.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:57 smithi029 ceph-mon[131166]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:57.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:57 smithi029 ceph-mon[131166]: from='client.? 172.21.15.29:0/401741948' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:21:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:57 smithi060 ceph-mon[131518]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:57 smithi060 ceph-mon[131518]: from='client.? 172.21.15.29:0/401741948' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:21:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:57 smithi008 ceph-mon[128788]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:57 smithi008 ceph-mon[128788]: from='client.? 172.21.15.29:0/401741948' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:21:59.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:21:59 smithi060 ceph-mon[131518]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:21:59 smithi008 ceph-mon[128788]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:21:59.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:21:59 smithi029 ceph-mon[131166]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:01.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:01 smithi029 ceph-mon[131166]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:01 smithi060 ceph-mon[131518]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:01.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:01 smithi008 ceph-mon[128788]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:03.602 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:03 smithi029 ceph-mon[131166]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:03.602 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-27T15:22:03.602 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:03 smithi060 ceph-mon[131518]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:03.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-27T15:22:03.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:03.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:03 smithi008 ceph-mon[128788]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:03.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-27T15:22:03.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:04.553 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:04 smithi029 ceph-mon[131166]: Deploying daemon osd.10 on smithi029 2023-11-27T15:22:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:04 smithi060 ceph-mon[131518]: Deploying daemon osd.10 on smithi029 2023-11-27T15:22:04.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:04 smithi008 ceph-mon[128788]: Deploying daemon osd.10 on smithi029 2023-11-27T15:22:05.595 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:05 smithi029 ceph-mon[131166]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:05 smithi060 ceph-mon[131518]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:05 smithi008 ceph-mon[128788]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:07.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:07 smithi029 ceph-mon[131166]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:07.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:07.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:07 smithi060 ceph-mon[131518]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:07 smithi008 ceph-mon[128788]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:09.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:09 smithi060 ceph-mon[131518]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:09.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:09 smithi008 ceph-mon[128788]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:09.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:09 smithi029 ceph-mon[131166]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:10.137 INFO:teuthology.orchestra.run.smithi029.stdout:Created osd(s) 10 on host 'smithi029' 2023-11-27T15:22:10.137 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.133+0000 7f4dfdffb700 1 -- 172.21.15.29:0/3907630148 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f4e0019ed00 con 0x7f4df00a1150 2023-11-27T15:22:10.139 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 -- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4df00a1150 msgr2=0x7f4df00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:10.139 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4df00a1150 0x7f4df00a3600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4df400b5c0 tx=0x7f4df4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:10.139 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 -- 172.21.15.29:0/3907630148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e00100d70 msgr2=0x7f4e0019d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:10.139 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e00100d70 0x7f4e0019d510 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4df800b970 tx=0x7f4df800bd30 comp rx=0 tx=0).stop 2023-11-27T15:22:10.140 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 -- 172.21.15.29:0/3907630148 shutdown_connections 2023-11-27T15:22:10.140 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4df00a1150 0x7f4df00a3600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:10.140 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e000ff1e0 0x7f4e00069170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:10.140 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e00100d70 0x7f4e0019d510 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:10.140 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 --2- 172.21.15.29:0/3907630148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e001003e0 0x7f4e0019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:10.140 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 -- 172.21.15.29:0/3907630148 >> 172.21.15.29:0/3907630148 conn(0x7f4e000fa7f0 msgr2=0x7f4e000fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:10.140 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 -- 172.21.15.29:0/3907630148 shutdown_connections 2023-11-27T15:22:10.140 INFO:teuthology.orchestra.run.smithi029.stderr:2023-11-27T15:22:10.135+0000 7f4e06eef700 1 -- 172.21.15.29:0/3907630148 wait complete. 2023-11-27T15:22:10.575 DEBUG:teuthology.orchestra.run.smithi029:osd.10> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.10.service 2023-11-27T15:22:10.578 INFO:tasks.cephadm:Deploying osd.11 on smithi060 with /dev/vg_nvme/lv_1... 2023-11-27T15:22:10.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-27T15:22:10.611 INFO:journalctl@ceph.osd.10.smithi029.stdout:-- Logs begin at Mon 2023-11-27 14:52:42 UTC. -- 2023-11-27T15:22:11.247 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:22:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:11 smithi008 ceph-mon[128788]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:11.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:11 smithi029 ceph-mon[131166]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:11.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:22:11 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:22:11.286+0000 7f32e4290640 -1 osd.10 0 log_to_monitors true 2023-11-27T15:22:11.496 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.496 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.496 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.496 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:11.496 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:11 smithi060 ceph-mon[131518]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:12.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:12 smithi029 ceph-mon[131166]: from='osd.10 [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:22:12.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:12 smithi029 ceph-mon[131166]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:22:12.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:12 smithi060 ceph-mon[131518]: from='osd.10 [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:22:12.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:12 smithi060 ceph-mon[131518]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:22:12.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:12 smithi008 ceph-mon[128788]: from='osd.10 [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:22:12.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:12 smithi008 ceph-mon[128788]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:22:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-27T15:22:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-27T15:22:12.743 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10+0 records in 2023-11-27T15:22:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:10+0 records out 2023-11-27T15:22:12.743 INFO:teuthology.orchestra.run.smithi060.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132673 s, 790 MB/s 2023-11-27T15:22:12.743 INFO:teuthology.orchestra.run.smithi060.stderr:--> Zapping successful for: 2023-11-27T15:22:13.209 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-27T15:22:13.209 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: osdmap e70: 11 total, 10 up, 11 in 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='osd.10 [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:13.210 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi060:vg_nvme/lv_1 2023-11-27T15:22:13.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-27T15:22:13.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: osdmap e70: 11 total, 10 up, 11 in 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='osd.10 [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:13.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:13.576 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-27T15:22:13.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: osdmap e70: 11 total, 10 up, 11 in 2023-11-27T15:22:13.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:13.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='osd.10 [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:22:13.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]: dispatch 2023-11-27T15:22:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:13.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:13.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:13.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:22:13.205+0000 7f32d9907700 -1 osd.10 0 waiting for initial osdmap 2023-11-27T15:22:13.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:22:13 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:22:13.224+0000 7f32d1ef5700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:22:13.958 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:22:14.514 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi029 2023-11-27T15:22:14.514 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi029 to 4293M 2023-11-27T15:22:14.514 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.514 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: osdmap e71: 11 total, 10 up, 11 in 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:22:14.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:14 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi029 2023-11-27T15:22:14.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi029 to 4293M 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: osdmap e71: 11 total, 10 up, 11 in 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:14.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:22:14.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:14 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi029 2023-11-27T15:22:14.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi029 to 4293M 2023-11-27T15:22:14.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi029", "root=default"]}]': finished 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: osdmap e71: 11 total, 10 up, 11 in 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:14.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:22:14.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:14 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:22:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.299+0000 7f9f61229700 1 -- 172.21.15.60:0/3542141007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f5c1022a0 msgr2=0x7f9f5c0fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.299+0000 7f9f61229700 1 --2- 172.21.15.60:0/3542141007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f5c1022a0 0x7f9f5c0fe380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009990 tx=0x7f9f4c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.299+0000 7f9f61229700 1 -- 172.21.15.60:0/3542141007 shutdown_connections 2023-11-27T15:22:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.299+0000 7f9f61229700 1 --2- 172.21.15.60:0/3542141007 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f5c0fea10 0x7f9f5c1088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.299+0000 7f9f61229700 1 --2- 172.21.15.60:0/3542141007 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f5c1018c0 0x7f9f5c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.299+0000 7f9f61229700 1 --2- 172.21.15.60:0/3542141007 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f5c1022a0 0x7f9f5c0fe380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.299+0000 7f9f61229700 1 -- 172.21.15.60:0/3542141007 >> 172.21.15.60:0/3542141007 conn(0x7f9f5c0f9f70 msgr2=0x7f9f5c0fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:15.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.299+0000 7f9f61229700 1 -- 172.21.15.60:0/3542141007 shutdown_connections 2023-11-27T15:22:15.304 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.299+0000 7f9f61229700 1 -- 172.21.15.60:0/3542141007 wait complete. 2023-11-27T15:22:15.305 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f61229700 1 Processor -- start 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f61229700 1 -- start start 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f61229700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f5c0fea10 0x7f9f5c196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f61229700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f5c1018c0 0x7f9f5c196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f61229700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f5c1022a0 0x7f9f5c19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f61229700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c19dbb0 con 0x7f9f5c0fea10 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f61229700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c19dcf0 con 0x7f9f5c1022a0 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f61229700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c19de80 con 0x7f9f5c1018c0 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f5a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f5c1018c0 0x7f9f5c196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f5a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f5c1018c0 0x7f9f5c196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35152/0 (socket says 172.21.15.60:35152) 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f5a59c700 1 -- 172.21.15.60:0/2788338355 learned_addr learned my addr 172.21.15.60:0/2788338355 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-27T15:22:15.306 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.300+0000 7f9f5b59e700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f5c1022a0 0x7f9f5c19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f5a59c700 1 -- 172.21.15.60:0/2788338355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f5c1022a0 msgr2=0x7f9f5c19d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f5a59c700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f5c1022a0 0x7f9f5c19d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f5ad9d700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f5c0fea10 0x7f9f5c196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f5a59c700 1 -- 172.21.15.60:0/2788338355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f5c0fea10 msgr2=0x7f9f5c196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f5a59c700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f5c0fea10 0x7f9f5c196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f5a59c700 1 -- 172.21.15.60:0/2788338355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c0095f0 con 0x7f9f5c1018c0 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f5a59c700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f5c1018c0 0x7f9f5c196560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00b5c0 tx=0x7f9f4c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f43fff700 1 -- 172.21.15.60:0/2788338355 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c01d070 con 0x7f9f5c1018c0 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f43fff700 1 -- 172.21.15.60:0/2788338355 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f4c022470 con 0x7f9f5c1018c0 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f61229700 1 -- 172.21.15.60:0/2788338355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c10bc50 con 0x7f9f5c1018c0 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f43fff700 1 -- 172.21.15.60:0/2788338355 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c017630 con 0x7f9f5c1018c0 2023-11-27T15:22:15.307 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.301+0000 7f9f61229700 1 -- 172.21.15.60:0/2788338355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c10bac0 con 0x7f9f5c1018c0 2023-11-27T15:22:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.302+0000 7f9f61229700 1 -- 172.21.15.60:0/2788338355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c066ea0 con 0x7f9f5c1018c0 2023-11-27T15:22:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.303+0000 7f9f43fff700 1 -- 172.21.15.60:0/2788338355 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9f4c0228e0 con 0x7f9f5c1018c0 2023-11-27T15:22:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.304+0000 7f9f43fff700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f440a1150 0x7f9f440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.304+0000 7f9f43fff700 1 -- 172.21.15.60:0/2788338355 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f9f4c0cd5c0 con 0x7f9f5c1018c0 2023-11-27T15:22:15.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.304+0000 7f9f5ad9d700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f440a1150 0x7f9f440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:15.309 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.305+0000 7f9f5ad9d700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f440a1150 0x7f9f440a3600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f50005950 tx=0x7f9f500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:15.312 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.309+0000 7f9f43fff700 1 -- 172.21.15.60:0/2788338355 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9f4c05c730 con 0x7f9f5c1018c0 2023-11-27T15:22:15.515 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:15.511+0000 7f9f61229700 1 -- 172.21.15.60:0/2788338355 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f5c060f30 con 0x7f9f440a1150 2023-11-27T15:22:15.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:15 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:22:15.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:15 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:22:15.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:15.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:15 smithi060 ceph-mon[131518]: osd.10 [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] boot 2023-11-27T15:22:15.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:15 smithi060 ceph-mon[131518]: osdmap e72: 11 total, 11 up, 11 in 2023-11-27T15:22:15.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:15.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:15 smithi060 ceph-mon[131518]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:15.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:15 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:22:15.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:15 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:22:15.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:15.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:15 smithi008 ceph-mon[128788]: osd.10 [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] boot 2023-11-27T15:22:15.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:15 smithi008 ceph-mon[128788]: osdmap e72: 11 total, 11 up, 11 in 2023-11-27T15:22:15.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:15.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:15 smithi008 ceph-mon[128788]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:15 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:22:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:15 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:22:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:15 smithi029 ceph-mon[131166]: osd.10 [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] boot 2023-11-27T15:22:15.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:15 smithi029 ceph-mon[131166]: osdmap e72: 11 total, 11 up, 11 in 2023-11-27T15:22:15.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:22:15.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:15 smithi029 ceph-mon[131166]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:22:16.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:16 smithi060 ceph-mon[131518]: osdmap e73: 11 total, 11 up, 11 in 2023-11-27T15:22:16.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:16 smithi060 ceph-mon[131518]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:16.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:22:16.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:22:16.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:16.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:16 smithi008 ceph-mon[128788]: osdmap e73: 11 total, 11 up, 11 in 2023-11-27T15:22:16.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:16 smithi008 ceph-mon[128788]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:16.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:22:16.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:22:16.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:16.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:16 smithi029 ceph-mon[131166]: osdmap e73: 11 total, 11 up, 11 in 2023-11-27T15:22:16.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:16 smithi029 ceph-mon[131166]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:16.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:22:16.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:22:16.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:17.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:17 smithi060 ceph-mon[131518]: osdmap e74: 11 total, 11 up, 11 in 2023-11-27T15:22:17.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:17 smithi060 ceph-mon[131518]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:17.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:17 smithi008 ceph-mon[128788]: osdmap e74: 11 total, 11 up, 11 in 2023-11-27T15:22:17.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:17 smithi008 ceph-mon[128788]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:17.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:17 smithi029 ceph-mon[131166]: osdmap e74: 11 total, 11 up, 11 in 2023-11-27T15:22:17.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:17 smithi029 ceph-mon[131166]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:18.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:18 smithi060 ceph-mon[131518]: osdmap e75: 11 total, 11 up, 11 in 2023-11-27T15:22:18.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:18 smithi060 ceph-mon[131518]: from='client.? 172.21.15.60:0/1702910368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac26363a-c419-4b6d-8dc7-5e337cc9b565"}]: dispatch 2023-11-27T15:22:18.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:18 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac26363a-c419-4b6d-8dc7-5e337cc9b565"}]: dispatch 2023-11-27T15:22:18.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:18 smithi060 ceph-mon[131518]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac26363a-c419-4b6d-8dc7-5e337cc9b565"}]': finished 2023-11-27T15:22:18.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:18 smithi060 ceph-mon[131518]: osdmap e76: 12 total, 11 up, 12 in 2023-11-27T15:22:18.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:18 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:18 smithi008 ceph-mon[128788]: osdmap e75: 11 total, 11 up, 11 in 2023-11-27T15:22:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:18 smithi008 ceph-mon[128788]: from='client.? 172.21.15.60:0/1702910368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac26363a-c419-4b6d-8dc7-5e337cc9b565"}]: dispatch 2023-11-27T15:22:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:18 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac26363a-c419-4b6d-8dc7-5e337cc9b565"}]: dispatch 2023-11-27T15:22:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:18 smithi008 ceph-mon[128788]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac26363a-c419-4b6d-8dc7-5e337cc9b565"}]': finished 2023-11-27T15:22:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:18 smithi008 ceph-mon[128788]: osdmap e76: 12 total, 11 up, 12 in 2023-11-27T15:22:18.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:18 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:18.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:18 smithi029 ceph-mon[131166]: osdmap e75: 11 total, 11 up, 11 in 2023-11-27T15:22:18.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:18 smithi029 ceph-mon[131166]: from='client.? 172.21.15.60:0/1702910368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac26363a-c419-4b6d-8dc7-5e337cc9b565"}]: dispatch 2023-11-27T15:22:18.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:18 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac26363a-c419-4b6d-8dc7-5e337cc9b565"}]: dispatch 2023-11-27T15:22:18.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:18 smithi029 ceph-mon[131166]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac26363a-c419-4b6d-8dc7-5e337cc9b565"}]': finished 2023-11-27T15:22:18.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:18 smithi029 ceph-mon[131166]: osdmap e76: 12 total, 11 up, 12 in 2023-11-27T15:22:18.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:18 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:19 smithi060 ceph-mon[131518]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:22:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:19 smithi060 ceph-mon[131518]: from='client.? 172.21.15.60:0/816254995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:22:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:19 smithi008 ceph-mon[128788]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:22:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:19 smithi008 ceph-mon[128788]: from='client.? 172.21.15.60:0/816254995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:22:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:19 smithi029 ceph-mon[131166]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:22:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:19 smithi029 ceph-mon[131166]: from='client.? 172.21.15.60:0/816254995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:22:21.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:21 smithi029 ceph-mon[131166]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-11-27T15:22:21.656 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:21 smithi060 ceph-mon[131518]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-11-27T15:22:21.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:21 smithi008 ceph-mon[128788]: pgmap v237: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-11-27T15:22:23.418 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:23 smithi060 ceph-mon[131518]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-27T15:22:23.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:23 smithi029 ceph-mon[131166]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-27T15:22:23.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:23 smithi008 ceph-mon[128788]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2023-11-27T15:22:25.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:25 smithi060 ceph-mon[131518]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:22:25.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-27T15:22:25.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:25.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:25 smithi060 ceph-mon[131518]: Deploying daemon osd.11 on smithi060 2023-11-27T15:22:25.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:25 smithi029 ceph-mon[131166]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:22:25.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:25 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-27T15:22:25.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:25 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:25.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:25 smithi029 ceph-mon[131166]: Deploying daemon osd.11 on smithi060 2023-11-27T15:22:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:25 smithi008 ceph-mon[128788]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:22:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-27T15:22:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:25 smithi008 ceph-mon[128788]: Deploying daemon osd.11 on smithi060 2023-11-27T15:22:27.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:27 smithi029 ceph-mon[131166]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:27.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:27 smithi060 ceph-mon[131518]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:27 smithi008 ceph-mon[128788]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:29.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:29 smithi060 ceph-mon[131518]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:29.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:29 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:29.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:29 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:29.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:29 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:29 smithi008 ceph-mon[128788]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:29 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:29 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:29 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:29 smithi029 ceph-mon[131166]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:29 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:29 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:29 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:31.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:31 smithi029 ceph-mon[131166]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:31.661 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:31 smithi060 ceph-mon[131518]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:31.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:31 smithi008 ceph-mon[128788]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:31.979 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 11 on host 'smithi060' 2023-11-27T15:22:31.979 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.974+0000 7f9f43fff700 1 -- 172.21.15.60:0/2788338355 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f9f5c060f30 con 0x7f9f440a1150 2023-11-27T15:22:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.976+0000 7f9f61229700 1 -- 172.21.15.60:0/2788338355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f440a1150 msgr2=0x7f9f440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.976+0000 7f9f61229700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f440a1150 0x7f9f440a3600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f50005950 tx=0x7f9f500058e0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.976+0000 7f9f61229700 1 -- 172.21.15.60:0/2788338355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f5c1018c0 msgr2=0x7f9f5c196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.976+0000 7f9f61229700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f5c1018c0 0x7f9f5c196560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00b5c0 tx=0x7f9f4c00f690 comp rx=0 tx=0).stop 2023-11-27T15:22:31.981 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.976+0000 7f9f61229700 1 -- 172.21.15.60:0/2788338355 shutdown_connections 2023-11-27T15:22:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.976+0000 7f9f61229700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f5c1022a0 0x7f9f5c19d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.976+0000 7f9f61229700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f5c0fea10 0x7f9f5c196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.976+0000 7f9f61229700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f440a1150 0x7f9f440a3600 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.976+0000 7f9f61229700 1 --2- 172.21.15.60:0/2788338355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f5c1018c0 0x7f9f5c196560 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.976+0000 7f9f61229700 1 -- 172.21.15.60:0/2788338355 >> 172.21.15.60:0/2788338355 conn(0x7f9f5c0f9f70 msgr2=0x7f9f5c0fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.977+0000 7f9f61229700 1 -- 172.21.15.60:0/2788338355 shutdown_connections 2023-11-27T15:22:31.982 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:31.977+0000 7f9f61229700 1 -- 172.21.15.60:0/2788338355 wait complete. 2023-11-27T15:22:32.462 DEBUG:teuthology.orchestra.run.smithi060:osd.11> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.11.service 2023-11-27T15:22:32.465 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-11-27T15:22:32.465 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd stat -f json 2023-11-27T15:22:32.494 INFO:journalctl@ceph.osd.11.smithi060.stdout:-- Logs begin at Mon 2023-11-27 14:51:59 UTC. -- 2023-11-27T15:22:32.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:22:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:22:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:22:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:22:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:22:32.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:22:32.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:22:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:22:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:22:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:33.158 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:22:32 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:22:32.868+0000 7f1931f50640 -1 osd.11 0 log_to_monitors true 2023-11-27T15:22:33.335 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:33.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:33 smithi060 ceph-mon[131518]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:33.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:33 smithi060 ceph-mon[131518]: from='osd.11 [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:22:33.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:33 smithi060 ceph-mon[131518]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:22:33.785 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:33 smithi008 ceph-mon[128788]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:33.785 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:33 smithi008 ceph-mon[128788]: from='osd.11 [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:22:33.785 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:33 smithi008 ceph-mon[128788]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:22:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:33 smithi029 ceph-mon[131166]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:33 smithi029 ceph-mon[131166]: from='osd.11 [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:22:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:33 smithi029 ceph-mon[131166]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:22:34.525 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:22:34 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:22:34.503+0000 7f19275c7700 -1 osd.11 0 waiting for initial osdmap 2023-11-27T15:22:34.525 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:22:34 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:22:34.520+0000 7f191fbb5700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:22:34.525 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:34 smithi060 ceph-mon[131518]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-27T15:22:34.525 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:34 smithi060 ceph-mon[131518]: osdmap e77: 12 total, 11 up, 12 in 2023-11-27T15:22:34.525 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:34 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:34.525 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:34 smithi060 ceph-mon[131518]: from='osd.11 [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:22:34.525 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:34 smithi060 ceph-mon[131518]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:22:34.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:34 smithi008 ceph-mon[128788]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-27T15:22:34.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:34 smithi008 ceph-mon[128788]: osdmap e77: 12 total, 11 up, 12 in 2023-11-27T15:22:34.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:34 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:34.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:34 smithi008 ceph-mon[128788]: from='osd.11 [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:22:34.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:34 smithi008 ceph-mon[128788]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:22:34.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.886+0000 7f1b6ab85700 1 -- 172.21.15.8:0/910301365 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b64074b20 msgr2=0x7f1b64072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.886+0000 7f1b6ab85700 1 --2- 172.21.15.8:0/910301365 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b64074b20 0x7f1b64072f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009990 tx=0x7f1b60009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.887+0000 7f1b6ab85700 1 -- 172.21.15.8:0/910301365 shutdown_connections 2023-11-27T15:22:34.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.887+0000 7f1b6ab85700 1 --2- 172.21.15.8:0/910301365 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b64107fe0 0x7f1b6410a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.887+0000 7f1b6ab85700 1 --2- 172.21.15.8:0/910301365 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b64074b20 0x7f1b64072f80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.887+0000 7f1b6ab85700 1 --2- 172.21.15.8:0/910301365 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64073550 0x7f1b640739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.887+0000 7f1b6ab85700 1 -- 172.21.15.8:0/910301365 >> 172.21.15.8:0/910301365 conn(0x7f1b640fb720 msgr2=0x7f1b640fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.887+0000 7f1b6ab85700 1 -- 172.21.15.8:0/910301365 shutdown_connections 2023-11-27T15:22:34.891 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.887+0000 7f1b6ab85700 1 -- 172.21.15.8:0/910301365 wait complete. 2023-11-27T15:22:34.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.888+0000 7f1b6ab85700 1 Processor -- start 2023-11-27T15:22:34.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.888+0000 7f1b6ab85700 1 -- start start 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.888+0000 7f1b6ab85700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64073550 0x7f1b64198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.888+0000 7f1b6ab85700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b64074b20 0x7f1b641985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.888+0000 7f1b6ab85700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b64107fe0 0x7f1b6419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.888+0000 7f1b6ab85700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6419fc00 con 0x7f1b64107fe0 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.888+0000 7f1b6ab85700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6419fd40 con 0x7f1b64074b20 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.888+0000 7f1b6ab85700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6419fed0 con 0x7f1b64073550 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b69b83700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64073550 0x7f1b64198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b6a384700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b64107fe0 0x7f1b6419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b69b83700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64073550 0x7f1b64198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:35282/0 (socket says 172.21.15.8:35282) 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b69b83700 1 -- 172.21.15.8:0/1421143712 learned_addr learned my addr 172.21.15.8:0/1421143712 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b69382700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b64074b20 0x7f1b641985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b6a384700 1 -- 172.21.15.8:0/1421143712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64073550 msgr2=0x7f1b64198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b6a384700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64073550 0x7f1b64198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b6a384700 1 -- 172.21.15.8:0/1421143712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b64074b20 msgr2=0x7f1b641985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b6a384700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b64074b20 0x7f1b641985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.894 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b6a384700 1 -- 172.21.15.8:0/1421143712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b600095f0 con 0x7f1b64107fe0 2023-11-27T15:22:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b6a384700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b64107fe0 0x7f1b6419f630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009f60 tx=0x7f1b5c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.889+0000 7f1b5affd700 1 -- 172.21.15.8:0/1421143712 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c018070 con 0x7f1b64107fe0 2023-11-27T15:22:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.890+0000 7f1b5affd700 1 -- 172.21.15.8:0/1421143712 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b5c00f040 con 0x7f1b64107fe0 2023-11-27T15:22:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.890+0000 7f1b6ab85700 1 -- 172.21.15.8:0/1421143712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6410dca0 con 0x7f1b64107fe0 2023-11-27T15:22:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.890+0000 7f1b5affd700 1 -- 172.21.15.8:0/1421143712 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c00ccc0 con 0x7f1b64107fe0 2023-11-27T15:22:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.890+0000 7f1b6ab85700 1 -- 172.21.15.8:0/1421143712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6410db10 con 0x7f1b64107fe0 2023-11-27T15:22:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.891+0000 7f1b6ab85700 1 -- 172.21.15.8:0/1421143712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b64106690 con 0x7f1b64107fe0 2023-11-27T15:22:34.896 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.893+0000 7f1b5affd700 1 -- 172.21.15.8:0/1421143712 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1b5c003a00 con 0x7f1b64107fe0 2023-11-27T15:22:34.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:34 smithi029 ceph-mon[131166]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-27T15:22:34.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:34 smithi029 ceph-mon[131166]: osdmap e77: 12 total, 11 up, 12 in 2023-11-27T15:22:34.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:34 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:34.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:34 smithi029 ceph-mon[131166]: from='osd.11 [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:22:34.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:34 smithi029 ceph-mon[131166]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.894+0000 7f1b5affd700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b500a11d0 0x7f1b500a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.894+0000 7f1b5affd700 1 -- 172.21.15.8:0/1421143712 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f1b5c0cc770 con 0x7f1b64107fe0 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.895+0000 7f1b69b83700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b500a11d0 0x7f1b500a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.895+0000 7f1b69b83700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b500a11d0 0x7f1b500a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000b2e0 tx=0x7f1b60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:34.897+0000 7f1b5affd700 1 -- 172.21.15.8:0/1421143712 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1b5c096470 con 0x7f1b64107fe0 2023-11-27T15:22:35.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.116+0000 7f1b6ab85700 1 -- 172.21.15.8:0/1421143712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1b6402cfe0 con 0x7f1b64107fe0 2023-11-27T15:22:35.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.117+0000 7f1b5affd700 1 -- 172.21.15.8:0/1421143712 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1b5c099a90 con 0x7f1b64107fe0 2023-11-27T15:22:35.121 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:22:35.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 -- 172.21.15.8:0/1421143712 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b500a11d0 msgr2=0x7f1b500a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b500a11d0 0x7f1b500a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000b2e0 tx=0x7f1b60005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 -- 172.21.15.8:0/1421143712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b64107fe0 msgr2=0x7f1b6419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b64107fe0 0x7f1b6419f630 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009f60 tx=0x7f1b5c009f90 comp rx=0 tx=0).stop 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 -- 172.21.15.8:0/1421143712 shutdown_connections 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1b500a11d0 0x7f1b500a3680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1b64073550 0x7f1b64198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b64107fe0 0x7f1b6419f630 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 --2- 172.21.15.8:0/1421143712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1b64074b20 0x7f1b641985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 -- 172.21.15.8:0/1421143712 >> 172.21.15.8:0/1421143712 conn(0x7f1b640fb720 msgr2=0x7f1b640fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 -- 172.21.15.8:0/1421143712 shutdown_connections 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:35.119+0000 7f1b6ab85700 1 -- 172.21.15.8:0/1421143712 wait complete. 2023-11-27T15:22:35.618 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":78,"num_osds":12,"num_up_osds":11,"osd_up_since":1701098534,"num_in_osds":12,"osd_in_since":1701098537,"num_remapped_pgs":0} 2023-11-27T15:22:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:22:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: osdmap e78: 12 total, 11 up, 12 in 2023-11-27T15:22:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: Detected new or changed devices on smithi060 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:35 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1421143712' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:22:35.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:35.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:22:35.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: osdmap e78: 12 total, 11 up, 12 in 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: Detected new or changed devices on smithi060 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:35.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:35 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1421143712' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:22:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:22:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2023-11-27T15:22:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: osdmap e78: 12 total, 11 up, 12 in 2023-11-27T15:22:35.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:35.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:35.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: Detected new or changed devices on smithi060 2023-11-27T15:22:35.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:22:35.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:35.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:35.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:35.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:35 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1421143712' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:22:36.619 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd stat -f json 2023-11-27T15:22:36.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:36 smithi060 ceph-mon[131518]: purged_snaps scrub starts 2023-11-27T15:22:36.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:36 smithi060 ceph-mon[131518]: purged_snaps scrub ok 2023-11-27T15:22:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:36 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi060 to 4293M 2023-11-27T15:22:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:36 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:22:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:36 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:22:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:36 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:22:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:36 smithi060 ceph-mon[131518]: osd.11 [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] boot 2023-11-27T15:22:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:36 smithi060 ceph-mon[131518]: osdmap e79: 12 total, 12 up, 12 in 2023-11-27T15:22:36.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:36 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:36.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:36 smithi008 ceph-mon[128788]: purged_snaps scrub starts 2023-11-27T15:22:36.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:36 smithi008 ceph-mon[128788]: purged_snaps scrub ok 2023-11-27T15:22:36.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:36 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi060 to 4293M 2023-11-27T15:22:36.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:36 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:22:36.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:36 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:22:36.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:36 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:22:36.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:36 smithi008 ceph-mon[128788]: osd.11 [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] boot 2023-11-27T15:22:36.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:36 smithi008 ceph-mon[128788]: osdmap e79: 12 total, 12 up, 12 in 2023-11-27T15:22:36.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:36 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:36.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:36 smithi029 ceph-mon[131166]: purged_snaps scrub starts 2023-11-27T15:22:36.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:36 smithi029 ceph-mon[131166]: purged_snaps scrub ok 2023-11-27T15:22:36.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:36 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi060 to 4293M 2023-11-27T15:22:36.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:36 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:22:36.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:36 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:22:36.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:36 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:22:36.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:36 smithi029 ceph-mon[131166]: osd.11 [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] boot 2023-11-27T15:22:36.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:36 smithi029 ceph-mon[131166]: osdmap e79: 12 total, 12 up, 12 in 2023-11-27T15:22:36.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:36 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:22:37.491 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:37.785 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:37 smithi008 ceph-mon[128788]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:37.785 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:37 smithi008 ceph-mon[128788]: osdmap e80: 12 total, 12 up, 12 in 2023-11-27T15:22:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:37 smithi060 ceph-mon[131518]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:37 smithi060 ceph-mon[131518]: osdmap e80: 12 total, 12 up, 12 in 2023-11-27T15:22:37.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:37 smithi029 ceph-mon[131166]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:37.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:37 smithi029 ceph-mon[131166]: osdmap e80: 12 total, 12 up, 12 in 2023-11-27T15:22:39.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.028+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3367814922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee01002f0 msgr2=0x7f6ee0100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:39.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.028+0000 7f6ee51f4700 1 --2- 172.21.15.8:0/3367814922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee01002f0 0x7f6ee0100700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009990 tx=0x7f6ed0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:39.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.028+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3367814922 shutdown_connections 2023-11-27T15:22:39.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.028+0000 7f6ee51f4700 1 --2- 172.21.15.8:0/3367814922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee0101e80 0x7f6ee0108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:39.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.028+0000 7f6ee51f4700 1 --2- 172.21.15.8:0/3367814922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee01002f0 0x7f6ee0100700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:39.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.028+0000 7f6ee51f4700 1 --2- 172.21.15.8:0/3367814922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ee01014f0 0x7f6ee0101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:39.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.028+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3367814922 >> 172.21.15.8:0/3367814922 conn(0x7f6ee00fb900 msgr2=0x7f6ee00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:39.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.029+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3367814922 shutdown_connections 2023-11-27T15:22:39.033 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.029+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3367814922 wait complete. 2023-11-27T15:22:39.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6ee51f4700 1 Processor -- start 2023-11-27T15:22:39.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6ee51f4700 1 -- start start 2023-11-27T15:22:39.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6ee51f4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee01002f0 0x7f6ee0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:39.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6ee51f4700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ee01014f0 0x7f6ee01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6ee51f4700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee0101e80 0x7f6ee019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6ee51f4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee019fe20 con 0x7f6ee01002f0 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6ee51f4700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee019ff60 con 0x7f6ee01014f0 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6ee51f4700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee01a00f0 con 0x7f6ee0101e80 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6eded9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee01002f0 0x7f6ee0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6eded9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee01002f0 0x7f6ee0198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40922/0 (socket says 172.21.15.8:40922) 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.030+0000 7f6eded9d700 1 -- 172.21.15.8:0/3548880337 learned_addr learned my addr 172.21.15.8:0/3548880337 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6edf59e700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee0101e80 0x7f6ee019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6ede59c700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ee01014f0 0x7f6ee01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6eded9d700 1 -- 172.21.15.8:0/3548880337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee0101e80 msgr2=0x7f6ee019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6eded9d700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee0101e80 0x7f6ee019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6eded9d700 1 -- 172.21.15.8:0/3548880337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ee01014f0 msgr2=0x7f6ee01987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:39.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6eded9d700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ee01014f0 0x7f6ee01987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:39.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6eded9d700 1 -- 172.21.15.8:0/3548880337 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed00095f0 con 0x7f6ee01002f0 2023-11-27T15:22:39.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6eded9d700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee01002f0 0x7f6ee0198290 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0004740 tx=0x7f6ed0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:39.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6ec7fff700 1 -- 172.21.15.8:0/3548880337 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed001d070 con 0x7f6ee01002f0 2023-11-27T15:22:39.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6ec7fff700 1 -- 172.21.15.8:0/3548880337 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ed0021410 con 0x7f6ee01002f0 2023-11-27T15:22:39.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.031+0000 7f6ec7fff700 1 -- 172.21.15.8:0/3548880337 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0017650 con 0x7f6ee01002f0 2023-11-27T15:22:39.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.032+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3548880337 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee010bcb0 con 0x7f6ee01002f0 2023-11-27T15:22:39.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.032+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3548880337 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee010bb20 con 0x7f6ee01002f0 2023-11-27T15:22:39.037 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.033+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3548880337 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee0066ea0 con 0x7f6ee01002f0 2023-11-27T15:22:39.038 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.035+0000 7f6ec7fff700 1 -- 172.21.15.8:0/3548880337 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6ed0021580 con 0x7f6ee01002f0 2023-11-27T15:22:39.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.036+0000 7f6ec7fff700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ec80a11d0 0x7f6ec80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:39.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.036+0000 7f6ec7fff700 1 -- 172.21.15.8:0/3548880337 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6ed00cdf40 con 0x7f6ee01002f0 2023-11-27T15:22:39.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.036+0000 7f6ede59c700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ec80a11d0 0x7f6ec80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:39.041 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.037+0000 7f6ede59c700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ec80a11d0 0x7f6ec80a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009c60 tx=0x7f6ed4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:39.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.039+0000 7f6ec7fff700 1 -- 172.21.15.8:0/3548880337 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6ed0097b20 con 0x7f6ee01002f0 2023-11-27T15:22:39.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.247+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3548880337 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f6ee01a0b10 con 0x7f6ee01002f0 2023-11-27T15:22:39.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.247+0000 7f6ec7fff700 1 -- 172.21.15.8:0/3548880337 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f6ed009b140 con 0x7f6ee01002f0 2023-11-27T15:22:39.251 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:22:39.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3548880337 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ec80a11d0 msgr2=0x7f6ec80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:39.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ec80a11d0 0x7f6ec80a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009c60 tx=0x7f6ed4009260 comp rx=0 tx=0).stop 2023-11-27T15:22:39.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3548880337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee01002f0 msgr2=0x7f6ee0198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:39.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee01002f0 0x7f6ee0198290 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0004740 tx=0x7f6ed0009580 comp rx=0 tx=0).stop 2023-11-27T15:22:39.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3548880337 shutdown_connections 2023-11-27T15:22:39.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ec80a11d0 0x7f6ec80a3680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:39.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ee0101e80 0x7f6ee019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:39.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ee01002f0 0x7f6ee0198290 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:39.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 --2- 172.21.15.8:0/3548880337 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ee01014f0 0x7f6ee01987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:39.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3548880337 >> 172.21.15.8:0/3548880337 conn(0x7f6ee00fb900 msgr2=0x7f6ee00fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:39.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3548880337 shutdown_connections 2023-11-27T15:22:39.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:39.250+0000 7f6ee51f4700 1 -- 172.21.15.8:0/3548880337 wait complete. 2023-11-27T15:22:39.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:39 smithi060 ceph-mon[131518]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:39 smithi008 ceph-mon[128788]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:39.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:39 smithi029 ceph-mon[131166]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:39.775 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1701098555,"num_in_osds":12,"osd_in_since":1701098537,"num_remapped_pgs":0} 2023-11-27T15:22:39.775 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-11-27T15:22:40.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:40 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3548880337' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:22:40.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:40 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3548880337' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:22:40.635 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:40.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:40 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3548880337' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:22:41.422 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:41 smithi008 ceph-mon[128788]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:41.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:41 smithi060 ceph-mon[131518]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:41.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:41 smithi029 ceph-mon[131166]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:42.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.054+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/3309611590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bb8100ee0 msgr2=0x7f5bb8104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.054+0000 7f5bbe2b6700 1 --2- 172.21.15.8:0/3309611590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bb8100ee0 0x7f5bb8104eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009920 tx=0x7f5bac009c30 comp rx=0 tx=0).stop 2023-11-27T15:22:42.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.054+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/3309611590 shutdown_connections 2023-11-27T15:22:42.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.054+0000 7f5bbe2b6700 1 --2- 172.21.15.8:0/3309611590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bb81055e0 0x7f5bb810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.054+0000 7f5bbe2b6700 1 --2- 172.21.15.8:0/3309611590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb8100540 0x7f5bb8100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.054+0000 7f5bbe2b6700 1 --2- 172.21.15.8:0/3309611590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bb8100ee0 0x7f5bb8104eb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.054+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/3309611590 >> 172.21.15.8:0/3309611590 conn(0x7f5bb80fbdc0 msgr2=0x7f5bb80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.055+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/3309611590 shutdown_connections 2023-11-27T15:22:42.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.055+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/3309611590 wait complete. 2023-11-27T15:22:42.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.055+0000 7f5bbe2b6700 1 Processor -- start 2023-11-27T15:22:42.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bbe2b6700 1 -- start start 2023-11-27T15:22:42.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bbe2b6700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bb8100540 0x7f5bb8196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bbe2b6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bb8100ee0 0x7f5bb8196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bbe2b6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb81055e0 0x7f5bb819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bbe2b6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb819b3e0 con 0x7f5bb8100ee0 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bbe2b6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb819b550 con 0x7f5bb81055e0 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bbe2b6700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb819b710 con 0x7f5bb8100540 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bb77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bb8100ee0 0x7f5bb8196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bb8100540 0x7f5bb8196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bb77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bb8100ee0 0x7f5bb8196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41738/0 (socket says 172.21.15.8:41738) 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bb7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bb8100540 0x7f5bb8196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:52876/0 (socket says 172.21.15.8:52876) 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.056+0000 7f5bb77fe700 1 -- 172.21.15.8:0/480803763 learned_addr learned my addr 172.21.15.8:0/480803763 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bbc853700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb81055e0 0x7f5bb819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bb77fe700 1 -- 172.21.15.8:0/480803763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bb8100540 msgr2=0x7f5bb8196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bb77fe700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bb8100540 0x7f5bb8196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bb77fe700 1 -- 172.21.15.8:0/480803763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb81055e0 msgr2=0x7f5bb819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bb77fe700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb81055e0 0x7f5bb819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bb77fe700 1 -- 172.21.15.8:0/480803763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba80095f0 con 0x7f5bb8100ee0 2023-11-27T15:22:42.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bb77fe700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bb8100ee0 0x7f5bb8196960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009230 tx=0x7f5bac00f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bb57fa700 1 -- 172.21.15.8:0/480803763 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac01c070 con 0x7f5bb8100ee0 2023-11-27T15:22:42.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bb57fa700 1 -- 172.21.15.8:0/480803763 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bac00baa0 con 0x7f5bb8100ee0 2023-11-27T15:22:42.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bb57fa700 1 -- 172.21.15.8:0/480803763 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0038e0 con 0x7f5bb8100ee0 2023-11-27T15:22:42.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.057+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/480803763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac009520 con 0x7f5bb8100ee0 2023-11-27T15:22:42.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.058+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/480803763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb8110360 con 0x7f5bb8100ee0 2023-11-27T15:22:42.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.059+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/480803763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb81101d0 con 0x7f5bb8100ee0 2023-11-27T15:22:42.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.061+0000 7f5bb57fa700 1 -- 172.21.15.8:0/480803763 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5bac020030 con 0x7f5bb8100ee0 2023-11-27T15:22:42.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.062+0000 7f5bb57fa700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5ba00a11d0 0x7f5ba00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.062+0000 7f5bb57fa700 1 -- 172.21.15.8:0/480803763 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5bac0cd2f0 con 0x7f5bb8100ee0 2023-11-27T15:22:42.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.062+0000 7f5bb7fff700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5ba00a11d0 0x7f5ba00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.063+0000 7f5bb7fff700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5ba00a11d0 0x7f5ba00a3680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8005fd0 tx=0x7f5ba8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.066+0000 7f5bb57fa700 1 -- 172.21.15.8:0/480803763 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5bac096ed0 con 0x7f5bb8100ee0 2023-11-27T15:22:42.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.275+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/480803763 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bb81104f0 con 0x7f5bb8100ee0 2023-11-27T15:22:42.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.276+0000 7f5bb57fa700 1 -- 172.21.15.8:0/480803763 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18279 (secure 0 0 0) 0x7f5bac09a4f0 con 0x7f5bb8100ee0 2023-11-27T15:22:42.280 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:22:42.280 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":80,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:22:36.503805+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:42.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.278+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/480803763 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5ba00a11d0 msgr2=0x7f5ba00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.278+0000 7f5bbe2b6700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5ba00a11d0 0x7f5ba00a3680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8005fd0 tx=0x7f5ba8009310 comp rx=0 tx=0).stop 2023-11-27T15:22:42.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.278+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/480803763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bb8100ee0 msgr2=0x7f5bb8196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.278+0000 7f5bbe2b6700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bb8100ee0 0x7f5bb8196960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009230 tx=0x7f5bac00f9e0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.278+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/480803763 shutdown_connections 2023-11-27T15:22:42.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.278+0000 7f5bbe2b6700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5ba00a11d0 0x7f5ba00a3680 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.278+0000 7f5bbe2b6700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5bb8100540 0x7f5bb8196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.278+0000 7f5bbe2b6700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5bb81055e0 0x7f5bb819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.278+0000 7f5bbe2b6700 1 --2- 172.21.15.8:0/480803763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5bb8100ee0 0x7f5bb8196960 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.278+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/480803763 >> 172.21.15.8:0/480803763 conn(0x7f5bb80fbdc0 msgr2=0x7f5bb80fc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.279+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/480803763 shutdown_connections 2023-11-27T15:22:42.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:42.279+0000 7f5bbe2b6700 1 -- 172.21.15.8:0/480803763 wait complete. 2023-11-27T15:22:42.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:42 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/480803763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:42.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:42 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/480803763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:42.725 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:22:42.726 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-11-27T15:22:42.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:42 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/480803763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:43.584 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:43.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:43 smithi029 ceph-mon[131166]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:43 smithi060 ceph-mon[131518]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:43.856 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:43 smithi008 ceph-mon[128788]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.056+0000 7fadba65e700 1 -- 172.21.15.8:0/3249297736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fadb4100540 msgr2=0x7fadb4100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.056+0000 7fadba65e700 1 --2- 172.21.15.8:0/3249297736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fadb4100540 0x7fadb4100910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fada40099c0 tx=0x7fada4009cd0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.056+0000 7fadba65e700 1 -- 172.21.15.8:0/3249297736 shutdown_connections 2023-11-27T15:22:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.056+0000 7fadba65e700 1 --2- 172.21.15.8:0/3249297736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadb41055e0 0x7fadb410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.056+0000 7fadba65e700 1 --2- 172.21.15.8:0/3249297736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fadb4100540 0x7fadb4100910 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.056+0000 7fadba65e700 1 --2- 172.21.15.8:0/3249297736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb4100ee0 0x7fadb4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.056+0000 7fadba65e700 1 -- 172.21.15.8:0/3249297736 >> 172.21.15.8:0/3249297736 conn(0x7fadb40fbdc0 msgr2=0x7fadb40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:45.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.056+0000 7fadba65e700 1 -- 172.21.15.8:0/3249297736 shutdown_connections 2023-11-27T15:22:45.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.056+0000 7fadba65e700 1 -- 172.21.15.8:0/3249297736 wait complete. 2023-11-27T15:22:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.057+0000 7fadba65e700 1 Processor -- start 2023-11-27T15:22:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.057+0000 7fadba65e700 1 -- start start 2023-11-27T15:22:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadba65e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fadb4100540 0x7fadb41963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadba65e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb4100ee0 0x7fadb41968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadba65e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadb41055e0 0x7fadb419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadba65e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb419b360 con 0x7fadb4100540 2023-11-27T15:22:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadba65e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb419b4d0 con 0x7fadb41055e0 2023-11-27T15:22:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadba65e700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb419b690 con 0x7fadb4100ee0 2023-11-27T15:22:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadb3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fadb4100540 0x7fadb41963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadb37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb4100ee0 0x7fadb41968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadb3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fadb4100540 0x7fadb41963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41754/0 (socket says 172.21.15.8:41754) 2023-11-27T15:22:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadb3fff700 1 -- 172.21.15.8:0/3116119151 learned_addr learned my addr 172.21.15.8:0/3116119151 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:45.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadb8bfb700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadb41055e0 0x7fadb419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadb3fff700 1 -- 172.21.15.8:0/3116119151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb4100ee0 msgr2=0x7fadb41968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadb3fff700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb4100ee0 0x7fadb41968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadb3fff700 1 -- 172.21.15.8:0/3116119151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadb41055e0 msgr2=0x7fadb419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadb3fff700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadb41055e0 0x7fadb419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.058+0000 7fadb3fff700 1 -- 172.21.15.8:0/3116119151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada40095f0 con 0x7fadb4100540 2023-11-27T15:22:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.059+0000 7fadb3fff700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fadb4100540 0x7fadb41963a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fada400b580 tx=0x7fada4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.059+0000 7fadb17fa700 1 -- 172.21.15.8:0/3116119151 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada401d070 con 0x7fadb4100540 2023-11-27T15:22:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.059+0000 7fadb17fa700 1 -- 172.21.15.8:0/3116119151 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fada4003f90 con 0x7fadb4100540 2023-11-27T15:22:45.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.059+0000 7fadba65e700 1 -- 172.21.15.8:0/3116119151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb4110320 con 0x7fadb4100540 2023-11-27T15:22:45.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.059+0000 7fadb17fa700 1 -- 172.21.15.8:0/3116119151 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada4022c10 con 0x7fadb4100540 2023-11-27T15:22:45.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.059+0000 7fadba65e700 1 -- 172.21.15.8:0/3116119151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb4110190 con 0x7fadb4100540 2023-11-27T15:22:45.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.060+0000 7fadba65e700 1 -- 172.21.15.8:0/3116119151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb41104b0 con 0x7fadb4100540 2023-11-27T15:22:45.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.062+0000 7fadb17fa700 1 -- 172.21.15.8:0/3116119151 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fada4022470 con 0x7fadb4100540 2023-11-27T15:22:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.064+0000 7fadb17fa700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fad9c0a11d0 0x7fad9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.064+0000 7fadb17fa700 1 -- 172.21.15.8:0/3116119151 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fada40cd870 con 0x7fadb4100540 2023-11-27T15:22:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.064+0000 7fadb37fe700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fad9c0a11d0 0x7fad9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.064+0000 7fadb37fe700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fad9c0a11d0 0x7fad9c0a3680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fadb41979c0 tx=0x7fada800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:45.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.068+0000 7fadb17fa700 1 -- 172.21.15.8:0/3116119151 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fada4097450 con 0x7fadb4100540 2023-11-27T15:22:45.276 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.273+0000 7fadba65e700 1 -- 172.21.15.8:0/3116119151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fadb41a3420 con 0x7fadb4100540 2023-11-27T15:22:45.277 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.273+0000 7fadb17fa700 1 -- 172.21.15.8:0/3116119151 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fada409aa70 con 0x7fadb4100540 2023-11-27T15:22:45.277 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:22:45.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 -- 172.21.15.8:0/3116119151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fad9c0a11d0 msgr2=0x7fad9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fad9c0a11d0 0x7fad9c0a3680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fadb41979c0 tx=0x7fada800b410 comp rx=0 tx=0).stop 2023-11-27T15:22:45.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 -- 172.21.15.8:0/3116119151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fadb4100540 msgr2=0x7fadb41963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fadb4100540 0x7fadb41963a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fada400b580 tx=0x7fada4005740 comp rx=0 tx=0).stop 2023-11-27T15:22:45.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 -- 172.21.15.8:0/3116119151 shutdown_connections 2023-11-27T15:22:45.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fadb41055e0 0x7fadb419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fadb4100540 0x7fadb41963a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fad9c0a11d0 0x7fad9c0a3680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 --2- 172.21.15.8:0/3116119151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fadb4100ee0 0x7fadb41968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 -- 172.21.15.8:0/3116119151 >> 172.21.15.8:0/3116119151 conn(0x7fadb40fbdc0 msgr2=0x7fadb40fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:45.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 -- 172.21.15.8:0/3116119151 shutdown_connections 2023-11-27T15:22:45.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:45.276+0000 7fadba65e700 1 -- 172.21.15.8:0/3116119151 wait complete. 2023-11-27T15:22:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:45 smithi008 ceph-mon[128788]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:45 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3116119151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:45.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:45 smithi029 ceph-mon[131166]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:45.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:45 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3116119151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:45.773 INFO:tasks.cephadm:Adding mds.a on smithi008 2023-11-27T15:22:45.773 INFO:tasks.cephadm:Adding mds.d on smithi008 2023-11-27T15:22:45.773 INFO:tasks.cephadm:Adding mds.g on smithi008 2023-11-27T15:22:45.774 INFO:tasks.cephadm:Adding mds.j on smithi008 2023-11-27T15:22:45.774 INFO:tasks.cephadm:Adding mds.b on smithi029 2023-11-27T15:22:45.774 INFO:tasks.cephadm:Adding mds.e on smithi029 2023-11-27T15:22:45.774 INFO:tasks.cephadm:Adding mds.h on smithi029 2023-11-27T15:22:45.774 INFO:tasks.cephadm:Adding mds.k on smithi029 2023-11-27T15:22:45.774 INFO:tasks.cephadm:Adding mds.c on smithi060 2023-11-27T15:22:45.774 INFO:tasks.cephadm:Adding mds.f on smithi060 2023-11-27T15:22:45.774 INFO:tasks.cephadm:Adding mds.i on smithi060 2023-11-27T15:22:45.774 INFO:tasks.cephadm:Adding mds.l on smithi060 2023-11-27T15:22:45.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph orch apply mds all '12;smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi029=b;smithi029=e;smithi029=h;smithi029=k;smithi060=c;smithi060=f;smithi060=i;smithi060=l' 2023-11-27T15:22:45.808 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:45 smithi060 ceph-mon[131518]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:45.809 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:45 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3116119151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:46.440 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:22:47.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:47 smithi029 ceph-mon[131166]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:47.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.658+0000 7f66395a0700 1 -- 172.21.15.60:0/2147274502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6634100360 msgr2=0x7f6634100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.658+0000 7f66395a0700 1 --2- 172.21.15.60:0/2147274502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6634100360 0x7f6634100770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6624009990 tx=0x7f6624009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.658+0000 7f66395a0700 1 -- 172.21.15.60:0/2147274502 shutdown_connections 2023-11-27T15:22:47.663 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.658+0000 7f66395a0700 1 --2- 172.21.15.60:0/2147274502 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6634100360 0x7f6634100770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.658+0000 7f66395a0700 1 --2- 172.21.15.60:0/2147274502 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6634101ef0 0x7f66341087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.658+0000 7f66395a0700 1 --2- 172.21.15.60:0/2147274502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6634101560 0x7f66341019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.659+0000 7f66395a0700 1 -- 172.21.15.60:0/2147274502 >> 172.21.15.60:0/2147274502 conn(0x7f66340fb970 msgr2=0x7f66340fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.659+0000 7f66395a0700 1 -- 172.21.15.60:0/2147274502 shutdown_connections 2023-11-27T15:22:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.659+0000 7f66395a0700 1 -- 172.21.15.60:0/2147274502 wait complete. 2023-11-27T15:22:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.659+0000 7f66395a0700 1 Processor -- start 2023-11-27T15:22:47.664 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.659+0000 7f66395a0700 1 -- start start 2023-11-27T15:22:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f66395a0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6634100360 0x7f6634196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f66395a0700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6634101560 0x7f66341965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f66395a0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6634101ef0 0x7f663419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f66395a0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663419dc20 con 0x7f6634101ef0 2023-11-27T15:22:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f66395a0700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663419dd60 con 0x7f6634101560 2023-11-27T15:22:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f66395a0700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663419def0 con 0x7f6634100360 2023-11-27T15:22:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f6632ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6634100360 0x7f6634196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f6632ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6634100360 0x7f6634196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.60:35272/0 (socket says 172.21.15.60:35272) 2023-11-27T15:22:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f6632ffd700 1 -- 172.21.15.60:0/3035852282 learned_addr learned my addr 172.21.15.60:0/3035852282 (peer_addr_for_me v2:172.21.15.60:0/0) 2023-11-27T15:22:47.666 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f66327fc700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6634101560 0x7f66341965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f66337fe700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6634101ef0 0x7f663419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f6632ffd700 1 -- 172.21.15.60:0/3035852282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6634101560 msgr2=0x7f66341965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f6632ffd700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6634101560 0x7f66341965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f6632ffd700 1 -- 172.21.15.60:0/3035852282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6634101ef0 msgr2=0x7f663419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f6632ffd700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6634101ef0 0x7f663419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f6632ffd700 1 -- 172.21.15.60:0/3035852282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240095f0 con 0x7f6634100360 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f6632ffd700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6634100360 0x7f6634196090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f662400b5c0 tx=0x7f662400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f661bfff700 1 -- 172.21.15.60:0/3035852282 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662401d070 con 0x7f6634100360 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f66395a0700 1 -- 172.21.15.60:0/3035852282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663410bd30 con 0x7f6634100360 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f661bfff700 1 -- 172.21.15.60:0/3035852282 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6624022470 con 0x7f6634100360 2023-11-27T15:22:47.667 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f661bfff700 1 -- 172.21.15.60:0/3035852282 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624017630 con 0x7f6634100360 2023-11-27T15:22:47.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.660+0000 7f66395a0700 1 -- 172.21.15.60:0/3035852282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663410bba0 con 0x7f6634100360 2023-11-27T15:22:47.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.661+0000 7f66395a0700 1 -- 172.21.15.60:0/3035852282 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66341072f0 con 0x7f6634100360 2023-11-27T15:22:47.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.662+0000 7f661bfff700 1 -- 172.21.15.60:0/3035852282 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f66240228e0 con 0x7f6634100360 2023-11-27T15:22:47.668 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.663+0000 7f661bfff700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f661c0a1150 0x7f661c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.663+0000 7f661bfff700 1 -- 172.21.15.60:0/3035852282 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66240cd680 con 0x7f6634100360 2023-11-27T15:22:47.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.664+0000 7f66327fc700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f661c0a1150 0x7f661c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.669 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.664+0000 7f66327fc700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f661c0a1150 0x7f661c0a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6628009a10 tx=0x7f6628009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:47.671 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.667+0000 7f661bfff700 1 -- 172.21.15.60:0/3035852282 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f662405c730 con 0x7f6634100360 2023-11-27T15:22:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:47 smithi060 ceph-mon[131518]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:47 smithi008 ceph-mon[128788]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:47.881 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.877+0000 7f66395a0700 1 -- 172.21.15.60:0/3035852282 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi029=b;smithi029=e;smithi029=h;smithi029=k;smithi060=c;smithi060=f;smithi060=i;smithi060=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f6634061bb0 con 0x7f661c0a1150 2023-11-27T15:22:47.889 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.886+0000 7f661bfff700 1 -- 172.21.15.60:0/3035852282 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f6634061bb0 con 0x7f661c0a1150 2023-11-27T15:22:47.890 INFO:teuthology.orchestra.run.smithi060.stdout:Scheduled mds.all update... 2023-11-27T15:22:47.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 -- 172.21.15.60:0/3035852282 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f661c0a1150 msgr2=0x7f661c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f661c0a1150 0x7f661c0a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6628009a10 tx=0x7f6628009190 comp rx=0 tx=0).stop 2023-11-27T15:22:47.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 -- 172.21.15.60:0/3035852282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6634100360 msgr2=0x7f6634196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6634100360 0x7f6634196090 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f662400b5c0 tx=0x7f662400f690 comp rx=0 tx=0).stop 2023-11-27T15:22:47.892 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 -- 172.21.15.60:0/3035852282 shutdown_connections 2023-11-27T15:22:47.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f661c0a1150 0x7f661c0a3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6634100360 0x7f6634196090 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6634101ef0 0x7f663419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 --2- 172.21.15.60:0/3035852282 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6634101560 0x7f66341965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 -- 172.21.15.60:0/3035852282 >> 172.21.15.60:0/3035852282 conn(0x7f66340fb970 msgr2=0x7f66340fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:47.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 -- 172.21.15.60:0/3035852282 shutdown_connections 2023-11-27T15:22:47.893 INFO:teuthology.orchestra.run.smithi060.stderr:2023-11-27T15:22:47.888+0000 7f66395a0700 1 -- 172.21.15.60:0/3035852282 wait complete. 2023-11-27T15:22:48.294 DEBUG:teuthology.orchestra.run.smithi008:mds.a> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.a.service 2023-11-27T15:22:48.300 DEBUG:teuthology.orchestra.run.smithi008:mds.d> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.d.service 2023-11-27T15:22:48.346 DEBUG:teuthology.orchestra.run.smithi008:mds.g> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.g.service 2023-11-27T15:22:48.380 INFO:journalctl@ceph.mds.a.smithi008.stdout:-- Logs begin at Mon 2023-11-27 14:56:10 UTC. -- 2023-11-27T15:22:48.384 DEBUG:teuthology.orchestra.run.smithi008:mds.j> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.j.service 2023-11-27T15:22:48.426 INFO:journalctl@ceph.mds.d.smithi008.stdout:-- Logs begin at Mon 2023-11-27 14:56:10 UTC. -- 2023-11-27T15:22:48.431 DEBUG:teuthology.orchestra.run.smithi029:mds.b> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.b.service 2023-11-27T15:22:48.433 DEBUG:teuthology.orchestra.run.smithi029:mds.e> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.e.service 2023-11-27T15:22:48.459 INFO:journalctl@ceph.mds.g.smithi008.stdout:-- Logs begin at Mon 2023-11-27 14:56:10 UTC. -- 2023-11-27T15:22:48.464 INFO:journalctl@ceph.mds.b.smithi029.stdout:-- Logs begin at Mon 2023-11-27 14:52:42 UTC. -- 2023-11-27T15:22:48.465 DEBUG:teuthology.orchestra.run.smithi029:mds.h> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.h.service 2023-11-27T15:22:48.491 INFO:journalctl@ceph.mds.e.smithi029.stdout:-- Logs begin at Mon 2023-11-27 14:52:42 UTC. -- 2023-11-27T15:22:48.493 DEBUG:teuthology.orchestra.run.smithi029:mds.k> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.k.service 2023-11-27T15:22:48.502 INFO:journalctl@ceph.mds.j.smithi008.stdout:-- Logs begin at Mon 2023-11-27 14:56:10 UTC. -- 2023-11-27T15:22:48.519 INFO:journalctl@ceph.mds.h.smithi029.stdout:-- Logs begin at Mon 2023-11-27 14:52:42 UTC. -- 2023-11-27T15:22:48.520 DEBUG:teuthology.orchestra.run.smithi060:mds.c> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.c.service 2023-11-27T15:22:48.523 DEBUG:teuthology.orchestra.run.smithi060:mds.f> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.f.service 2023-11-27T15:22:48.547 INFO:journalctl@ceph.mds.k.smithi029.stdout:-- Logs begin at Mon 2023-11-27 14:52:42 UTC. -- 2023-11-27T15:22:48.554 INFO:journalctl@ceph.mds.c.smithi060.stdout:-- Logs begin at Mon 2023-11-27 14:51:59 UTC. -- 2023-11-27T15:22:48.555 DEBUG:teuthology.orchestra.run.smithi060:mds.i> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i.service 2023-11-27T15:22:48.582 INFO:journalctl@ceph.mds.f.smithi060.stdout:-- Logs begin at Mon 2023-11-27 14:51:59 UTC. -- 2023-11-27T15:22:48.583 DEBUG:teuthology.orchestra.run.smithi060:mds.l> sudo journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.l.service 2023-11-27T15:22:48.610 INFO:journalctl@ceph.mds.i.smithi060.stdout:-- Logs begin at Mon 2023-11-27 14:51:59 UTC. -- 2023-11-27T15:22:48.610 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-11-27T15:22:48.611 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:48.637 INFO:journalctl@ceph.mds.l.smithi060.stdout:-- Logs begin at Mon 2023-11-27 14:51:59 UTC. -- 2023-11-27T15:22:48.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.819+0000 7f69b7dff700 1 -- 172.21.15.8:0/383102386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69b0101e50 msgr2=0x7f69b0108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.819+0000 7f69b7dff700 1 --2- 172.21.15.8:0/383102386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69b0101e50 0x7f69b0108710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f69a800b0a0 tx=0x7f69a800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.819+0000 7f69b7dff700 1 -- 172.21.15.8:0/383102386 shutdown_connections 2023-11-27T15:22:48.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.819+0000 7f69b7dff700 1 --2- 172.21.15.8:0/383102386 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69b0101e50 0x7f69b0108710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.819+0000 7f69b7dff700 1 --2- 172.21.15.8:0/383102386 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69b01002d0 0x7f69b01006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.819+0000 7f69b7dff700 1 --2- 172.21.15.8:0/383102386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69b01014c0 0x7f69b0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.819+0000 7f69b7dff700 1 -- 172.21.15.8:0/383102386 >> 172.21.15.8:0/383102386 conn(0x7f69b00fb8b0 msgr2=0x7f69b00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:48.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.819+0000 7f69b7dff700 1 -- 172.21.15.8:0/383102386 shutdown_connections 2023-11-27T15:22:48.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.820+0000 7f69b7dff700 1 -- 172.21.15.8:0/383102386 wait complete. 2023-11-27T15:22:48.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.820+0000 7f69b7dff700 1 Processor -- start 2023-11-27T15:22:48.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b7dff700 1 -- start start 2023-11-27T15:22:48.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b7dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69b01002d0 0x7f69b0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b7dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69b01014c0 0x7f69b0198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b7dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69b0101e50 0x7f69b019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b7dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b019fca0 con 0x7f69b01014c0 2023-11-27T15:22:48.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b7dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b019fde0 con 0x7f69b01002d0 2023-11-27T15:22:48.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b7dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b019ff70 con 0x7f69b0101e50 2023-11-27T15:22:48.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b539a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69b01014c0 0x7f69b0198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b5b9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69b01002d0 0x7f69b0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b639c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69b0101e50 0x7f69b019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b5b9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69b01002d0 0x7f69b0198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50922/0 (socket says 172.21.15.8:50922) 2023-11-27T15:22:48.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.821+0000 7f69b5b9b700 1 -- 172.21.15.8:0/2279767130 learned_addr learned my addr 172.21.15.8:0/2279767130 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:48.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69b5b9b700 1 -- 172.21.15.8:0/2279767130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69b0101e50 msgr2=0x7f69b019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69b5b9b700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69b0101e50 0x7f69b019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69b5b9b700 1 -- 172.21.15.8:0/2279767130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69b01014c0 msgr2=0x7f69b0198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69b5b9b700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69b01014c0 0x7f69b0198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69b5b9b700 1 -- 172.21.15.8:0/2279767130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a0009520 con 0x7f69b01002d0 2023-11-27T15:22:48.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69b639c700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69b0101e50 0x7f69b019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:48.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69b5b9b700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69b01002d0 0x7f69b0198120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00e970 tx=0x7f69ac00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:48.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69a6ffd700 1 -- 172.21.15.8:0/2279767130 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac009790 con 0x7f69b01002d0 2023-11-27T15:22:48.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69b7dff700 1 -- 172.21.15.8:0/2279767130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a8009d00 con 0x7f69b01002d0 2023-11-27T15:22:48.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69a6ffd700 1 -- 172.21.15.8:0/2279767130 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69ac00c7e0 con 0x7f69b01002d0 2023-11-27T15:22:48.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69a6ffd700 1 -- 172.21.15.8:0/2279767130 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac003c70 con 0x7f69b01002d0 2023-11-27T15:22:48.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.822+0000 7f69b7dff700 1 -- 172.21.15.8:0/2279767130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b010bbe0 con 0x7f69b01002d0 2023-11-27T15:22:48.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.823+0000 7f69b7dff700 1 -- 172.21.15.8:0/2279767130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b0067af0 con 0x7f69b01002d0 2023-11-27T15:22:48.829 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.825+0000 7f69a6ffd700 1 -- 172.21.15.8:0/2279767130 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f69ac00cc50 con 0x7f69b01002d0 2023-11-27T15:22:48.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.826+0000 7f69a6ffd700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f699c0a1150 0x7f699c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.827+0000 7f69a6ffd700 1 -- 172.21.15.8:0/2279767130 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f69ac014070 con 0x7f69b01002d0 2023-11-27T15:22:48.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.827+0000 7f69b539a700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f699c0a1150 0x7f699c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.827+0000 7f69b539a700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f699c0a1150 0x7f699c0a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009df0 tx=0x7f69a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:48.830+0000 7f69a6ffd700 1 -- 172.21.15.8:0/2279767130 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f69ac096210 con 0x7f69b01002d0 2023-11-27T15:22:49.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.041+0000 7f69b7dff700 1 -- 172.21.15.8:0/2279767130 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69b01a0ae0 con 0x7f69b01002d0 2023-11-27T15:22:49.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.042+0000 7f69a6ffd700 1 -- 172.21.15.8:0/2279767130 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18279 (secure 0 0 0) 0x7f69ac003dd0 con 0x7f69b01002d0 2023-11-27T15:22:49.047 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:22:49.047 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":80,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:22:36.503805+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:49.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 -- 172.21.15.8:0/2279767130 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f699c0a1150 msgr2=0x7f699c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f699c0a1150 0x7f699c0a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009df0 tx=0x7f69a0009260 comp rx=0 tx=0).stop 2023-11-27T15:22:49.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 -- 172.21.15.8:0/2279767130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69b01002d0 msgr2=0x7f69b0198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69b01002d0 0x7f69b0198120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00e970 tx=0x7f69ac00ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:49.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 -- 172.21.15.8:0/2279767130 shutdown_connections 2023-11-27T15:22:49.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f699c0a1150 0x7f699c0a3600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f69b0101e50 0x7f69b019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f69b01002d0 0x7f69b0198120 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 --2- 172.21.15.8:0/2279767130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69b01014c0 0x7f69b0198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 -- 172.21.15.8:0/2279767130 >> 172.21.15.8:0/2279767130 conn(0x7f69b00fb8b0 msgr2=0x7f69b0106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:49.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 -- 172.21.15.8:0/2279767130 shutdown_connections 2023-11-27T15:22:49.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.045+0000 7f69b7dff700 1 -- 172.21.15.8:0/2279767130 wait complete. 2023-11-27T15:22:49.067 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi029=b;smithi029=e;smithi029=h;smithi029=k;smithi060=c;smithi060=f;smithi060=i;smithi060=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:49.067 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: Saving service mds.all spec with placement smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi029=b;smithi029=e;smithi029=h;smithi029=k;smithi060=c;smithi060=f;smithi060=i;smithi060=l;count:12 2023-11-27T15:22:49.067 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:49.067 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:49.067 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:49.067 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:49.067 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:49.067 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:22:49.067 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:49.068 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:22:49.068 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:22:49.068 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:22:49.068 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:49.068 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:49.068 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:49.068 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:48 smithi008 ceph-mon[128788]: Deploying daemon mds.l on smithi060 2023-11-27T15:22:49.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:22:49.070 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:22:49.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi029=b;smithi029=e;smithi029=h;smithi029=k;smithi060=c;smithi060=f;smithi060=i;smithi060=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:49.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: Saving service mds.all spec with placement smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi029=b;smithi029=e;smithi029=h;smithi029=k;smithi060=c;smithi060=f;smithi060=i;smithi060=l;count:12 2023-11-27T15:22:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:22:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:22:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:22:49.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:22:49.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:49.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:49.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:49.149 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:48 smithi029 ceph-mon[131166]: Deploying daemon mds.l on smithi060 2023-11-27T15:22:49.241 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi029=b;smithi029=e;smithi029=h;smithi029=k;smithi060=c;smithi060=f;smithi060=i;smithi060=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:49.241 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: Saving service mds.all spec with placement smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi029=b;smithi029=e;smithi029=h;smithi029=k;smithi060=c;smithi060=f;smithi060=i;smithi060=l;count:12 2023-11-27T15:22:49.241 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:49.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:49.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:49.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:49.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:49.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:22:49.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:49.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:22:49.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:22:49.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:22:49.243 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:49.243 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:49.243 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:49.243 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:48 smithi060 ceph-mon[131518]: Deploying daemon mds.l on smithi060 2023-11-27T15:22:49.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.274+0000 7f86429ff700 1 -- 172.21.15.8:0/3920207446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c100290 msgr2=0x7f863c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.274+0000 7f86429ff700 1 --2- 172.21.15.8:0/3920207446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c100290 0x7f863c1006a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f862c009990 tx=0x7f862c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.274+0000 7f86429ff700 1 -- 172.21.15.8:0/3920207446 shutdown_connections 2023-11-27T15:22:49.278 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.274+0000 7f86429ff700 1 --2- 172.21.15.8:0/3920207446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c100290 0x7f863c1006a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.274+0000 7f86429ff700 1 --2- 172.21.15.8:0/3920207446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c101e10 0x7f863c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.274+0000 7f86429ff700 1 --2- 172.21.15.8:0/3920207446 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.274+0000 7f86429ff700 1 -- 172.21.15.8:0/3920207446 >> 172.21.15.8:0/3920207446 conn(0x7f863c0fb870 msgr2=0x7f863c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:49.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.274+0000 7f86429ff700 1 -- 172.21.15.8:0/3920207446 shutdown_connections 2023-11-27T15:22:49.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.274+0000 7f86429ff700 1 -- 172.21.15.8:0/3920207446 wait complete. 2023-11-27T15:22:49.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.275+0000 7f86429ff700 1 Processor -- start 2023-11-27T15:22:49.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f86429ff700 1 -- start start 2023-11-27T15:22:49.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f86429ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c100290 0x7f863c193d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f86429ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f86429ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c101e10 0x7f863c1943b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f86429ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c1ac900 con 0x7f863c101480 2023-11-27T15:22:49.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f86429ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c1aca40 con 0x7f863c100290 2023-11-27T15:22:49.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f86429ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c1acc00 con 0x7f863c101e10 2023-11-27T15:22:49.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f863b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f863b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41802/0 (socket says 172.21.15.8:41802) 2023-11-27T15:22:49.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f863b7fe700 1 -- 172.21.15.8:0/2232056617 learned_addr learned my addr 172.21.15.8:0/2232056617 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:49.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f8640f9c700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c101e10 0x7f863c1943b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f863bfff700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c100290 0x7f863c193d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f863b7fe700 1 -- 172.21.15.8:0/2232056617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c101e10 msgr2=0x7f863c1943b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f863b7fe700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c101e10 0x7f863c1943b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f863b7fe700 1 -- 172.21.15.8:0/2232056617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c100290 msgr2=0x7f863c193d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.282 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f863b7fe700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c100290 0x7f863c193d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.276+0000 7f863b7fe700 1 -- 172.21.15.8:0/2232056617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0095f0 con 0x7f863c101480 2023-11-27T15:22:49.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.277+0000 7f863b7fe700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c199320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f863000e9a0 tx=0x7f863000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:49.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.277+0000 7f86397fa700 1 -- 172.21.15.8:0/2232056617 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630009790 con 0x7f863c101480 2023-11-27T15:22:49.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.277+0000 7f86429ff700 1 -- 172.21.15.8:0/2232056617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c10bbe0 con 0x7f863c101480 2023-11-27T15:22:49.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.277+0000 7f86397fa700 1 -- 172.21.15.8:0/2232056617 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8630010430 con 0x7f863c101480 2023-11-27T15:22:49.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.277+0000 7f86397fa700 1 -- 172.21.15.8:0/2232056617 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630018680 con 0x7f863c101480 2023-11-27T15:22:49.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.277+0000 7f86429ff700 1 -- 172.21.15.8:0/2232056617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c10ba50 con 0x7f863c101480 2023-11-27T15:22:49.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.278+0000 7f86429ff700 1 -- 172.21.15.8:0/2232056617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863c103e40 con 0x7f863c101480 2023-11-27T15:22:49.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.280+0000 7f86397fa700 1 -- 172.21.15.8:0/2232056617 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f86300105a0 con 0x7f863c101480 2023-11-27T15:22:49.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.281+0000 7f86397fa700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86240a1150 0x7f86240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.282+0000 7f86397fa700 1 -- 172.21.15.8:0/2232056617 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8630014070 con 0x7f863c101480 2023-11-27T15:22:49.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.282+0000 7f863bfff700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86240a1150 0x7f86240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.282+0000 7f863bfff700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86240a1150 0x7f86240a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f862c00b2e0 tx=0x7f862c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:49.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.285+0000 7f86397fa700 1 -- 172.21.15.8:0/2232056617 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8630051730 con 0x7f863c101480 2023-11-27T15:22:49.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.497+0000 7f86429ff700 1 -- 172.21.15.8:0/2232056617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f863c067aa0 con 0x7f863c101480 2023-11-27T15:22:49.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.498+0000 7f86397fa700 1 -- 172.21.15.8:0/2232056617 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8630096210 con 0x7f863c101480 2023-11-27T15:22:49.502 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:22:49.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.500+0000 7f86429ff700 1 -- 172.21.15.8:0/2232056617 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86240a1150 msgr2=0x7f86240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.500+0000 7f86429ff700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86240a1150 0x7f86240a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f862c00b2e0 tx=0x7f862c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.500+0000 7f86429ff700 1 -- 172.21.15.8:0/2232056617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 msgr2=0x7f863c199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.500+0000 7f86429ff700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c199320 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f863000e9a0 tx=0x7f863000ecb0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.500+0000 7f86429ff700 1 -- 172.21.15.8:0/2232056617 shutdown_connections 2023-11-27T15:22:49.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.500+0000 7f86429ff700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86240a1150 0x7f86240a3600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.500+0000 7f86429ff700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c101e10 0x7f863c1943b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.500+0000 7f86429ff700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c100290 0x7f863c193d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.500+0000 7f86429ff700 1 --2- 172.21.15.8:0/2232056617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c199320 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.500+0000 7f86429ff700 1 -- 172.21.15.8:0/2232056617 >> 172.21.15.8:0/2232056617 conn(0x7f863c0fb870 msgr2=0x7f863c106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:49.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.501+0000 7f86429ff700 1 -- 172.21.15.8:0/2232056617 shutdown_connections 2023-11-27T15:22:49.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.501+0000 7f86429ff700 1 -- 172.21.15.8:0/2232056617 wait complete. 2023-11-27T15:22:49.524 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:49.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.722+0000 7f4a517ff700 1 -- 172.21.15.8:0/135517824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a4c1002d0 msgr2=0x7f4a4c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.722+0000 7f4a517ff700 1 --2- 172.21.15.8:0/135517824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a4c1002d0 0x7f4a4c1006e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009990 tx=0x7f4a3c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.723+0000 7f4a517ff700 1 -- 172.21.15.8:0/135517824 shutdown_connections 2023-11-27T15:22:49.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.723+0000 7f4a517ff700 1 --2- 172.21.15.8:0/135517824 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a4c1002d0 0x7f4a4c1006e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.723+0000 7f4a517ff700 1 --2- 172.21.15.8:0/135517824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a4c101e50 0x7f4a4c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.723+0000 7f4a517ff700 1 --2- 172.21.15.8:0/135517824 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4c1014c0 0x7f4a4c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.723+0000 7f4a517ff700 1 -- 172.21.15.8:0/135517824 >> 172.21.15.8:0/135517824 conn(0x7f4a4c0fb8b0 msgr2=0x7f4a4c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:49.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.723+0000 7f4a517ff700 1 -- 172.21.15.8:0/135517824 shutdown_connections 2023-11-27T15:22:49.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.723+0000 7f4a517ff700 1 -- 172.21.15.8:0/135517824 wait complete. 2023-11-27T15:22:49.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a517ff700 1 Processor -- start 2023-11-27T15:22:49.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a517ff700 1 -- start start 2023-11-27T15:22:49.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a517ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a4c1002d0 0x7f4a4c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a517ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4c1014c0 0x7f4a4c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a517ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a4c101e50 0x7f4a4c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a517ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c19fda0 con 0x7f4a4c1002d0 2023-11-27T15:22:49.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a517ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c19fee0 con 0x7f4a4c1014c0 2023-11-27T15:22:49.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a517ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c1a0070 con 0x7f4a4c101e50 2023-11-27T15:22:49.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a4affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a4c1002d0 0x7f4a4c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a4b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a4c101e50 0x7f4a4c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a4affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a4c1002d0 0x7f4a4c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41816/0 (socket says 172.21.15.8:41816) 2023-11-27T15:22:49.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a4affd700 1 -- 172.21.15.8:0/1096583718 learned_addr learned my addr 172.21.15.8:0/1096583718 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:49.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.724+0000 7f4a43fff700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4c1014c0 0x7f4a4c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a4affd700 1 -- 172.21.15.8:0/1096583718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a4c101e50 msgr2=0x7f4a4c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a4affd700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a4c101e50 0x7f4a4c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a4affd700 1 -- 172.21.15.8:0/1096583718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4c1014c0 msgr2=0x7f4a4c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a4affd700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4c1014c0 0x7f4a4c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a4affd700 1 -- 172.21.15.8:0/1096583718 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c0095f0 con 0x7f4a4c1002d0 2023-11-27T15:22:49.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a4affd700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a4c1002d0 0x7f4a4c198220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c004710 tx=0x7f4a3c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:49.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a48ff9700 1 -- 172.21.15.8:0/1096583718 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c01d070 con 0x7f4a4c1002d0 2023-11-27T15:22:49.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a517ff700 1 -- 172.21.15.8:0/1096583718 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c10bc20 con 0x7f4a4c1002d0 2023-11-27T15:22:49.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a48ff9700 1 -- 172.21.15.8:0/1096583718 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3c021410 con 0x7f4a4c1002d0 2023-11-27T15:22:49.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a48ff9700 1 -- 172.21.15.8:0/1096583718 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c017650 con 0x7f4a4c1002d0 2023-11-27T15:22:49.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.725+0000 7f4a517ff700 1 -- 172.21.15.8:0/1096583718 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c10ba90 con 0x7f4a4c1002d0 2023-11-27T15:22:49.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.727+0000 7f4a517ff700 1 -- 172.21.15.8:0/1096583718 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c192980 con 0x7f4a4c1002d0 2023-11-27T15:22:49.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.729+0000 7f4a48ff9700 1 -- 172.21.15.8:0/1096583718 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4a3c021580 con 0x7f4a4c1002d0 2023-11-27T15:22:49.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.730+0000 7f4a48ff9700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4a380a1150 0x7f4a380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.730+0000 7f4a48ff9700 1 -- 172.21.15.8:0/1096583718 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4a3c0cd800 con 0x7f4a4c1002d0 2023-11-27T15:22:49.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.730+0000 7f4a43fff700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4a380a1150 0x7f4a380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.731+0000 7f4a43fff700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4a380a1150 0x7f4a380a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009c60 tx=0x7f4a34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:49.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.734+0000 7f4a48ff9700 1 -- 172.21.15.8:0/1096583718 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4a3c097480 con 0x7f4a4c1002d0 2023-11-27T15:22:49.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.992+0000 7f4a517ff700 1 -- 172.21.15.8:0/1096583718 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a4c02cfe0 con 0x7f4a4c1002d0 2023-11-27T15:22:49.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.993+0000 7f4a48ff9700 1 -- 172.21.15.8:0/1096583718 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f4a3c09aaa0 con 0x7f4a4c1002d0 2023-11-27T15:22:49.997 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:22:49.997 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-11-27T15:22:49.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.995+0000 7f4a517ff700 1 -- 172.21.15.8:0/1096583718 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4a380a1150 msgr2=0x7f4a380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.995+0000 7f4a517ff700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4a380a1150 0x7f4a380a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4a34009c60 tx=0x7f4a34009260 comp rx=0 tx=0).stop 2023-11-27T15:22:50.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.995+0000 7f4a517ff700 1 -- 172.21.15.8:0/1096583718 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a4c1002d0 msgr2=0x7f4a4c198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.995+0000 7f4a517ff700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a4c1002d0 0x7f4a4c198220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c004710 tx=0x7f4a3c009580 comp rx=0 tx=0).stop 2023-11-27T15:22:50.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.995+0000 7f4a517ff700 1 -- 172.21.15.8:0/1096583718 shutdown_connections 2023-11-27T15:22:50.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.995+0000 7f4a517ff700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4a380a1150 0x7f4a380a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.995+0000 7f4a517ff700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a4c101e50 0x7f4a4c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.995+0000 7f4a517ff700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a4c1002d0 0x7f4a4c198220 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.995+0000 7f4a517ff700 1 --2- 172.21.15.8:0/1096583718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a4c1014c0 0x7f4a4c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.996+0000 7f4a517ff700 1 -- 172.21.15.8:0/1096583718 >> 172.21.15.8:0/1096583718 conn(0x7f4a4c0fb8b0 msgr2=0x7f4a4c106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.996+0000 7f4a517ff700 1 -- 172.21.15.8:0/1096583718 shutdown_connections 2023-11-27T15:22:50.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:49.996+0000 7f4a517ff700 1 -- 172.21.15.8:0/1096583718 wait complete. 2023-11-27T15:22:50.001 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 1 2023-11-27T15:22:50.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:49 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2279767130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:50.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:49 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2232056617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:50.313 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:49 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2279767130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:50.313 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:49 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2232056617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:50.313 INFO:journalctl@ceph.mds.l.smithi060.stdout:Nov 27 15:22:50 smithi060 systemd[1]: Starting Ceph mds.l for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:22:50.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:49 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2279767130' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:50.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:49 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2232056617' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:50.574 INFO:journalctl@ceph.mds.l.smithi060.stdout:Nov 27 15:22:50 smithi060 podman[156754]: 2023-11-27T15:22:50.575 INFO:journalctl@ceph.mds.l.smithi060.stdout:Nov 27 15:22:50 smithi060 bash[156754]: b49cc785c53964ba7bbabda2c13977d78cb795d74b656d0442b036ab95b10348 2023-11-27T15:22:50.575 INFO:journalctl@ceph.mds.l.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-l[156770]: starting mds.l at 2023-11-27T15:22:50.894 INFO:journalctl@ceph.mds.l.smithi060.stdout:Nov 27 15:22:50 smithi060 systemd[1]: Started Ceph mds.l for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:22:51.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1096583718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:51.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:51.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.60:6832/3958511263,v1:172.21.15.60:6833/3958511263] up:boot 2023-11-27T15:22:51.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: fsmap 1 up:standby 2023-11-27T15:22:51.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:22:51.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:51.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:51.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:51.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:51.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:51.148 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:50 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:51.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1096583718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:51.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:51.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.60:6832/3958511263,v1:172.21.15.60:6833/3958511263] up:boot 2023-11-27T15:22:51.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: fsmap 1 up:standby 2023-11-27T15:22:51.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:22:51.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:51.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:51.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:51.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:51.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:51.326 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:50 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:51.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1096583718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:51.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:51.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.60:6832/3958511263,v1:172.21.15.60:6833/3958511263] up:boot 2023-11-27T15:22:51.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: fsmap 1 up:standby 2023-11-27T15:22:51.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:22:51.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:51.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:51.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:51.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:51.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:51.365 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:50 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:52.019 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:52.110 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:51 smithi029 ceph-mon[131166]: Deploying daemon mds.k on smithi029 2023-11-27T15:22:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.229+0000 7ff928dff700 1 -- 172.21.15.8:0/2920213088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9241002d0 msgr2=0x7ff9241006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.229+0000 7ff928dff700 1 --2- 172.21.15.8:0/2920213088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9241002d0 0x7ff9241006e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff914009990 tx=0x7ff914009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.229+0000 7ff928dff700 1 -- 172.21.15.8:0/2920213088 shutdown_connections 2023-11-27T15:22:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.229+0000 7ff928dff700 1 --2- 172.21.15.8:0/2920213088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff924101e50 0x7ff924108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.229+0000 7ff928dff700 1 --2- 172.21.15.8:0/2920213088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9241002d0 0x7ff9241006e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.229+0000 7ff928dff700 1 --2- 172.21.15.8:0/2920213088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9241014c0 0x7ff924101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.229+0000 7ff928dff700 1 -- 172.21.15.8:0/2920213088 >> 172.21.15.8:0/2920213088 conn(0x7ff9240fb8b0 msgr2=0x7ff9240fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.230+0000 7ff928dff700 1 -- 172.21.15.8:0/2920213088 shutdown_connections 2023-11-27T15:22:52.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.230+0000 7ff928dff700 1 -- 172.21.15.8:0/2920213088 wait complete. 2023-11-27T15:22:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff928dff700 1 Processor -- start 2023-11-27T15:22:52.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff928dff700 1 -- start start 2023-11-27T15:22:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff928dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9241002d0 0x7ff9241982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff928dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9241014c0 0x7ff924198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff928dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff924101e50 0x7ff92419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff928dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92419fe40 con 0x7ff9241014c0 2023-11-27T15:22:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff928dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92419ff80 con 0x7ff924101e50 2023-11-27T15:22:52.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff928dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9241a0110 con 0x7ff9241002d0 2023-11-27T15:22:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff92259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9241002d0 0x7ff9241982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff92259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9241002d0 0x7ff9241982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58590/0 (socket says 172.21.15.8:58590) 2023-11-27T15:22:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff92259c700 1 -- 172.21.15.8:0/459500582 learned_addr learned my addr 172.21.15.8:0/459500582 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff921d9b700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9241014c0 0x7ff924198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.231+0000 7ff922d9d700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff924101e50 0x7ff92419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff922d9d700 1 -- 172.21.15.8:0/459500582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9241002d0 msgr2=0x7ff9241982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:52.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff922d9d700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9241002d0 0x7ff9241982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff922d9d700 1 -- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9241014c0 msgr2=0x7ff924198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:52.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff922d9d700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9241014c0 0x7ff924198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff922d9d700 1 -- 172.21.15.8:0/459500582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c009c40 con 0x7ff924101e50 2023-11-27T15:22:52.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff921d9b700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9241014c0 0x7ff924198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:52.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff922d9d700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff924101e50 0x7ff92419f870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00efb0 tx=0x7ff91c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:52.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff9137fe700 1 -- 172.21.15.8:0/459500582 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c003a00 con 0x7ff924101e50 2023-11-27T15:22:52.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff9137fe700 1 -- 172.21.15.8:0/459500582 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff91c00f040 con 0x7ff924101e50 2023-11-27T15:22:52.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff928dff700 1 -- 172.21.15.8:0/459500582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9140095f0 con 0x7ff924101e50 2023-11-27T15:22:52.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.232+0000 7ff9137fe700 1 -- 172.21.15.8:0/459500582 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c004540 con 0x7ff924101e50 2023-11-27T15:22:52.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.233+0000 7ff928dff700 1 -- 172.21.15.8:0/459500582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92410bca0 con 0x7ff924101e50 2023-11-27T15:22:52.241 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.234+0000 7ff928dff700 1 -- 172.21.15.8:0/459500582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff924067af0 con 0x7ff924101e50 2023-11-27T15:22:52.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.235+0000 7ff9137fe700 1 -- 172.21.15.8:0/459500582 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff91c003bf0 con 0x7ff924101e50 2023-11-27T15:22:52.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.237+0000 7ff9137fe700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff90c0a11d0 0x7ff90c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:52.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.237+0000 7ff9137fe700 1 -- 172.21.15.8:0/459500582 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff91c0cc080 con 0x7ff924101e50 2023-11-27T15:22:52.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.237+0000 7ff92259c700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff90c0a11d0 0x7ff90c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:52.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.238+0000 7ff92259c700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff90c0a11d0 0x7ff90c0a3680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff914006010 tx=0x7ff914005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:52.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.241+0000 7ff9137fe700 1 -- 172.21.15.8:0/459500582 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff91c095c60 con 0x7ff924101e50 2023-11-27T15:22:52.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:51 smithi060 ceph-mon[131518]: Deploying daemon mds.k on smithi029 2023-11-27T15:22:52.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:51 smithi008 ceph-mon[128788]: Deploying daemon mds.k on smithi029 2023-11-27T15:22:52.397 INFO:journalctl@ceph.mds.k.smithi029.stdout:Nov 27 15:22:52 smithi029 systemd[1]: Starting Ceph mds.k for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:22:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.494+0000 7ff928dff700 1 -- 172.21.15.8:0/459500582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9241a0c70 con 0x7ff924101e50 2023-11-27T15:22:52.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.495+0000 7ff9137fe700 1 -- 172.21.15.8:0/459500582 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7ff91c084500 con 0x7ff924101e50 2023-11-27T15:22:52.499 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:22:52.499 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-27T15:22:52.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 -- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff90c0a11d0 msgr2=0x7ff90c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:52.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff90c0a11d0 0x7ff90c0a3680 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff914006010 tx=0x7ff914005c40 comp rx=0 tx=0).stop 2023-11-27T15:22:52.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 -- 172.21.15.8:0/459500582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff924101e50 msgr2=0x7ff92419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:52.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff924101e50 0x7ff92419f870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00efb0 tx=0x7ff91c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 -- 172.21.15.8:0/459500582 shutdown_connections 2023-11-27T15:22:52.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff90c0a11d0 0x7ff90c0a3680 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9241002d0 0x7ff9241982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff924101e50 0x7ff92419f870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 --2- 172.21.15.8:0/459500582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9241014c0 0x7ff924198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:52.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 -- 172.21.15.8:0/459500582 >> 172.21.15.8:0/459500582 conn(0x7ff9240fb8b0 msgr2=0x7ff9240fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:52.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 -- 172.21.15.8:0/459500582 shutdown_connections 2023-11-27T15:22:52.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:52.498+0000 7ff928dff700 1 -- 172.21.15.8:0/459500582 wait complete. 2023-11-27T15:22:52.504 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 2 2023-11-27T15:22:52.897 INFO:journalctl@ceph.mds.k.smithi029.stdout:Nov 27 15:22:52 smithi029 podman[155433]: 2023-11-27T15:22:53.208 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:52 smithi029 ceph-mon[131166]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:53.208 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:52 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/459500582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:53.209 INFO:journalctl@ceph.mds.k.smithi029.stdout:Nov 27 15:22:53 smithi029 bash[155433]: 5207d321fe1c892436dc3ecfbc930fe810a5b0e7fd50c9e1ea65063ca31ba3c5 2023-11-27T15:22:53.209 INFO:journalctl@ceph.mds.k.smithi029.stdout:Nov 27 15:22:53 smithi029 ceph-mds[155473]: -- 172.21.15.29:0/1879778236 <== mon.2 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562175acd520 con 0x562175aa2c00 2023-11-27T15:22:53.209 INFO:journalctl@ceph.mds.k.smithi029.stdout:Nov 27 15:22:53 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-k[155449]: starting mds.k at 2023-11-27T15:22:53.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:52 smithi060 ceph-mon[131518]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:53.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:52 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/459500582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:53.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:52 smithi008 ceph-mon[128788]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:53.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:52 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/459500582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:53.647 INFO:journalctl@ceph.mds.k.smithi029.stdout:Nov 27 15:22:53 smithi029 systemd[1]: Started Ceph mds.k for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:22:54.515 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:54.516 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:54.516 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:54.516 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:54.516 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:54.516 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:54.516 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:54 smithi008 ceph-mon[128788]: Deploying daemon mds.j on smithi008 2023-11-27T15:22:54.516 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:54 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.29:6832/192235656,v1:172.21.15.29:6833/192235656] up:boot 2023-11-27T15:22:54.516 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:54 smithi008 ceph-mon[128788]: fsmap 2 up:standby 2023-11-27T15:22:54.516 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:54 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:22:54.522 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:54.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:54.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:54.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:54.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:54 smithi060 ceph-mon[131518]: Deploying daemon mds.j on smithi008 2023-11-27T15:22:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:54 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.29:6832/192235656,v1:172.21.15.29:6833/192235656] up:boot 2023-11-27T15:22:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:54 smithi060 ceph-mon[131518]: fsmap 2 up:standby 2023-11-27T15:22:54.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:54 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:22:54.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:54.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:54.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:54.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:54.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:54.650 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:54.650 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:54 smithi029 ceph-mon[131166]: Deploying daemon mds.j on smithi008 2023-11-27T15:22:54.650 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:54 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.29:6832/192235656,v1:172.21.15.29:6833/192235656] up:boot 2023-11-27T15:22:54.650 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:54 smithi029 ceph-mon[131166]: fsmap 2 up:standby 2023-11-27T15:22:54.650 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:54 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:22:54.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.735+0000 7fd354fff700 1 -- 172.21.15.8:0/3843999422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd350101e50 msgr2=0x7fd350108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:54.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.735+0000 7fd354fff700 1 --2- 172.21.15.8:0/3843999422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd350101e50 0x7fd350108710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd34800b0a0 tx=0x7fd34800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:54.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.735+0000 7fd354fff700 1 -- 172.21.15.8:0/3843999422 shutdown_connections 2023-11-27T15:22:54.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.735+0000 7fd354fff700 1 --2- 172.21.15.8:0/3843999422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd3501002d0 0x7fd3501006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:54.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.735+0000 7fd354fff700 1 --2- 172.21.15.8:0/3843999422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd350101e50 0x7fd350108710 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:54.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.735+0000 7fd354fff700 1 --2- 172.21.15.8:0/3843999422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3501014c0 0x7fd350101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:54.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.735+0000 7fd354fff700 1 -- 172.21.15.8:0/3843999422 >> 172.21.15.8:0/3843999422 conn(0x7fd3500fb8b0 msgr2=0x7fd3500fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:54.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.735+0000 7fd354fff700 1 -- 172.21.15.8:0/3843999422 shutdown_connections 2023-11-27T15:22:54.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.735+0000 7fd354fff700 1 -- 172.21.15.8:0/3843999422 wait complete. 2023-11-27T15:22:54.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.736+0000 7fd354fff700 1 Processor -- start 2023-11-27T15:22:54.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.736+0000 7fd354fff700 1 -- start start 2023-11-27T15:22:54.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd354fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3501002d0 0x7fd3500748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd354fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3501014c0 0x7fd350072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd354fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd350101e50 0x7fd350073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd354fff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd350073bc0 con 0x7fd3501014c0 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd354fff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd350073d30 con 0x7fd3501002d0 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd354fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3501a01b0 con 0x7fd350101e50 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd347fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3501014c0 0x7fd350072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd34e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3501002d0 0x7fd3500748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd347fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3501014c0 0x7fd350072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54112/0 (socket says 172.21.15.8:54112) 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd34e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3501002d0 0x7fd3500748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:55206/0 (socket says 172.21.15.8:55206) 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd347fff700 1 -- 172.21.15.8:0/3029621322 learned_addr learned my addr 172.21.15.8:0/3029621322 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd34ed9d700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd350101e50 0x7fd350073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd34e59c700 1 -- 172.21.15.8:0/3029621322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd350101e50 msgr2=0x7fd350073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:54.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd34e59c700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd350101e50 0x7fd350073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:54.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd34e59c700 1 -- 172.21.15.8:0/3029621322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3501014c0 msgr2=0x7fd350072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:54.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd34e59c700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3501014c0 0x7fd350072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:54.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.737+0000 7fd34e59c700 1 -- 172.21.15.8:0/3029621322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348009d00 con 0x7fd3501002d0 2023-11-27T15:22:54.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.738+0000 7fd347fff700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3501014c0 0x7fd350072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:54.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.738+0000 7fd34e59c700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3501002d0 0x7fd3500748a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd34000e9c0 tx=0x7fd34000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:54.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.738+0000 7fd3477fe700 1 -- 172.21.15.8:0/3029621322 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd340004220 con 0x7fd3501002d0 2023-11-27T15:22:54.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.738+0000 7fd354fff700 1 -- 172.21.15.8:0/3029621322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35010bca0 con 0x7fd3501002d0 2023-11-27T15:22:54.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.738+0000 7fd3477fe700 1 -- 172.21.15.8:0/3029621322 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd340004380 con 0x7fd3501002d0 2023-11-27T15:22:54.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.738+0000 7fd3477fe700 1 -- 172.21.15.8:0/3029621322 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd340003be0 con 0x7fd3501002d0 2023-11-27T15:22:54.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.738+0000 7fd354fff700 1 -- 172.21.15.8:0/3029621322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35010bb10 con 0x7fd3501002d0 2023-11-27T15:22:54.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.739+0000 7fd354fff700 1 -- 172.21.15.8:0/3029621322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3501077d0 con 0x7fd3501002d0 2023-11-27T15:22:54.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.741+0000 7fd3477fe700 1 -- 172.21.15.8:0/3029621322 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd340003d40 con 0x7fd3501002d0 2023-11-27T15:22:54.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.742+0000 7fd3477fe700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd33c0a1150 0x7fd33c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:54.746 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.743+0000 7fd3477fe700 1 -- 172.21.15.8:0/3029621322 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd340014070 con 0x7fd3501002d0 2023-11-27T15:22:54.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.743+0000 7fd347fff700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd33c0a1150 0x7fd33c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:54.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.743+0000 7fd347fff700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd33c0a1150 0x7fd33c0a3600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd338011550 tx=0x7fd338009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:54.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:54.745+0000 7fd3477fe700 1 -- 172.21.15.8:0/3029621322 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd340096490 con 0x7fd3501002d0 2023-11-27T15:22:55.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.040+0000 7fd354fff700 1 -- 172.21.15.8:0/3029621322 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd350061700 con 0x7fd3501002d0 2023-11-27T15:22:55.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.041+0000 7fd3477fe700 1 -- 172.21.15.8:0/3029621322 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2011 (secure 0 0 0) 0x7fd34001f340 con 0x7fd3501002d0 2023-11-27T15:22:55.044 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:22:55.045 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-27T15:22:55.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 -- 172.21.15.8:0/3029621322 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd33c0a1150 msgr2=0x7fd33c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd33c0a1150 0x7fd33c0a3600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd338011550 tx=0x7fd338009260 comp rx=0 tx=0).stop 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 -- 172.21.15.8:0/3029621322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3501002d0 msgr2=0x7fd3500748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3501002d0 0x7fd3500748a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd34000e9c0 tx=0x7fd34000ecd0 comp rx=0 tx=0).stop 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 -- 172.21.15.8:0/3029621322 shutdown_connections 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd33c0a1150 0x7fd33c0a3600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd350101e50 0x7fd350073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd3501002d0 0x7fd3500748a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 --2- 172.21.15.8:0/3029621322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3501014c0 0x7fd350072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 -- 172.21.15.8:0/3029621322 >> 172.21.15.8:0/3029621322 conn(0x7fd3500fb8b0 msgr2=0x7fd350104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 -- 172.21.15.8:0/3029621322 shutdown_connections 2023-11-27T15:22:55.050 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:55.043+0000 7fd354fff700 1 -- 172.21.15.8:0/3029621322 wait complete. 2023-11-27T15:22:55.051 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 3 2023-11-27T15:22:55.559 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:55 smithi008 ceph-mon[128788]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:55.560 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:55 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3029621322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:55.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:55 smithi060 ceph-mon[131518]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:55.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:55 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3029621322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:55.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:55 smithi029 ceph-mon[131166]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:55.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:55 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3029621322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:55.863 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:22:55 smithi008 systemd[1]: Starting Ceph mds.j for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:22:56.355 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:22:56 smithi008 podman[161829]: 2023-11-27T15:22:56.355 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:22:56 smithi008 bash[161829]: 6e2cca2824034b956a32f591cd39acff7bdf2585b6f3c72affdebd21c4131154 2023-11-27T15:22:56.355 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:22:56 smithi008 ceph-mds[161869]: -- 172.21.15.8:0/2299887490 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55818523b520 con 0x558185212800 2023-11-27T15:22:56.355 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:22:56 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-j[161845]: starting mds.j at 2023-11-27T15:22:56.613 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:22:56 smithi008 systemd[1]: Started Ceph mds.j for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:22:57.067 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:57.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.284+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/3765994552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b54107f30 msgr2=0x7f0b5410a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.284+0000 7f0b5ab9d700 1 --2- 172.21.15.8:0/3765994552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b54107f30 0x7f0b5410a350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00b0a0 tx=0x7f0b4c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.284+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/3765994552 shutdown_connections 2023-11-27T15:22:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.284+0000 7f0b5ab9d700 1 --2- 172.21.15.8:0/3765994552 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b54074a90 0x7f0b54072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.284+0000 7f0b5ab9d700 1 --2- 172.21.15.8:0/3765994552 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b54107f30 0x7f0b5410a350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.284+0000 7f0b5ab9d700 1 --2- 172.21.15.8:0/3765994552 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b540734c0 0x7f0b54073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.285+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/3765994552 >> 172.21.15.8:0/3765994552 conn(0x7f0b540fb6b0 msgr2=0x7f0b540fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.285+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/3765994552 shutdown_connections 2023-11-27T15:22:57.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.285+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/3765994552 wait complete. 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.286+0000 7f0b5ab9d700 1 Processor -- start 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.286+0000 7f0b5ab9d700 1 -- start start 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.286+0000 7f0b5ab9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b540734c0 0x7f0b54198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.286+0000 7f0b5ab9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b54074a90 0x7f0b541985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.286+0000 7f0b5ab9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b54107f30 0x7f0b5419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.286+0000 7f0b5ab9d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5419fbe0 con 0x7f0b540734c0 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.286+0000 7f0b5ab9d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5419fd20 con 0x7f0b54107f30 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.286+0000 7f0b5ab9d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5419feb0 con 0x7f0b54074a90 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.286+0000 7f0b59b9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b540734c0 0x7f0b54198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.286+0000 7f0b59b9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b540734c0 0x7f0b54198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54156/0 (socket says 172.21.15.8:54156) 2023-11-27T15:22:57.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.287+0000 7f0b59b9b700 1 -- 172.21.15.8:0/1678120923 learned_addr learned my addr 172.21.15.8:0/1678120923 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.287+0000 7f0b5939a700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b54074a90 0x7f0b541985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.287+0000 7f0b5a39c700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b54107f30 0x7f0b5419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.287+0000 7f0b59b9b700 1 -- 172.21.15.8:0/1678120923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b54074a90 msgr2=0x7f0b541985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.287+0000 7f0b59b9b700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b54074a90 0x7f0b541985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.287+0000 7f0b59b9b700 1 -- 172.21.15.8:0/1678120923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b54107f30 msgr2=0x7f0b5419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.287+0000 7f0b59b9b700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b54107f30 0x7f0b5419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.287+0000 7f0b59b9b700 1 -- 172.21.15.8:0/1678120923 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c009d00 con 0x7f0b540734c0 2023-11-27T15:22:57.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.287+0000 7f0b59b9b700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b540734c0 0x7f0b54198060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000e970 tx=0x7f0b5000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.287+0000 7f0b4affd700 1 -- 172.21.15.8:0/1678120923 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50009790 con 0x7f0b540734c0 2023-11-27T15:22:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.288+0000 7f0b4affd700 1 -- 172.21.15.8:0/1678120923 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b5000c7e0 con 0x7f0b540734c0 2023-11-27T15:22:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.288+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/1678120923 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5410dc70 con 0x7f0b540734c0 2023-11-27T15:22:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.288+0000 7f0b4affd700 1 -- 172.21.15.8:0/1678120923 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50003c70 con 0x7f0b540734c0 2023-11-27T15:22:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.288+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/1678120923 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5410dae0 con 0x7f0b540734c0 2023-11-27T15:22:57.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.289+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/1678120923 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b541066c0 con 0x7f0b540734c0 2023-11-27T15:22:57.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.292+0000 7f0b4affd700 1 -- 172.21.15.8:0/1678120923 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0b5000cc50 con 0x7f0b540734c0 2023-11-27T15:22:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.294+0000 7f0b4affd700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0b400a1150 0x7f0b400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:57.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.294+0000 7f0b4affd700 1 -- 172.21.15.8:0/1678120923 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0b50014070 con 0x7f0b540734c0 2023-11-27T15:22:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.294+0000 7f0b5939a700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0b400a1150 0x7f0b400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:57.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.295+0000 7f0b5939a700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0b400a1150 0x7f0b400a3600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400ac50 tx=0x7f0b4400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:57.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.297+0000 7f0b4affd700 1 -- 172.21.15.8:0/1678120923 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0b50096370 con 0x7f0b540734c0 2023-11-27T15:22:57.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.557+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/1678120923 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b5402cfe0 con 0x7f0b540734c0 2023-11-27T15:22:57.562 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.558+0000 7f0b4affd700 1 -- 172.21.15.8:0/1678120923 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2765 (secure 0 0 0) 0x7f0b50099990 con 0x7f0b540734c0 2023-11-27T15:22:57.562 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:22:57.562 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[]} 2023-11-27T15:22:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/1678120923 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0b400a1150 msgr2=0x7f0b400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0b400a1150 0x7f0b400a3600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400ac50 tx=0x7f0b4400a380 comp rx=0 tx=0).stop 2023-11-27T15:22:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/1678120923 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b540734c0 msgr2=0x7f0b54198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b540734c0 0x7f0b54198060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0b5000e970 tx=0x7f0b5000ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/1678120923 shutdown_connections 2023-11-27T15:22:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0b54107f30 0x7f0b5419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b540734c0 0x7f0b54198060 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0b400a1150 0x7f0b400a3600 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 --2- 172.21.15.8:0/1678120923 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0b54074a90 0x7f0b541985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/1678120923 >> 172.21.15.8:0/1678120923 conn(0x7f0b540fb6b0 msgr2=0x7f0b540fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/1678120923 shutdown_connections 2023-11-27T15:22:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:57.561+0000 7f0b5ab9d700 1 -- 172.21.15.8:0/1678120923 wait complete. 2023-11-27T15:22:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 4 2023-11-27T15:22:57.583 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:57.583 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.8:6834/2969037056,v1:172.21.15.8:6835/2969037056] up:boot 2023-11-27T15:22:57.583 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: fsmap 3 up:standby 2023-11-27T15:22:57.583 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:22:57.583 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:57.583 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:57.584 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:57.584 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:57.584 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:57.584 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:57.584 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:57 smithi008 ceph-mon[128788]: Deploying daemon mds.d on smithi008 2023-11-27T15:22:57.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:57.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.8:6834/2969037056,v1:172.21.15.8:6835/2969037056] up:boot 2023-11-27T15:22:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: fsmap 3 up:standby 2023-11-27T15:22:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:22:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:57.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:57 smithi029 ceph-mon[131166]: Deploying daemon mds.d on smithi008 2023-11-27T15:22:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.8:6834/2969037056,v1:172.21.15.8:6835/2969037056] up:boot 2023-11-27T15:22:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: fsmap 3 up:standby 2023-11-27T15:22:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:22:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:22:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:22:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:22:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:57 smithi060 ceph-mon[131518]: Deploying daemon mds.d on smithi008 2023-11-27T15:22:58.585 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:58 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1678120923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:58.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:58 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1678120923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:58.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:58 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1678120923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:59.113 INFO:journalctl@ceph.mds.d.smithi008.stdout:Nov 27 15:22:58 smithi008 systemd[1]: Starting Ceph mds.d for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:22:59.586 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:22:59 smithi008 ceph-mon[128788]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:59.613 INFO:journalctl@ceph.mds.d.smithi008.stdout:Nov 27 15:22:59 smithi008 podman[162218]: 2023-11-27T15:22:59.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:22:59 smithi029 ceph-mon[131166]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:22:59.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.799+0000 7f38cfbff700 1 -- 172.21.15.8:0/151038189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38c8100d40 msgr2=0x7f38c8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:59.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.799+0000 7f38cfbff700 1 --2- 172.21.15.8:0/151038189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38c8100d40 0x7f38c8103120 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009990 tx=0x7f38c4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:59.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.800+0000 7f38cfbff700 1 -- 172.21.15.8:0/151038189 shutdown_connections 2023-11-27T15:22:59.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.800+0000 7f38cfbff700 1 --2- 172.21.15.8:0/151038189 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c8105f80 0x7f38c81083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:59.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.800+0000 7f38cfbff700 1 --2- 172.21.15.8:0/151038189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38c8100d40 0x7f38c8103120 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:59.804 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.800+0000 7f38cfbff700 1 --2- 172.21.15.8:0/151038189 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38c8103660 0x7f38c8105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:59.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.800+0000 7f38cfbff700 1 -- 172.21.15.8:0/151038189 >> 172.21.15.8:0/151038189 conn(0x7f38c80fa7c0 msgr2=0x7f38c80fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:59.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.800+0000 7f38cfbff700 1 -- 172.21.15.8:0/151038189 shutdown_connections 2023-11-27T15:22:59.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.800+0000 7f38cfbff700 1 -- 172.21.15.8:0/151038189 wait complete. 2023-11-27T15:22:59.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.801+0000 7f38cfbff700 1 Processor -- start 2023-11-27T15:22:59.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.801+0000 7f38cfbff700 1 -- start start 2023-11-27T15:22:59.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.801+0000 7f38cfbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c8100d40 0x7f38c81006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:59.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.801+0000 7f38cfbff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38c8103660 0x7f38c80fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:59.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.801+0000 7f38cfbff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38c8105f80 0x7f38c80ff230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.801+0000 7f38cfbff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80ff9d0 con 0x7f38c8103660 2023-11-27T15:22:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.801+0000 7f38cfbff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80ffb40 con 0x7f38c8105f80 2023-11-27T15:22:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.801+0000 7f38cfbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c81a0110 con 0x7f38c8100d40 2023-11-27T15:22:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38cd19a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38c8103660 0x7f38c80fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38cd19a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38c8103660 0x7f38c80fecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54180/0 (socket says 172.21.15.8:54180) 2023-11-27T15:22:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38cd99b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c8100d40 0x7f38c81006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38cd19a700 1 -- 172.21.15.8:0/1899717334 learned_addr learned my addr 172.21.15.8:0/1899717334 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:22:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38ce19c700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38c8105f80 0x7f38c80ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38cd19a700 1 -- 172.21.15.8:0/1899717334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c8100d40 msgr2=0x7f38c81006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38cd19a700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c8100d40 0x7f38c81006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38cd19a700 1 -- 172.21.15.8:0/1899717334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38c8105f80 msgr2=0x7f38c80ff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38cd19a700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38c8105f80 0x7f38c80ff230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38cd19a700 1 -- 172.21.15.8:0/1899717334 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c40095f0 con 0x7f38c8103660 2023-11-27T15:22:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.802+0000 7f38cd19a700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38c8103660 0x7f38c80fecf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f38b800e9a0 tx=0x7f38b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.803+0000 7f38beffd700 1 -- 172.21.15.8:0/1899717334 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8009790 con 0x7f38c8103660 2023-11-27T15:22:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.803+0000 7f38cfbff700 1 -- 172.21.15.8:0/1899717334 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c810bc70 con 0x7f38c8103660 2023-11-27T15:22:59.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.803+0000 7f38beffd700 1 -- 172.21.15.8:0/1899717334 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38b8010430 con 0x7f38c8103660 2023-11-27T15:22:59.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.803+0000 7f38beffd700 1 -- 172.21.15.8:0/1899717334 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8018680 con 0x7f38c8103660 2023-11-27T15:22:59.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.803+0000 7f38cfbff700 1 -- 172.21.15.8:0/1899717334 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c810bae0 con 0x7f38c8103660 2023-11-27T15:22:59.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.804+0000 7f38cfbff700 1 -- 172.21.15.8:0/1899717334 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c80fc330 con 0x7f38c8103660 2023-11-27T15:22:59.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.805+0000 7f38beffd700 1 -- 172.21.15.8:0/1899717334 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f38b80105a0 con 0x7f38c8103660 2023-11-27T15:22:59.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.807+0000 7f38beffd700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f38b40a1150 0x7f38b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:59.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.807+0000 7f38beffd700 1 -- 172.21.15.8:0/1899717334 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f38b8014070 con 0x7f38c8103660 2023-11-27T15:22:59.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.807+0000 7f38cd99b700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f38b40a1150 0x7f38b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:59.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.808+0000 7f38cd99b700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f38b40a1150 0x7f38b40a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f38c4000c00 tx=0x7f38c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:59.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:22:59.811+0000 7f38beffd700 1 -- 172.21.15.8:0/1899717334 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f38b8051730 con 0x7f38c8103660 2023-11-27T15:22:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:22:59 smithi060 ceph-mon[131518]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:00.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.068+0000 7f38cfbff700 1 -- 172.21.15.8:0/1899717334 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38c802cfe0 con 0x7f38c8103660 2023-11-27T15:23:00.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.068+0000 7f38beffd700 1 -- 172.21.15.8:0/1899717334 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2765 (secure 0 0 0) 0x7f38b8096220 con 0x7f38c8103660 2023-11-27T15:23:00.072 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:00.072 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[]} 2023-11-27T15:23:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.070+0000 7f38cfbff700 1 -- 172.21.15.8:0/1899717334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f38b40a1150 msgr2=0x7f38b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:00.074 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.070+0000 7f38cfbff700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f38b40a1150 0x7f38b40a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f38c4000c00 tx=0x7f38c4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.070+0000 7f38cfbff700 1 -- 172.21.15.8:0/1899717334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38c8103660 msgr2=0x7f38c80fecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.070+0000 7f38cfbff700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38c8103660 0x7f38c80fecf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f38b800e9a0 tx=0x7f38b800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:23:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.070+0000 7f38cfbff700 1 -- 172.21.15.8:0/1899717334 shutdown_connections 2023-11-27T15:23:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.071+0000 7f38cfbff700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f38b40a1150 0x7f38b40a3600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.071+0000 7f38cfbff700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f38c8100d40 0x7f38c81006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.071+0000 7f38cfbff700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f38c8105f80 0x7f38c80ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.071+0000 7f38cfbff700 1 --2- 172.21.15.8:0/1899717334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f38c8103660 0x7f38c80fecf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.071+0000 7f38cfbff700 1 -- 172.21.15.8:0/1899717334 >> 172.21.15.8:0/1899717334 conn(0x7f38c80fa7c0 msgr2=0x7f38c81019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:00.075 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.071+0000 7f38cfbff700 1 -- 172.21.15.8:0/1899717334 shutdown_connections 2023-11-27T15:23:00.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:00.071+0000 7f38cfbff700 1 -- 172.21.15.8:0/1899717334 wait complete. 2023-11-27T15:23:00.076 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 4 2023-11-27T15:23:00.091 INFO:journalctl@ceph.mds.d.smithi008.stdout:Nov 27 15:22:59 smithi008 ceph-mds[162263]: -- 172.21.15.8:0/91069894 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f82746d520 con 0x55f827444800 2023-11-27T15:23:00.091 INFO:journalctl@ceph.mds.d.smithi008.stdout:Nov 27 15:22:59 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-d[162234]: starting mds.d at 2023-11-27T15:23:00.091 INFO:journalctl@ceph.mds.d.smithi008.stdout:Nov 27 15:22:59 smithi008 bash[162218]: db1de43e36153c23d6f94bb1cef2eba313ba73c864c3790b04ac590442f0717d 2023-11-27T15:23:00.092 INFO:journalctl@ceph.mds.d.smithi008.stdout:Nov 27 15:22:59 smithi008 systemd[1]: Started Ceph mds.d for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:23:00.886 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:00.886 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:00.886 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:00.886 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:00.886 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:00.887 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:00.887 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: Deploying daemon mds.e on smithi029 2023-11-27T15:23:00.887 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1899717334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:00.887 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:00.887 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.8:6836/2810841935,v1:172.21.15.8:6837/2810841935] up:boot 2023-11-27T15:23:00.888 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: fsmap 4 up:standby 2023-11-27T15:23:00.888 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:00 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:23:01.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: Deploying daemon mds.e on smithi029 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1899717334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.8:6836/2810841935,v1:172.21.15.8:6837/2810841935] up:boot 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: fsmap 4 up:standby 2023-11-27T15:23:01.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:00 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:23:01.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: Deploying daemon mds.e on smithi029 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1899717334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.8:6836/2810841935,v1:172.21.15.8:6837/2810841935] up:boot 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: fsmap 4 up:standby 2023-11-27T15:23:01.114 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:00 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:23:01.772 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:23:01 smithi029 systemd[1]: Starting Ceph mds.e for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:23:02.034 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:23:01 smithi029 podman[155781]: 2023-11-27T15:23:02.094 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.310+0000 7f55fbbff700 1 -- 172.21.15.8:0/3730118531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f41002d0 msgr2=0x7f55f41006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.310+0000 7f55fbbff700 1 --2- 172.21.15.8:0/3730118531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f41002d0 0x7f55f41006e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009990 tx=0x7f55f0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.310+0000 7f55fbbff700 1 -- 172.21.15.8:0/3730118531 shutdown_connections 2023-11-27T15:23:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.310+0000 7f55fbbff700 1 --2- 172.21.15.8:0/3730118531 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f4101e50 0x7f55f4108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.310+0000 7f55fbbff700 1 --2- 172.21.15.8:0/3730118531 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f41002d0 0x7f55f41006e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.310+0000 7f55fbbff700 1 --2- 172.21.15.8:0/3730118531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55f41014c0 0x7f55f4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.310+0000 7f55fbbff700 1 -- 172.21.15.8:0/3730118531 >> 172.21.15.8:0/3730118531 conn(0x7f55f40fb8b0 msgr2=0x7f55f40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.310+0000 7f55fbbff700 1 -- 172.21.15.8:0/3730118531 shutdown_connections 2023-11-27T15:23:02.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.310+0000 7f55fbbff700 1 -- 172.21.15.8:0/3730118531 wait complete. 2023-11-27T15:23:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.311+0000 7f55fbbff700 1 Processor -- start 2023-11-27T15:23:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.311+0000 7f55fbbff700 1 -- start start 2023-11-27T15:23:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.311+0000 7f55fbbff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f41002d0 0x7f55f41960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.311+0000 7f55fbbff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55f41014c0 0x7f55f41965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.311+0000 7f55fbbff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f4101e50 0x7f55f419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.311+0000 7f55fbbff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f419dc20 con 0x7f55f41014c0 2023-11-27T15:23:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.311+0000 7f55fbbff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f419dd60 con 0x7f55f41002d0 2023-11-27T15:23:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55fbbff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f419def0 con 0x7f55f4101e50 2023-11-27T15:23:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55f999b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f41002d0 0x7f55f41960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:02.317 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55f999b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f41002d0 0x7f55f41960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50160/0 (socket says 172.21.15.8:50160) 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55f919a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55f41014c0 0x7f55f41965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55f999b700 1 -- 172.21.15.8:0/1998681056 learned_addr learned my addr 172.21.15.8:0/1998681056 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55fa19c700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f4101e50 0x7f55f419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55f999b700 1 -- 172.21.15.8:0/1998681056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f4101e50 msgr2=0x7f55f419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55f999b700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f4101e50 0x7f55f419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55f999b700 1 -- 172.21.15.8:0/1998681056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55f41014c0 msgr2=0x7f55f41965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55f999b700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55f41014c0 0x7f55f41965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55f999b700 1 -- 172.21.15.8:0/1998681056 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00095f0 con 0x7f55f41002d0 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55f999b700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f41002d0 0x7f55f41960a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f55f0004730 tx=0x7f55f0004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.312+0000 7f55eaffd700 1 -- 172.21.15.8:0/1998681056 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f001d070 con 0x7f55f41002d0 2023-11-27T15:23:02.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.313+0000 7f55eaffd700 1 -- 172.21.15.8:0/1998681056 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55f0021410 con 0x7f55f41002d0 2023-11-27T15:23:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.313+0000 7f55eaffd700 1 -- 172.21.15.8:0/1998681056 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0017650 con 0x7f55f41002d0 2023-11-27T15:23:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.313+0000 7f55fbbff700 1 -- 172.21.15.8:0/1998681056 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f410bca0 con 0x7f55f41002d0 2023-11-27T15:23:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.313+0000 7f55fbbff700 1 -- 172.21.15.8:0/1998681056 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f410bb10 con 0x7f55f41002d0 2023-11-27T15:23:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.314+0000 7f55fbbff700 1 -- 172.21.15.8:0/1998681056 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f4107860 con 0x7f55f41002d0 2023-11-27T15:23:02.319 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.315+0000 7f55eaffd700 1 -- 172.21.15.8:0/1998681056 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f55f0021580 con 0x7f55f41002d0 2023-11-27T15:23:02.320 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.317+0000 7f55eaffd700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f55e00a1150 0x7f55e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.317+0000 7f55eaffd700 1 -- 172.21.15.8:0/1998681056 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f55f00cd8f0 con 0x7f55f41002d0 2023-11-27T15:23:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.317+0000 7f55f919a700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f55e00a1150 0x7f55e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:02.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.318+0000 7f55f919a700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f55e00a1150 0x7f55e00a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009c60 tx=0x7f55e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:02.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.320+0000 7f55eaffd700 1 -- 172.21.15.8:0/1998681056 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f55f00974d0 con 0x7f55f41002d0 2023-11-27T15:23:02.397 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:23:02 smithi029 bash[155781]: 519815cda3b45884cdd9d2e3b1b3723b8a265e678edbfe843e34e6991799a560 2023-11-27T15:23:02.397 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:23:02 smithi029 ceph-mds[155822]: -- 172.21.15.29:0/3941937211 <== mon.2 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cad1065520 con 0x55cad103ac00 2023-11-27T15:23:02.397 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:23:02 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-e[155797]: starting mds.e at 2023-11-27T15:23:02.398 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:23:02 smithi029 systemd[1]: Started Ceph mds.e for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:23:02.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.572+0000 7f55fbbff700 1 -- 172.21.15.8:0/1998681056 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55f402cfe0 con 0x7f55f41002d0 2023-11-27T15:23:02.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.573+0000 7f55eaffd700 1 -- 172.21.15.8:0/1998681056 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4273 (secure 0 0 0) 0x7f55f009aaf0 con 0x7f55f41002d0 2023-11-27T15:23:02.577 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:02.578 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-11-27T15:23:02.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 -- 172.21.15.8:0/1998681056 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f55e00a1150 msgr2=0x7f55e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:02.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f55e00a1150 0x7f55e00a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009c60 tx=0x7f55e4009260 comp rx=0 tx=0).stop 2023-11-27T15:23:02.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 -- 172.21.15.8:0/1998681056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f41002d0 msgr2=0x7f55f41960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:02.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f41002d0 0x7f55f41960a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f55f0004730 tx=0x7f55f0004760 comp rx=0 tx=0).stop 2023-11-27T15:23:02.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 -- 172.21.15.8:0/1998681056 shutdown_connections 2023-11-27T15:23:02.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f55e00a1150 0x7f55e00a3600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:02.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f55f4101e50 0x7f55f419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:02.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f55f41002d0 0x7f55f41960a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:02.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 --2- 172.21.15.8:0/1998681056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f55f41014c0 0x7f55f41965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:02.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 -- 172.21.15.8:0/1998681056 >> 172.21.15.8:0/1998681056 conn(0x7f55f40fb8b0 msgr2=0x7f55f41049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:02.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 -- 172.21.15.8:0/1998681056 shutdown_connections 2023-11-27T15:23:02.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:02.576+0000 7f55fbbff700 1 -- 172.21.15.8:0/1998681056 wait complete. 2023-11-27T15:23:02.581 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 6 2023-11-27T15:23:03.149 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.29:6834/789896869,v1:172.21.15.29:6835/789896869] up:boot 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: fsmap 5 up:standby 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: Deploying daemon mds.f on smithi060 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:03.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:03 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1998681056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:03.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.29:6834/789896869,v1:172.21.15.29:6835/789896869] up:boot 2023-11-27T15:23:03.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: fsmap 5 up:standby 2023-11-27T15:23:03.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:23:03.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:03.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:03.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:03.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:03.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:03.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:03.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: Deploying daemon mds.f on smithi060 2023-11-27T15:23:03.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:03.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:03 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1998681056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:03.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.29:6834/789896869,v1:172.21.15.29:6835/789896869] up:boot 2023-11-27T15:23:03.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: fsmap 5 up:standby 2023-11-27T15:23:03.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:23:03.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:03.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:03.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:03.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:03.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:03.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:03.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: Deploying daemon mds.f on smithi060 2023-11-27T15:23:03.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:03.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:03 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1998681056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:03.825 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:23:03 smithi060 systemd[1]: Starting Ceph mds.f for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:23:04.194 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:23:03 smithi060 podman[157095]: 2023-11-27T15:23:04.194 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:23:04 smithi060 bash[157095]: 5cc9e2fdeb8947adcf4a78666b9835fd972611752dab1815321704c518d5764b 2023-11-27T15:23:04.194 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:23:04 smithi060 ceph-mds[157135]: -- 172.21.15.60:0/1027712041 <== mon.1 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5639370a7520 con 0x56393707d000 2023-11-27T15:23:04.194 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:23:04 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-f[157111]: starting mds.f at 2023-11-27T15:23:04.574 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:23:04 smithi060 systemd[1]: Started Ceph mds.f for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:23:04.600 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:04.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.811+0000 7f26d63ff700 1 -- 172.21.15.8:0/1050116399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d01002d0 msgr2=0x7f26d01006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:04.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.811+0000 7f26d63ff700 1 --2- 172.21.15.8:0/1050116399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d01002d0 0x7f26d01006e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009990 tx=0x7f26c0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:04.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.811+0000 7f26d63ff700 1 -- 172.21.15.8:0/1050116399 shutdown_connections 2023-11-27T15:23:04.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.811+0000 7f26d63ff700 1 --2- 172.21.15.8:0/1050116399 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d0101e50 0x7f26d0108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:04.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.811+0000 7f26d63ff700 1 --2- 172.21.15.8:0/1050116399 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d01002d0 0x7f26d01006e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:04.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.811+0000 7f26d63ff700 1 --2- 172.21.15.8:0/1050116399 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26d01014c0 0x7f26d0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:04.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.811+0000 7f26d63ff700 1 -- 172.21.15.8:0/1050116399 >> 172.21.15.8:0/1050116399 conn(0x7f26d00fb8b0 msgr2=0x7f26d00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:04.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.811+0000 7f26d63ff700 1 -- 172.21.15.8:0/1050116399 shutdown_connections 2023-11-27T15:23:04.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.811+0000 7f26d63ff700 1 -- 172.21.15.8:0/1050116399 wait complete. 2023-11-27T15:23:04.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.812+0000 7f26d63ff700 1 Processor -- start 2023-11-27T15:23:04.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26d63ff700 1 -- start start 2023-11-27T15:23:04.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26d63ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26d01002d0 0x7f26d01960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:04.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26d63ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01014c0 0x7f26d01965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:04.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26d63ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d0101e50 0x7f26d019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:04.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26d63ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d019dc20 con 0x7f26d01002d0 2023-11-27T15:23:04.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26d63ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d019dd60 con 0x7f26d0101e50 2023-11-27T15:23:04.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26d63ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d019def0 con 0x7f26d01014c0 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26cffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26d01002d0 0x7f26d01960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26cffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26d01002d0 0x7f26d01960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34666/0 (socket says 172.21.15.8:34666) 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26cffff700 1 -- 172.21.15.8:0/399032270 learned_addr learned my addr 172.21.15.8:0/399032270 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26cf7fe700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01014c0 0x7f26d01965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.813+0000 7f26d499c700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d0101e50 0x7f26d019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26cffff700 1 -- 172.21.15.8:0/399032270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01014c0 msgr2=0x7f26d01965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26cffff700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01014c0 0x7f26d01965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26cffff700 1 -- 172.21.15.8:0/399032270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d0101e50 msgr2=0x7f26d019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26cffff700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d0101e50 0x7f26d019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26cffff700 1 -- 172.21.15.8:0/399032270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c00095f0 con 0x7f26d01002d0 2023-11-27T15:23:04.819 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26cffff700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26d01002d0 0x7f26d01960a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005740 tx=0x7f26c0005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:04.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26cd7fa700 1 -- 172.21.15.8:0/399032270 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c001d070 con 0x7f26d01002d0 2023-11-27T15:23:04.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26cd7fa700 1 -- 172.21.15.8:0/399032270 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26c0021410 con 0x7f26d01002d0 2023-11-27T15:23:04.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26cd7fa700 1 -- 172.21.15.8:0/399032270 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0017650 con 0x7f26d01002d0 2023-11-27T15:23:04.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26d63ff700 1 -- 172.21.15.8:0/399032270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d010bca0 con 0x7f26d01002d0 2023-11-27T15:23:04.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.814+0000 7f26d63ff700 1 -- 172.21.15.8:0/399032270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d010bb10 con 0x7f26d01002d0 2023-11-27T15:23:04.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.815+0000 7f26d63ff700 1 -- 172.21.15.8:0/399032270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d0107860 con 0x7f26d01002d0 2023-11-27T15:23:04.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.817+0000 7f26cd7fa700 1 -- 172.21.15.8:0/399032270 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f26c0021580 con 0x7f26d01002d0 2023-11-27T15:23:04.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.820+0000 7f26cd7fa700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26b80a1150 0x7f26b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:04.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.820+0000 7f26cd7fa700 1 -- 172.21.15.8:0/399032270 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f26c00cdb20 con 0x7f26d01002d0 2023-11-27T15:23:04.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.820+0000 7f26cf7fe700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26b80a1150 0x7f26b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:04.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.821+0000 7f26cf7fe700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26b80a1150 0x7f26b80a3600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009c60 tx=0x7f26c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:04.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:04.823+0000 7f26cd7fa700 1 -- 172.21.15.8:0/399032270 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f26c0097700 con 0x7f26d01002d0 2023-11-27T15:23:05.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.078+0000 7f26d63ff700 1 -- 172.21.15.8:0/399032270 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26d002cfe0 con 0x7f26d01002d0 2023-11-27T15:23:05.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.079+0000 7f26cd7fa700 1 -- 172.21.15.8:0/399032270 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5027 (secure 0 0 0) 0x7f26c009ad20 con 0x7f26d01002d0 2023-11-27T15:23:05.083 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:05.083 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-11-27T15:23:05.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.081+0000 7f26d63ff700 1 -- 172.21.15.8:0/399032270 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26b80a1150 msgr2=0x7f26b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:05.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.081+0000 7f26d63ff700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26b80a1150 0x7f26b80a3600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009c60 tx=0x7f26c4009260 comp rx=0 tx=0).stop 2023-11-27T15:23:05.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.081+0000 7f26d63ff700 1 -- 172.21.15.8:0/399032270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26d01002d0 msgr2=0x7f26d01960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:05.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.081+0000 7f26d63ff700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26d01002d0 0x7f26d01960a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005740 tx=0x7f26c0005770 comp rx=0 tx=0).stop 2023-11-27T15:23:05.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.081+0000 7f26d63ff700 1 -- 172.21.15.8:0/399032270 shutdown_connections 2023-11-27T15:23:05.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.081+0000 7f26d63ff700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26d0101e50 0x7f26d019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:05.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.081+0000 7f26d63ff700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26d01002d0 0x7f26d01960a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:05.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.081+0000 7f26d63ff700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26b80a1150 0x7f26b80a3600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:05.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.081+0000 7f26d63ff700 1 --2- 172.21.15.8:0/399032270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26d01014c0 0x7f26d01965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:05.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.081+0000 7f26d63ff700 1 -- 172.21.15.8:0/399032270 >> 172.21.15.8:0/399032270 conn(0x7f26d00fb8b0 msgr2=0x7f26d01049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:05.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.082+0000 7f26d63ff700 1 -- 172.21.15.8:0/399032270 shutdown_connections 2023-11-27T15:23:05.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:05.082+0000 7f26d63ff700 1 -- 172.21.15.8:0/399032270 wait complete. 2023-11-27T15:23:05.087 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 7 2023-11-27T15:23:05.308 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.60:6834/612130958,v1:172.21.15.60:6835/612130958] up:boot 2023-11-27T15:23:05.308 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: fsmap 6 up:standby 2023-11-27T15:23:05.308 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:23:05.308 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:05.308 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:05.308 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:05.308 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:05.308 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:05.309 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:05.309 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: Deploying daemon mds.c on smithi060 2023-11-27T15:23:05.309 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:05.309 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:05 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/399032270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:05.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.60:6834/612130958,v1:172.21.15.60:6835/612130958] up:boot 2023-11-27T15:23:05.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: fsmap 6 up:standby 2023-11-27T15:23:05.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:23:05.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:05.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:05.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:05.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:05.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:05.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:05.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: Deploying daemon mds.c on smithi060 2023-11-27T15:23:05.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:05.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:05 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/399032270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:05.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.60:6834/612130958,v1:172.21.15.60:6835/612130958] up:boot 2023-11-27T15:23:05.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: fsmap 6 up:standby 2023-11-27T15:23:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:23:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: Deploying daemon mds.c on smithi060 2023-11-27T15:23:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:05.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:05 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/399032270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:06.150 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:23:05 smithi060 systemd[1]: Starting Ceph mds.c for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:23:06.401 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:23:06 smithi060 podman[157436]: 2023-11-27T15:23:06.401 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:23:06 smithi060 ceph-mds[157483]: -- 172.21.15.60:0/936995894 <== mon.1 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5632e500f520 con 0x5632e4fe4800 2023-11-27T15:23:06.824 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:23:06 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-c[157452]: starting mds.c at 2023-11-27T15:23:06.824 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:23:06 smithi060 bash[157436]: 229b83c376dc5b9197f123f41edc6a9c60eb7c782e990a849f0cb4e34e54eaab 2023-11-27T15:23:06.825 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:23:06 smithi060 systemd[1]: Started Ceph mds.c for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:23:07.105 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:07.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.317+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3781772977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1781002d0 msgr2=0x7fa1781006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:07.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.317+0000 7fa17d5ff700 1 --2- 172.21.15.8:0/3781772977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1781002d0 0x7fa1781006e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1680098f0 tx=0x7fa168009c00 comp rx=0 tx=0).stop 2023-11-27T15:23:07.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.317+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3781772977 shutdown_connections 2023-11-27T15:23:07.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.317+0000 7fa17d5ff700 1 --2- 172.21.15.8:0/3781772977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa178101e50 0x7fa178108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:07.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.317+0000 7fa17d5ff700 1 --2- 172.21.15.8:0/3781772977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1781002d0 0x7fa1781006e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:07.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.317+0000 7fa17d5ff700 1 --2- 172.21.15.8:0/3781772977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa1781014c0 0x7fa178101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:07.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.317+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3781772977 >> 172.21.15.8:0/3781772977 conn(0x7fa1780fb8b0 msgr2=0x7fa1780fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:07.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.317+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3781772977 shutdown_connections 2023-11-27T15:23:07.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.317+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3781772977 wait complete. 2023-11-27T15:23:07.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa17d5ff700 1 Processor -- start 2023-11-27T15:23:07.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa17d5ff700 1 -- start start 2023-11-27T15:23:07.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa17d5ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1781002d0 0x7fa1781982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa17d5ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1781014c0 0x7fa178198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa17d5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa178101e50 0x7fa17819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa17d5ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17819fe70 con 0x7fa1781014c0 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa17d5ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17819ffb0 con 0x7fa1781002d0 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa17d5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1781a0140 con 0x7fa178101e50 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa176ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1781002d0 0x7fa1781982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa176ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1781002d0 0x7fa1781982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50188/0 (socket says 172.21.15.8:50188) 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa176ffd700 1 -- 172.21.15.8:0/3575280734 learned_addr learned my addr 172.21.15.8:0/3575280734 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.318+0000 7fa16ffff700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1781014c0 0x7fa178198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa1777fe700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa178101e50 0x7fa17819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:07.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa176ffd700 1 -- 172.21.15.8:0/3575280734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa178101e50 msgr2=0x7fa17819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa176ffd700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa178101e50 0x7fa17819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa176ffd700 1 -- 172.21.15.8:0/3575280734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1781014c0 msgr2=0x7fa178198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa176ffd700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1781014c0 0x7fa178198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa176ffd700 1 -- 172.21.15.8:0/3575280734 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168009520 con 0x7fa1781002d0 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa1777fe700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa178101e50 0x7fa17819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa176ffd700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1781002d0 0x7fa1781982f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa168005e90 tx=0x7fa16800b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa174ff9700 1 -- 172.21.15.8:0/3575280734 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16801c070 con 0x7fa1781002d0 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3575280734 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17810bca0 con 0x7fa1781002d0 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa174ff9700 1 -- 172.21.15.8:0/3575280734 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa16800f7f0 con 0x7fa1781002d0 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.319+0000 7fa174ff9700 1 -- 172.21.15.8:0/3575280734 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa168003b00 con 0x7fa1781002d0 2023-11-27T15:23:07.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.320+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3575280734 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17810bb10 con 0x7fa1781002d0 2023-11-27T15:23:07.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.320+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3575280734 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17810be30 con 0x7fa1781002d0 2023-11-27T15:23:07.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.322+0000 7fa174ff9700 1 -- 172.21.15.8:0/3575280734 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa168020030 con 0x7fa1781002d0 2023-11-27T15:23:07.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.324+0000 7fa174ff9700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa1640a1150 0x7fa1640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:07.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.324+0000 7fa174ff9700 1 -- 172.21.15.8:0/3575280734 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa1680cd510 con 0x7fa1781002d0 2023-11-27T15:23:07.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.324+0000 7fa16ffff700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa1640a1150 0x7fa1640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:07.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.325+0000 7fa16ffff700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa1640a1150 0x7fa1640a3600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa160009c60 tx=0x7fa160009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:07.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.328+0000 7fa174ff9700 1 -- 172.21.15.8:0/3575280734 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa1680970f0 con 0x7fa1781002d0 2023-11-27T15:23:07.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.589+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3575280734 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa178192a00 con 0x7fa1781002d0 2023-11-27T15:23:07.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.589+0000 7fa174ff9700 1 -- 172.21.15.8:0/3575280734 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5781 (secure 0 0 0) 0x7fa16809a710 con 0x7fa1781002d0 2023-11-27T15:23:07.593 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:07.593 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-11-27T15:23:07.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3575280734 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa1640a1150 msgr2=0x7fa1640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:07.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa1640a1150 0x7fa1640a3600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa160009c60 tx=0x7fa160009260 comp rx=0 tx=0).stop 2023-11-27T15:23:07.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3575280734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1781002d0 msgr2=0x7fa1781982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:07.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1781002d0 0x7fa1781982f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa168005e90 tx=0x7fa16800b730 comp rx=0 tx=0).stop 2023-11-27T15:23:07.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3575280734 shutdown_connections 2023-11-27T15:23:07.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa1640a1150 0x7fa1640a3600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:07.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa178101e50 0x7fa17819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:07.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa1781002d0 0x7fa1781982f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:07.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 --2- 172.21.15.8:0/3575280734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1781014c0 0x7fa178198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:07.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3575280734 >> 172.21.15.8:0/3575280734 conn(0x7fa1780fb8b0 msgr2=0x7fa178106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:07.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3575280734 shutdown_connections 2023-11-27T15:23:07.597 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:07.592+0000 7fa17d5ff700 1 -- 172.21.15.8:0/3575280734 wait complete. 2023-11-27T15:23:07.597 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 8 2023-11-27T15:23:07.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:07 smithi029 ceph-mon[131166]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:07.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:07.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:07.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:07 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.60:6836/287097286,v1:172.21.15.60:6837/287097286] up:boot 2023-11-27T15:23:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:07 smithi029 ceph-mon[131166]: fsmap 7 up:standby 2023-11-27T15:23:07.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:23:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:07 smithi060 ceph-mon[131518]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:07 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.60:6836/287097286,v1:172.21.15.60:6837/287097286] up:boot 2023-11-27T15:23:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:07 smithi060 ceph-mon[131518]: fsmap 7 up:standby 2023-11-27T15:23:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:23:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:07 smithi008 ceph-mon[128788]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:07.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:07 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.60:6836/287097286,v1:172.21.15.60:6837/287097286] up:boot 2023-11-27T15:23:07.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:07 smithi008 ceph-mon[128788]: fsmap 7 up:standby 2023-11-27T15:23:07.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:23:08.398 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:23:08 smithi029 systemd[1]: Starting Ceph mds.h for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:23:08.810 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:08 smithi029 ceph-mon[131166]: Deploying daemon mds.h on smithi029 2023-11-27T15:23:08.811 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:08 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3575280734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:08.811 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:23:08 smithi029 podman[156126]: 2023-11-27T15:23:08.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:08 smithi060 ceph-mon[131518]: Deploying daemon mds.h on smithi029 2023-11-27T15:23:08.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:08 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3575280734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:08 smithi008 ceph-mon[128788]: Deploying daemon mds.h on smithi029 2023-11-27T15:23:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:08 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3575280734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:09.147 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:23:08 smithi029 ceph-mds[156166]: -- 172.21.15.29:0/4205085407 <== mon.2 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564d200bf520 con 0x564d20094c00 2023-11-27T15:23:09.147 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:23:08 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-h[156142]: starting mds.h at 2023-11-27T15:23:09.147 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:23:08 smithi029 bash[156126]: edcff02d9e519641cd22737759b4db80329df449277f23be7b4fb77494d93ccd 2023-11-27T15:23:09.148 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:23:08 smithi029 systemd[1]: Started Ceph mds.h for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:23:09.616 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:09 smithi060 ceph-mon[131518]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:09.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:09.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:09.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.826+0000 7f9f54dff700 1 -- 172.21.15.8:0/2573897624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50100d80 msgr2=0x7f9f50103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:09.830 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.826+0000 7f9f54dff700 1 --2- 172.21.15.8:0/2573897624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50100d80 0x7f9f50103160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009990 tx=0x7f9f40009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:09.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.826+0000 7f9f54dff700 1 -- 172.21.15.8:0/2573897624 shutdown_connections 2023-11-27T15:23:09.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.826+0000 7f9f54dff700 1 --2- 172.21.15.8:0/2573897624 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f50105fc0 0x7f9f501083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:09.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.826+0000 7f9f54dff700 1 --2- 172.21.15.8:0/2573897624 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50100d80 0x7f9f50103160 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:09.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.826+0000 7f9f54dff700 1 --2- 172.21.15.8:0/2573897624 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f501036a0 0x7f9f50105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:09.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.826+0000 7f9f54dff700 1 -- 172.21.15.8:0/2573897624 >> 172.21.15.8:0/2573897624 conn(0x7f9f500fa7c0 msgr2=0x7f9f500fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:09.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.826+0000 7f9f54dff700 1 -- 172.21.15.8:0/2573897624 shutdown_connections 2023-11-27T15:23:09.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.827+0000 7f9f54dff700 1 -- 172.21.15.8:0/2573897624 wait complete. 2023-11-27T15:23:09.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.828+0000 7f9f54dff700 1 Processor -- start 2023-11-27T15:23:09.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.828+0000 7f9f54dff700 1 -- start start 2023-11-27T15:23:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f54dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50100d80 0x7f9f501960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f54dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f501036a0 0x7f9f50196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f54dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f50105fc0 0x7f9f5019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f54dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5019dc40 con 0x7f9f50100d80 2023-11-27T15:23:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f54dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5019dd80 con 0x7f9f501036a0 2023-11-27T15:23:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f54dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5019df10 con 0x7f9f50105fc0 2023-11-27T15:23:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f4e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50100d80 0x7f9f501960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f4ed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f50105fc0 0x7f9f5019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f4e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50100d80 0x7f9f501960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34706/0 (socket says 172.21.15.8:34706) 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f4e59c700 1 -- 172.21.15.8:0/2556178217 learned_addr learned my addr 172.21.15.8:0/2556178217 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f4dd9b700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f501036a0 0x7f9f50196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f4e59c700 1 -- 172.21.15.8:0/2556178217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f50105fc0 msgr2=0x7f9f5019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f4e59c700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f50105fc0 0x7f9f5019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f4e59c700 1 -- 172.21.15.8:0/2556178217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f501036a0 msgr2=0x7f9f50196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f4e59c700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f501036a0 0x7f9f50196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.829+0000 7f9f4e59c700 1 -- 172.21.15.8:0/2556178217 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400095f0 con 0x7f9f50100d80 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.830+0000 7f9f4e59c700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50100d80 0x7f9f501960c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b680 tx=0x7f9f40005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.830+0000 7f9f3f7fe700 1 -- 172.21.15.8:0/2556178217 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4001d070 con 0x7f9f50100d80 2023-11-27T15:23:09.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.830+0000 7f9f3f7fe700 1 -- 172.21.15.8:0/2556178217 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f40021410 con 0x7f9f50100d80 2023-11-27T15:23:09.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.830+0000 7f9f54dff700 1 -- 172.21.15.8:0/2556178217 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5010bc70 con 0x7f9f50100d80 2023-11-27T15:23:09.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.830+0000 7f9f3f7fe700 1 -- 172.21.15.8:0/2556178217 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40017650 con 0x7f9f50100d80 2023-11-27T15:23:09.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.830+0000 7f9f54dff700 1 -- 172.21.15.8:0/2556178217 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5010bae0 con 0x7f9f50100d80 2023-11-27T15:23:09.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.831+0000 7f9f54dff700 1 -- 172.21.15.8:0/2556178217 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f500fc330 con 0x7f9f50100d80 2023-11-27T15:23:09.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.833+0000 7f9f3f7fe700 1 -- 172.21.15.8:0/2556178217 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9f400217f0 con 0x7f9f50100d80 2023-11-27T15:23:09.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.835+0000 7f9f3f7fe700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f380a11d0 0x7f9f380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:09.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.835+0000 7f9f3f7fe700 1 -- 172.21.15.8:0/2556178217 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9f400cd8a0 con 0x7f9f50100d80 2023-11-27T15:23:09.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.835+0000 7f9f4dd9b700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f380a11d0 0x7f9f380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:09.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.836+0000 7f9f4dd9b700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f380a11d0 0x7f9f380a3680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009c60 tx=0x7f9f44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:09.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:09.839+0000 7f9f3f7fe700 1 -- 172.21.15.8:0/2556178217 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9f40097480 con 0x7f9f50100d80 2023-11-27T15:23:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:09 smithi008 ceph-mon[128788]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:09.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:09 smithi029 ceph-mon[131166]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:09.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:10.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.092+0000 7f9f54dff700 1 -- 172.21.15.8:0/2556178217 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f5002cfe0 con 0x7f9f50100d80 2023-11-27T15:23:10.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.093+0000 7f9f3f7fe700 1 -- 172.21.15.8:0/2556178217 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6538 (secure 0 0 0) 0x7f9f4009aaa0 con 0x7f9f50100d80 2023-11-27T15:23:10.097 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:10.097 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-11-27T15:23:10.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.095+0000 7f9f54dff700 1 -- 172.21.15.8:0/2556178217 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f380a11d0 msgr2=0x7f9f380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:10.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.095+0000 7f9f54dff700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f380a11d0 0x7f9f380a3680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009c60 tx=0x7f9f44009260 comp rx=0 tx=0).stop 2023-11-27T15:23:10.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.095+0000 7f9f54dff700 1 -- 172.21.15.8:0/2556178217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50100d80 msgr2=0x7f9f501960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:10.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.095+0000 7f9f54dff700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50100d80 0x7f9f501960c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000b680 tx=0x7f9f40005740 comp rx=0 tx=0).stop 2023-11-27T15:23:10.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.095+0000 7f9f54dff700 1 -- 172.21.15.8:0/2556178217 shutdown_connections 2023-11-27T15:23:10.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.095+0000 7f9f54dff700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9f380a11d0 0x7f9f380a3680 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:10.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.095+0000 7f9f54dff700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9f50105fc0 0x7f9f5019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:10.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.095+0000 7f9f54dff700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f50100d80 0x7f9f501960c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:10.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.095+0000 7f9f54dff700 1 --2- 172.21.15.8:0/2556178217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9f501036a0 0x7f9f50196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:10.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.095+0000 7f9f54dff700 1 -- 172.21.15.8:0/2556178217 >> 172.21.15.8:0/2556178217 conn(0x7f9f500fa7c0 msgr2=0x7f9f501019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:10.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.096+0000 7f9f54dff700 1 -- 172.21.15.8:0/2556178217 shutdown_connections 2023-11-27T15:23:10.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:10.096+0000 7f9f54dff700 1 -- 172.21.15.8:0/2556178217 wait complete. 2023-11-27T15:23:10.100 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 9 2023-11-27T15:23:10.495 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:23:10 smithi060 systemd[1]: Starting Ceph mds.i for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:23:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:10 smithi060 ceph-mon[131518]: Deploying daemon mds.i on smithi060 2023-11-27T15:23:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:10 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] up:boot 2023-11-27T15:23:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:10 smithi060 ceph-mon[131518]: fsmap 8 up:standby 2023-11-27T15:23:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:10 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:23:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:10 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2556178217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:10 smithi008 ceph-mon[128788]: Deploying daemon mds.i on smithi060 2023-11-27T15:23:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:10 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] up:boot 2023-11-27T15:23:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:10 smithi008 ceph-mon[128788]: fsmap 8 up:standby 2023-11-27T15:23:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:10 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:23:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:10 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2556178217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:10.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:10 smithi029 ceph-mon[131166]: Deploying daemon mds.i on smithi060 2023-11-27T15:23:10.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:10 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] up:boot 2023-11-27T15:23:10.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:10 smithi029 ceph-mon[131166]: fsmap 8 up:standby 2023-11-27T15:23:10.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:10 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:23:10.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:10 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2556178217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:11.091 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:23:10 smithi060 podman[157788]: 2023-11-27T15:23:11.091 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:23:11 smithi060 bash[157788]: 7a380fe8bc5b47eac656d1a8fd51c5fd70554de13ff24205dd39b455af051c54 2023-11-27T15:23:11.091 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:23:11 smithi060 systemd[1]: Started Ceph mds.i for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:23:11.574 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:23:11 smithi060 ceph-mds[157828]: -- 172.21.15.60:0/2415166653 <== mon.1 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56373e485520 con 0x56373e45b000 2023-11-27T15:23:11.574 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:23:11 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: starting mds.i at 2023-11-27T15:23:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:11 smithi008 ceph-mon[128788]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:11.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:11.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:11.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:11.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:11.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:11 smithi029 ceph-mon[131166]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:11.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:11.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:11.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:11.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:11.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:12.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:11 smithi060 ceph-mon[131518]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:12.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:12.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:12.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:12.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:12.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:12.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:12.119 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:12.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.321+0000 7f237f9ff700 1 -- 172.21.15.8:0/4028211190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23781002d0 msgr2=0x7f23781006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.321+0000 7f237f9ff700 1 --2- 172.21.15.8:0/4028211190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23781002d0 0x7f23781006e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2374009990 tx=0x7f2374009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.321+0000 7f237f9ff700 1 -- 172.21.15.8:0/4028211190 shutdown_connections 2023-11-27T15:23:12.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.321+0000 7f237f9ff700 1 --2- 172.21.15.8:0/4028211190 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2378101e50 0x7f2378108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.321+0000 7f237f9ff700 1 --2- 172.21.15.8:0/4028211190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f23781002d0 0x7f23781006e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.321+0000 7f237f9ff700 1 --2- 172.21.15.8:0/4028211190 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781014c0 0x7f2378101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.321+0000 7f237f9ff700 1 -- 172.21.15.8:0/4028211190 >> 172.21.15.8:0/4028211190 conn(0x7f23780fb8b0 msgr2=0x7f23780fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:12.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.321+0000 7f237f9ff700 1 -- 172.21.15.8:0/4028211190 shutdown_connections 2023-11-27T15:23:12.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.321+0000 7f237f9ff700 1 -- 172.21.15.8:0/4028211190 wait complete. 2023-11-27T15:23:12.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.322+0000 7f237f9ff700 1 Processor -- start 2023-11-27T15:23:12.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.322+0000 7f237f9ff700 1 -- start start 2023-11-27T15:23:12.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.322+0000 7f237f9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23781002d0 0x7f2378198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.322+0000 7f237f9ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781014c0 0x7f23781987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.322+0000 7f237f9ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2378101e50 0x7f237819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.322+0000 7f237f9ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237819fde0 con 0x7f2378101e50 2023-11-27T15:23:12.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.322+0000 7f237f9ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237819ff20 con 0x7f23781014c0 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237f9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23781a00b0 con 0x7f23781002d0 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237df9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2378101e50 0x7f237819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237d79b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23781002d0 0x7f2378198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237df9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2378101e50 0x7f237819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51188/0 (socket says 172.21.15.8:51188) 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237df9c700 1 -- 172.21.15.8:0/315570148 learned_addr learned my addr 172.21.15.8:0/315570148 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237d79b700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23781002d0 0x7f2378198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:36698/0 (socket says 172.21.15.8:36698) 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237cf9a700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781014c0 0x7f23781987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237df9c700 1 -- 172.21.15.8:0/315570148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23781002d0 msgr2=0x7f2378198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237df9c700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23781002d0 0x7f2378198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237df9c700 1 -- 172.21.15.8:0/315570148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781014c0 msgr2=0x7f23781987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237df9c700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781014c0 0x7f23781987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237df9c700 1 -- 172.21.15.8:0/315570148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740095f0 con 0x7f2378101e50 2023-11-27T15:23:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.323+0000 7f237df9c700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2378101e50 0x7f237819f810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f237000ef30 tx=0x7f237000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.324+0000 7f236e7fc700 1 -- 172.21.15.8:0/315570148 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370018070 con 0x7f2378101e50 2023-11-27T15:23:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.324+0000 7f236e7fc700 1 -- 172.21.15.8:0/315570148 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f237000f040 con 0x7f2378101e50 2023-11-27T15:23:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.324+0000 7f236e7fc700 1 -- 172.21.15.8:0/315570148 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237000cd00 con 0x7f2378101e50 2023-11-27T15:23:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.324+0000 7f237f9ff700 1 -- 172.21.15.8:0/315570148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237810bca0 con 0x7f2378101e50 2023-11-27T15:23:12.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.324+0000 7f237f9ff700 1 -- 172.21.15.8:0/315570148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237810bb10 con 0x7f2378101e50 2023-11-27T15:23:12.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.325+0000 7f237f9ff700 1 -- 172.21.15.8:0/315570148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23781a06f0 con 0x7f2378101e50 2023-11-27T15:23:12.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.327+0000 7f236e7fc700 1 -- 172.21.15.8:0/315570148 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2370003a00 con 0x7f2378101e50 2023-11-27T15:23:12.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.328+0000 7f236e7fc700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f23640a11d0 0x7f23640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:12.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.328+0000 7f236e7fc700 1 -- 172.21.15.8:0/315570148 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f23700cc8d0 con 0x7f2378101e50 2023-11-27T15:23:12.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.328+0000 7f237d79b700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f23640a11d0 0x7f23640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:12.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.329+0000 7f237d79b700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f23640a11d0 0x7f23640a3680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f237400b2e0 tx=0x7f2374005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:12.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.330+0000 7f236e7fc700 1 -- 172.21.15.8:0/315570148 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f23700964b0 con 0x7f2378101e50 2023-11-27T15:23:12.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.609+0000 7f237f9ff700 1 -- 172.21.15.8:0/315570148 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23781a0ca0 con 0x7f2378101e50 2023-11-27T15:23:12.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.610+0000 7f236e7fc700 1 -- 172.21.15.8:0/315570148 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7297 (secure 0 0 0) 0x7f2370099ad0 con 0x7f2378101e50 2023-11-27T15:23:12.614 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:12.614 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-27T15:23:12.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 -- 172.21.15.8:0/315570148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f23640a11d0 msgr2=0x7f23640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f23640a11d0 0x7f23640a3680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f237400b2e0 tx=0x7f2374005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 -- 172.21.15.8:0/315570148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2378101e50 msgr2=0x7f237819f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:12.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2378101e50 0x7f237819f810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f237000ef30 tx=0x7f237000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 -- 172.21.15.8:0/315570148 shutdown_connections 2023-11-27T15:23:12.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f23640a11d0 0x7f23640a3680 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f23781002d0 0x7f2378198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2378101e50 0x7f237819f810 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 --2- 172.21.15.8:0/315570148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f23781014c0 0x7f23781987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:12.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 -- 172.21.15.8:0/315570148 >> 172.21.15.8:0/315570148 conn(0x7f23780fb8b0 msgr2=0x7f2378106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:12.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 -- 172.21.15.8:0/315570148 shutdown_connections 2023-11-27T15:23:12.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:12.613+0000 7f237f9ff700 1 -- 172.21.15.8:0/315570148 wait complete. 2023-11-27T15:23:12.618 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 10 2023-11-27T15:23:12.738 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:12 smithi008 ceph-mon[128788]: Deploying daemon mds.g on smithi008 2023-11-27T15:23:12.738 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:12 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.60:6838/3185908756,v1:172.21.15.60:6839/3185908756] up:boot 2023-11-27T15:23:12.738 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:12 smithi008 ceph-mon[128788]: fsmap 9 up:standby 2023-11-27T15:23:12.738 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:23:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:12 smithi029 ceph-mon[131166]: Deploying daemon mds.g on smithi008 2023-11-27T15:23:12.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:12 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.60:6838/3185908756,v1:172.21.15.60:6839/3185908756] up:boot 2023-11-27T15:23:12.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:12 smithi029 ceph-mon[131166]: fsmap 9 up:standby 2023-11-27T15:23:12.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:23:13.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:12 smithi060 ceph-mon[131518]: Deploying daemon mds.g on smithi008 2023-11-27T15:23:13.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:12 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.60:6838/3185908756,v1:172.21.15.60:6839/3185908756] up:boot 2023-11-27T15:23:13.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:12 smithi060 ceph-mon[131518]: fsmap 9 up:standby 2023-11-27T15:23:13.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:23:13.680 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:13 smithi008 ceph-mon[128788]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:13.680 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:13 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/315570148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:13.681 INFO:journalctl@ceph.mds.g.smithi008.stdout:Nov 27 15:23:13 smithi008 systemd[1]: Starting Ceph mds.g for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:23:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:13 smithi029 ceph-mon[131166]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:13 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/315570148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:14.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:13 smithi060 ceph-mon[131518]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:14.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:13 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/315570148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:14.590 INFO:journalctl@ceph.mds.g.smithi008.stdout:Nov 27 15:23:14 smithi008 podman[162832]: 2023-11-27T15:23:14.638 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:14.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.857+0000 7fe334fff700 1 -- 172.21.15.8:0/2785576890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe330108190 msgr2=0x7fe33010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:14.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.857+0000 7fe334fff700 1 --2- 172.21.15.8:0/2785576890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe330108190 0x7fe33010a610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe32800b0a0 tx=0x7fe32800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:23:14.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.857+0000 7fe334fff700 1 -- 172.21.15.8:0/2785576890 shutdown_connections 2023-11-27T15:23:14.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.857+0000 7fe334fff700 1 --2- 172.21.15.8:0/2785576890 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe330108190 0x7fe33010a610 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:14.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.857+0000 7fe334fff700 1 --2- 172.21.15.8:0/2785576890 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe330074b30 0x7fe330072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:14.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.857+0000 7fe334fff700 1 --2- 172.21.15.8:0/2785576890 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe330073550 0x7fe3300739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:14.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.857+0000 7fe334fff700 1 -- 172.21.15.8:0/2785576890 >> 172.21.15.8:0/2785576890 conn(0x7fe3300fb8b0 msgr2=0x7fe3300fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:14.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.857+0000 7fe334fff700 1 -- 172.21.15.8:0/2785576890 shutdown_connections 2023-11-27T15:23:14.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.857+0000 7fe334fff700 1 -- 172.21.15.8:0/2785576890 wait complete. 2023-11-27T15:23:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.858+0000 7fe334fff700 1 Processor -- start 2023-11-27T15:23:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.858+0000 7fe334fff700 1 -- start start 2023-11-27T15:23:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe334fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe330073550 0x7fe3301983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe334fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe330074b30 0x7fe3301988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe334fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe330108190 0x7fe33019f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe334fff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33019ff30 con 0x7fe330074b30 2023-11-27T15:23:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe334fff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3301a0070 con 0x7fe330073550 2023-11-27T15:23:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe334fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3301a0200 con 0x7fe330108190 2023-11-27T15:23:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe32e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe330073550 0x7fe3301983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe32e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe330073550 0x7fe3301983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:46846/0 (socket says 172.21.15.8:46846) 2023-11-27T15:23:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe32e59c700 1 -- 172.21.15.8:0/955774843 learned_addr learned my addr 172.21.15.8:0/955774843 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe32ed9d700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe330108190 0x7fe33019f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:14.864 INFO:journalctl@ceph.mds.g.smithi008.stdout:Nov 27 15:23:14 smithi008 ceph-mds[162872]: -- 172.21.15.8:0/3173998291 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d3eb3e7520 con 0x55d3eb3bf000 2023-11-27T15:23:14.864 INFO:journalctl@ceph.mds.g.smithi008.stdout:Nov 27 15:23:14 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-g[162848]: starting mds.g at 2023-11-27T15:23:14.864 INFO:journalctl@ceph.mds.g.smithi008.stdout:Nov 27 15:23:14 smithi008 bash[162832]: 24375935d184dd0693c92b589f21debbc542e3ebd27699027d031a460c7a79b6 2023-11-27T15:23:14.865 INFO:journalctl@ceph.mds.g.smithi008.stdout:Nov 27 15:23:14 smithi008 systemd[1]: Started Ceph mds.g for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:23:14.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.859+0000 7fe32dd9b700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe330074b30 0x7fe3301988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:14.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.860+0000 7fe32e59c700 1 -- 172.21.15.8:0/955774843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe330108190 msgr2=0x7fe33019f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:14.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.860+0000 7fe32e59c700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe330108190 0x7fe33019f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:14.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.860+0000 7fe32e59c700 1 -- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe330074b30 msgr2=0x7fe3301988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:14.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.860+0000 7fe32e59c700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe330074b30 0x7fe3301988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:14.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.860+0000 7fe32e59c700 1 -- 172.21.15.8:0/955774843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe328009d00 con 0x7fe330073550 2023-11-27T15:23:14.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.860+0000 7fe32dd9b700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe330074b30 0x7fe3301988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:14.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.860+0000 7fe32ed9d700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe330108190 0x7fe33019f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:14.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.860+0000 7fe32e59c700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe330073550 0x7fe3301983b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe32000e970 tx=0x7fe32000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:14.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.860+0000 7fe31f7fe700 1 -- 172.21.15.8:0/955774843 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe320009790 con 0x7fe330073550 2023-11-27T15:23:14.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.861+0000 7fe334fff700 1 -- 172.21.15.8:0/955774843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33010ded0 con 0x7fe330073550 2023-11-27T15:23:14.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.861+0000 7fe31f7fe700 1 -- 172.21.15.8:0/955774843 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe32000c7e0 con 0x7fe330073550 2023-11-27T15:23:14.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.861+0000 7fe31f7fe700 1 -- 172.21.15.8:0/955774843 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3200054e0 con 0x7fe330073550 2023-11-27T15:23:14.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.861+0000 7fe334fff700 1 -- 172.21.15.8:0/955774843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33010dd40 con 0x7fe330073550 2023-11-27T15:23:14.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.862+0000 7fe334fff700 1 -- 172.21.15.8:0/955774843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe330106840 con 0x7fe330073550 2023-11-27T15:23:14.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.864+0000 7fe31f7fe700 1 -- 172.21.15.8:0/955774843 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe32000c950 con 0x7fe330073550 2023-11-27T15:23:14.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.865+0000 7fe31f7fe700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3180a1150 0x7fe3180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:14.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.865+0000 7fe31f7fe700 1 -- 172.21.15.8:0/955774843 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe320014070 con 0x7fe330073550 2023-11-27T15:23:14.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.865+0000 7fe32dd9b700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3180a1150 0x7fe3180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:14.870 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.866+0000 7fe32dd9b700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3180a1150 0x7fe3180a3600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe330074850 tx=0x7fe324009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:14.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:14.867+0000 7fe31f7fe700 1 -- 172.21.15.8:0/955774843 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe320096900 con 0x7fe330073550 2023-11-27T15:23:15.126 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.122+0000 7fe334fff700 1 -- 172.21.15.8:0/955774843 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe33002cfe0 con 0x7fe330073550 2023-11-27T15:23:15.127 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.123+0000 7fe31f7fe700 1 -- 172.21.15.8:0/955774843 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7297 (secure 0 0 0) 0x7fe320099f20 con 0x7fe330073550 2023-11-27T15:23:15.127 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:15.127 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-27T15:23:15.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 -- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3180a1150 msgr2=0x7fe3180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3180a1150 0x7fe3180a3600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe330074850 tx=0x7fe324009260 comp rx=0 tx=0).stop 2023-11-27T15:23:15.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 -- 172.21.15.8:0/955774843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe330073550 msgr2=0x7fe3301983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:15.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe330073550 0x7fe3301983b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe32000e970 tx=0x7fe32000ec80 comp rx=0 tx=0).stop 2023-11-27T15:23:15.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 -- 172.21.15.8:0/955774843 shutdown_connections 2023-11-27T15:23:15.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3180a1150 0x7fe3180a3600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe330108190 0x7fe33019f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe330073550 0x7fe3301983b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 --2- 172.21.15.8:0/955774843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe330074b30 0x7fe3301988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:15.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 -- 172.21.15.8:0/955774843 >> 172.21.15.8:0/955774843 conn(0x7fe3300fb8b0 msgr2=0x7fe3300fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:15.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 -- 172.21.15.8:0/955774843 shutdown_connections 2023-11-27T15:23:15.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:15.126+0000 7fe334fff700 1 -- 172.21.15.8:0/955774843 wait complete. 2023-11-27T15:23:15.131 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 10 2023-11-27T15:23:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:15 smithi008 ceph-mon[128788]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:15.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:15 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:15.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:15 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/955774843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:15 smithi029 ceph-mon[131166]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:15.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:15.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:15.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:15 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:15.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:15 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/955774843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:16.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:15 smithi060 ceph-mon[131518]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:16.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:16.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:16.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:16.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:16.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:16.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:15 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:16.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:15 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/955774843' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:16.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:16 smithi008 ceph-mon[128788]: Deploying daemon mds.a on smithi008 2023-11-27T15:23:16.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:16 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.8:6838/2208262973,v1:172.21.15.8:6839/2208262973] up:boot 2023-11-27T15:23:16.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:16 smithi008 ceph-mon[128788]: fsmap 10 up:standby 2023-11-27T15:23:16.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:16 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:23:16.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:16 smithi008 ceph-mon[128788]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:16.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:16 smithi029 ceph-mon[131166]: Deploying daemon mds.a on smithi008 2023-11-27T15:23:16.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:16 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.8:6838/2208262973,v1:172.21.15.8:6839/2208262973] up:boot 2023-11-27T15:23:16.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:16 smithi029 ceph-mon[131166]: fsmap 10 up:standby 2023-11-27T15:23:16.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:16 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:23:16.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:16 smithi029 ceph-mon[131166]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:17.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:16 smithi060 ceph-mon[131518]: Deploying daemon mds.a on smithi008 2023-11-27T15:23:17.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:16 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.8:6838/2208262973,v1:172.21.15.8:6839/2208262973] up:boot 2023-11-27T15:23:17.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:16 smithi060 ceph-mon[131518]: fsmap 10 up:standby 2023-11-27T15:23:17.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:16 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:23:17.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:16 smithi060 ceph-mon[131518]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:17.159 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.411+0000 7f5cccfff700 1 -- 172.21.15.8:0/1486890124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc81002d0 msgr2=0x7f5cc81006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.411+0000 7f5cccfff700 1 --2- 172.21.15.8:0/1486890124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc81002d0 0x7f5cc81006e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009990 tx=0x7f5cb8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.411+0000 7f5cccfff700 1 -- 172.21.15.8:0/1486890124 shutdown_connections 2023-11-27T15:23:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.411+0000 7f5cccfff700 1 --2- 172.21.15.8:0/1486890124 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc81002d0 0x7f5cc81006e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.411+0000 7f5cccfff700 1 --2- 172.21.15.8:0/1486890124 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc8101e50 0x7f5cc8108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.411+0000 7f5cccfff700 1 --2- 172.21.15.8:0/1486890124 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cc81014c0 0x7f5cc8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.411+0000 7f5cccfff700 1 -- 172.21.15.8:0/1486890124 >> 172.21.15.8:0/1486890124 conn(0x7f5cc80fb8b0 msgr2=0x7f5cc80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:17.416 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.411+0000 7f5cccfff700 1 -- 172.21.15.8:0/1486890124 shutdown_connections 2023-11-27T15:23:17.416 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.411+0000 7f5cccfff700 1 -- 172.21.15.8:0/1486890124 wait complete. 2023-11-27T15:23:17.416 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.413+0000 7f5cccfff700 1 Processor -- start 2023-11-27T15:23:17.416 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.413+0000 7f5cccfff700 1 -- start start 2023-11-27T15:23:17.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.413+0000 7f5cccfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc81002d0 0x7f5cc8196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:17.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.413+0000 7f5cccfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cc81014c0 0x7f5cc8196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:17.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.413+0000 7f5cccfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc8101e50 0x7f5cc819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:17.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.413+0000 7f5cccfff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc819db80 con 0x7f5cc81014c0 2023-11-27T15:23:17.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.413+0000 7f5cccfff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc819dcc0 con 0x7f5cc8101e50 2023-11-27T15:23:17.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.413+0000 7f5cccfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc819de50 con 0x7f5cc81002d0 2023-11-27T15:23:17.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc5d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cc81014c0 0x7f5cc8196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc81002d0 0x7f5cc8196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc81002d0 0x7f5cc8196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:36760/0 (socket says 172.21.15.8:36760) 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc659c700 1 -- 172.21.15.8:0/4132608208 learned_addr learned my addr 172.21.15.8:0/4132608208 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc6d9d700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc8101e50 0x7f5cc819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc659c700 1 -- 172.21.15.8:0/4132608208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc8101e50 msgr2=0x7f5cc819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc659c700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc8101e50 0x7f5cc819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc659c700 1 -- 172.21.15.8:0/4132608208 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cc81014c0 msgr2=0x7f5cc8196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc659c700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cc81014c0 0x7f5cc8196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc659c700 1 -- 172.21.15.8:0/4132608208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb80095f0 con 0x7f5cc81002d0 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc5d9b700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cc81014c0 0x7f5cc8196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.414+0000 7f5cc659c700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc81002d0 0x7f5cc8196000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800b680 tx=0x7f5cb8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:17.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.415+0000 7f5cb77fe700 1 -- 172.21.15.8:0/4132608208 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb801d070 con 0x7f5cc81002d0 2023-11-27T15:23:17.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.415+0000 7f5cb77fe700 1 -- 172.21.15.8:0/4132608208 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cb8021410 con 0x7f5cc81002d0 2023-11-27T15:23:17.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.415+0000 7f5cccfff700 1 -- 172.21.15.8:0/4132608208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc810bc20 con 0x7f5cc81002d0 2023-11-27T15:23:17.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.415+0000 7f5cb77fe700 1 -- 172.21.15.8:0/4132608208 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8017650 con 0x7f5cc81002d0 2023-11-27T15:23:17.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.415+0000 7f5cccfff700 1 -- 172.21.15.8:0/4132608208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc810ba90 con 0x7f5cc81002d0 2023-11-27T15:23:17.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.416+0000 7f5cccfff700 1 -- 172.21.15.8:0/4132608208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc810bdb0 con 0x7f5cc81002d0 2023-11-27T15:23:17.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.418+0000 7f5cb77fe700 1 -- 172.21.15.8:0/4132608208 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5cb8021580 con 0x7f5cc81002d0 2023-11-27T15:23:17.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.419+0000 7f5cb77fe700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5cb00a1150 0x7f5cb00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:17.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.419+0000 7f5cb77fe700 1 -- 172.21.15.8:0/4132608208 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5cb80cdad0 con 0x7f5cc81002d0 2023-11-27T15:23:17.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.420+0000 7f5cc5d9b700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5cb00a1150 0x7f5cb00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:17.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.420+0000 7f5cc5d9b700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5cb00a1150 0x7f5cb00a3600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc005950 tx=0x7f5cbc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:17.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.422+0000 7f5cb77fe700 1 -- 172.21.15.8:0/4132608208 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5cb80976b0 con 0x7f5cc81002d0 2023-11-27T15:23:17.613 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:23:17 smithi008 systemd[1]: Starting Ceph mds.a for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:23:17.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.701+0000 7f5cccfff700 1 -- 172.21.15.8:0/4132608208 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cc8067af0 con 0x7f5cc81002d0 2023-11-27T15:23:17.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.702+0000 7f5cb77fe700 1 -- 172.21.15.8:0/4132608208 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8052 (secure 0 0 0) 0x7f5cc8067af0 con 0x7f5cc81002d0 2023-11-27T15:23:17.706 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:17.707 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-11-27T15:23:17.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 -- 172.21.15.8:0/4132608208 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5cb00a1150 msgr2=0x7f5cb00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:17.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5cb00a1150 0x7f5cb00a3600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc005950 tx=0x7f5cbc00a300 comp rx=0 tx=0).stop 2023-11-27T15:23:17.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 -- 172.21.15.8:0/4132608208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc81002d0 msgr2=0x7f5cc8196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:17.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc81002d0 0x7f5cc8196000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5cb800b680 tx=0x7f5cb8005740 comp rx=0 tx=0).stop 2023-11-27T15:23:17.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 -- 172.21.15.8:0/4132608208 shutdown_connections 2023-11-27T15:23:17.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5cb00a1150 0x7f5cb00a3600 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5cc81002d0 0x7f5cc8196000 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.709 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5cc8101e50 0x7f5cc819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 --2- 172.21.15.8:0/4132608208 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cc81014c0 0x7f5cc8196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 -- 172.21.15.8:0/4132608208 >> 172.21.15.8:0/4132608208 conn(0x7f5cc80fb8b0 msgr2=0x7f5cc81049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:17.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 -- 172.21.15.8:0/4132608208 shutdown_connections 2023-11-27T15:23:17.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:17.705+0000 7f5cccfff700 1 -- 172.21.15.8:0/4132608208 wait complete. 2023-11-27T15:23:17.710 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 11 2023-11-27T15:23:17.994 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:17 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4132608208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:17.994 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:23:17 smithi008 podman[163268]: 2023-11-27T15:23:18.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:17 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/4132608208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:18.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:17 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/4132608208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:18.363 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:23:17 smithi008 bash[163268]: d8a917533494299b173f81f80ec0359ffaa15553bdc1f957873c4fba5976f05c 2023-11-27T15:23:18.363 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:23:18 smithi008 ceph-mds[163310]: -- 172.21.15.8:0/2895862414 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5585e06e5520 con 0x5585e06bcc00 2023-11-27T15:23:18.363 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:23:18 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-a[163286]: starting mds.a at 2023-11-27T15:23:18.364 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:23:18 smithi008 systemd[1]: Started Ceph mds.a for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:23:19.243 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:19.243 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:19.243 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:19.243 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:19.243 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:19.243 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:19.243 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: Deploying daemon mds.b on smithi029 2023-11-27T15:23:19.243 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:19.243 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] up:boot 2023-11-27T15:23:19.244 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: fsmap 11 up:standby 2023-11-27T15:23:19.244 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:19 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:23:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:19.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:19.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: Deploying daemon mds.b on smithi029 2023-11-27T15:23:19.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:19.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] up:boot 2023-11-27T15:23:19.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: fsmap 11 up:standby 2023-11-27T15:23:19.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:19 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:23:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:23:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:23:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:19.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: Deploying daemon mds.b on smithi029 2023-11-27T15:23:19.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:19.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] up:boot 2023-11-27T15:23:19.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: fsmap 11 up:standby 2023-11-27T15:23:19.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:19 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:23:19.729 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:19.898 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:23:19 smithi029 systemd[1]: Starting Ceph mds.b for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:23:19.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.941+0000 7fa77bdff700 1 -- 172.21.15.8:0/3712378116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7741002d0 msgr2=0x7fa7741006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:19.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.941+0000 7fa77bdff700 1 --2- 172.21.15.8:0/3712378116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7741002d0 0x7fa7741006e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa770009990 tx=0x7fa770009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:19.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.941+0000 7fa77bdff700 1 -- 172.21.15.8:0/3712378116 shutdown_connections 2023-11-27T15:23:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.941+0000 7fa77bdff700 1 --2- 172.21.15.8:0/3712378116 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7741002d0 0x7fa7741006e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.941+0000 7fa77bdff700 1 --2- 172.21.15.8:0/3712378116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa774101e50 0x7fa774108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.941+0000 7fa77bdff700 1 --2- 172.21.15.8:0/3712378116 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7741014c0 0x7fa774101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.941+0000 7fa77bdff700 1 -- 172.21.15.8:0/3712378116 >> 172.21.15.8:0/3712378116 conn(0x7fa7740fb8b0 msgr2=0x7fa7740fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.941+0000 7fa77bdff700 1 -- 172.21.15.8:0/3712378116 shutdown_connections 2023-11-27T15:23:19.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.941+0000 7fa77bdff700 1 -- 172.21.15.8:0/3712378116 wait complete. 2023-11-27T15:23:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.942+0000 7fa77bdff700 1 Processor -- start 2023-11-27T15:23:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.942+0000 7fa77bdff700 1 -- start start 2023-11-27T15:23:19.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77bdff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7741002d0 0x7fa774196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77bdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7741014c0 0x7fa774196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77bdff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa774101e50 0x7fa77419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77bdff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77419dba0 con 0x7fa774101e50 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77bdff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77419dce0 con 0x7fa7741002d0 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77bdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77419de70 con 0x7fa7741014c0 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77a39c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa774101e50 0x7fa77419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa779b9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7741002d0 0x7fa774196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77a39c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa774101e50 0x7fa77419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51254/0 (socket says 172.21.15.8:51254) 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77a39c700 1 -- 172.21.15.8:0/918154806 learned_addr learned my addr 172.21.15.8:0/918154806 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77939a700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7741014c0 0x7fa774196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77a39c700 1 -- 172.21.15.8:0/918154806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7741014c0 msgr2=0x7fa774196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:19.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77a39c700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7741014c0 0x7fa774196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:19.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77a39c700 1 -- 172.21.15.8:0/918154806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7741002d0 msgr2=0x7fa774196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:19.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.943+0000 7fa77a39c700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7741002d0 0x7fa774196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:19.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.944+0000 7fa77a39c700 1 -- 172.21.15.8:0/918154806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700095f0 con 0x7fa774101e50 2023-11-27T15:23:19.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.944+0000 7fa77a39c700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa774101e50 0x7fa77419d5d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00ef30 tx=0x7fa76c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:19.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.944+0000 7fa76affd700 1 -- 172.21.15.8:0/918154806 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c018070 con 0x7fa774101e50 2023-11-27T15:23:19.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.944+0000 7fa76affd700 1 -- 172.21.15.8:0/918154806 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa76c00f040 con 0x7fa774101e50 2023-11-27T15:23:19.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.944+0000 7fa77bdff700 1 -- 172.21.15.8:0/918154806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77410bc20 con 0x7fa774101e50 2023-11-27T15:23:19.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.944+0000 7fa76affd700 1 -- 172.21.15.8:0/918154806 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00cd00 con 0x7fa774101e50 2023-11-27T15:23:19.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.944+0000 7fa77bdff700 1 -- 172.21.15.8:0/918154806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77410ba90 con 0x7fa774101e50 2023-11-27T15:23:19.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.945+0000 7fa77bdff700 1 -- 172.21.15.8:0/918154806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa774107200 con 0x7fa774101e50 2023-11-27T15:23:19.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.947+0000 7fa76affd700 1 -- 172.21.15.8:0/918154806 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa76c003a00 con 0x7fa774101e50 2023-11-27T15:23:19.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.949+0000 7fa76affd700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7600a1150 0x7fa7600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:19.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.949+0000 7fa76affd700 1 -- 172.21.15.8:0/918154806 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa76c0cc060 con 0x7fa774101e50 2023-11-27T15:23:19.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.949+0000 7fa779b9b700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7600a1150 0x7fa7600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:19.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.950+0000 7fa779b9b700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7600a1150 0x7fa7600a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa77000b2e0 tx=0x7fa770005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:19.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:19.953+0000 7fa76affd700 1 -- 172.21.15.8:0/918154806 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa76c095c40 con 0x7fa774101e50 2023-11-27T15:23:20.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.208+0000 7fa77bdff700 1 -- 172.21.15.8:0/918154806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa77402cfe0 con 0x7fa774101e50 2023-11-27T15:23:20.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.209+0000 7fa76affd700 1 -- 172.21.15.8:0/918154806 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8804 (secure 0 0 0) 0x7fa76c099260 con 0x7fa774101e50 2023-11-27T15:23:20.213 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:20.213 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2023-11-27T15:23:20.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 -- 172.21.15.8:0/918154806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7600a1150 msgr2=0x7fa7600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:20.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7600a1150 0x7fa7600a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa77000b2e0 tx=0x7fa770005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:20.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 -- 172.21.15.8:0/918154806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa774101e50 msgr2=0x7fa77419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:20.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa774101e50 0x7fa77419d5d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00ef30 tx=0x7fa76c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:23:20.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 -- 172.21.15.8:0/918154806 shutdown_connections 2023-11-27T15:23:20.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa774101e50 0x7fa77419d5d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:20.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7741002d0 0x7fa774196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:20.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7600a1150 0x7fa7600a3600 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:20.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 --2- 172.21.15.8:0/918154806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7741014c0 0x7fa774196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:20.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 -- 172.21.15.8:0/918154806 >> 172.21.15.8:0/918154806 conn(0x7fa7740fb8b0 msgr2=0x7fa7740fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:20.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 -- 172.21.15.8:0/918154806 shutdown_connections 2023-11-27T15:23:20.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:20.212+0000 7fa77bdff700 1 -- 172.21.15.8:0/918154806 wait complete. 2023-11-27T15:23:20.217 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 12 2023-11-27T15:23:20.342 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:20 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/918154806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:20.342 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:23:20 smithi029 podman[156468]: 2023-11-27T15:23:20.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:20 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/918154806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:20.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:20 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/918154806' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:20.647 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:23:20 smithi029 ceph-mds[156508]: -- 172.21.15.29:0/2386291955 <== mon.2 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556179f99520 con 0x556179f6e800 2023-11-27T15:23:20.647 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:23:20 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-b[156484]: starting mds.b at 2023-11-27T15:23:20.648 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:23:20 smithi029 bash[156468]: dcc0411c8ed8403410612cfca5140874bcd323761f24e3267680080980f1998b 2023-11-27T15:23:20.648 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:23:20 smithi029 systemd[1]: Started Ceph mds.b for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:23:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:21 smithi060 ceph-mon[131518]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:21 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:21 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:21 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:21 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:21 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:21 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:21 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] up:boot 2023-11-27T15:23:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:21 smithi060 ceph-mon[131518]: fsmap 12 up:standby 2023-11-27T15:23:21.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:21 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:23:21.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:21 smithi008 ceph-mon[128788]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:21.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:21 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:21 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:21 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:21 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:21 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:21 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:21.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:21 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] up:boot 2023-11-27T15:23:21.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:21 smithi008 ceph-mon[128788]: fsmap 12 up:standby 2023-11-27T15:23:21.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:21 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:23:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:21 smithi029 ceph-mon[131166]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:21 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:21 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:21 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:21 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:21 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:21.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:21 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:21.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:21 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] up:boot 2023-11-27T15:23:21.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:21 smithi029 ceph-mon[131166]: fsmap 12 up:standby 2023-11-27T15:23:21.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:21 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:23:22.235 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:22.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.465+0000 7fb1805ff700 1 -- 172.21.15.8:0/189806993 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb178074b30 msgr2=0x7fb178072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.465+0000 7fb1805ff700 1 --2- 172.21.15.8:0/189806993 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb178074b30 0x7fb178072f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1740098f0 tx=0x7fb174009c00 comp rx=0 tx=0).stop 2023-11-27T15:23:22.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.465+0000 7fb1805ff700 1 -- 172.21.15.8:0/189806993 shutdown_connections 2023-11-27T15:23:22.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.465+0000 7fb1805ff700 1 --2- 172.21.15.8:0/189806993 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb17810c8a0 0x7fb17810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.465+0000 7fb1805ff700 1 --2- 172.21.15.8:0/189806993 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb178074b30 0x7fb178072f80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.465+0000 7fb1805ff700 1 --2- 172.21.15.8:0/189806993 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb178073550 0x7fb1780739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.465+0000 7fb1805ff700 1 -- 172.21.15.8:0/189806993 >> 172.21.15.8:0/189806993 conn(0x7fb1780fb8b0 msgr2=0x7fb1780fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.465+0000 7fb1805ff700 1 -- 172.21.15.8:0/189806993 shutdown_connections 2023-11-27T15:23:22.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.465+0000 7fb1805ff700 1 -- 172.21.15.8:0/189806993 wait complete. 2023-11-27T15:23:22.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.466+0000 7fb1805ff700 1 Processor -- start 2023-11-27T15:23:22.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.466+0000 7fb1805ff700 1 -- start start 2023-11-27T15:23:22.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb1805ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178073550 0x7fb17819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb1805ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb178074b30 0x7fb17819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb1805ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb17810c8a0 0x7fb1781a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb1805ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1781a4230 con 0x7fb178074b30 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb1805ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1781a4370 con 0x7fb17810c8a0 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb1805ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1781a4500 con 0x7fb178073550 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17e39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178073550 0x7fb17819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17db9a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb178074b30 0x7fb17819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17e39b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178073550 0x7fb17819c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:54988/0 (socket says 172.21.15.8:54988) 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17e39b700 1 -- 172.21.15.8:0/3413263713 learned_addr learned my addr 172.21.15.8:0/3413263713 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17eb9c700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb17810c8a0 0x7fb1781a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17eb9c700 1 -- 172.21.15.8:0/3413263713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178073550 msgr2=0x7fb17819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17eb9c700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178073550 0x7fb17819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17eb9c700 1 -- 172.21.15.8:0/3413263713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb178074b30 msgr2=0x7fb17819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17eb9c700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb178074b30 0x7fb17819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17eb9c700 1 -- 172.21.15.8:0/3413263713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174009520 con 0x7fb17810c8a0 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17e39b700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178073550 0x7fb17819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.467+0000 7fb17db9a700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb178074b30 0x7fb17819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.468+0000 7fb17eb9c700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb17810c8a0 0x7fb1781a3c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb17000efc0 tx=0x7fb17000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.468+0000 7fb16f7fe700 1 -- 172.21.15.8:0/3413263713 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170018070 con 0x7fb17810c8a0 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.468+0000 7fb1805ff700 1 -- 172.21.15.8:0/3413263713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb178112290 con 0x7fb17810c8a0 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.468+0000 7fb16f7fe700 1 -- 172.21.15.8:0/3413263713 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb17000f040 con 0x7fb17810c8a0 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.468+0000 7fb16f7fe700 1 -- 172.21.15.8:0/3413263713 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17000cd30 con 0x7fb17810c8a0 2023-11-27T15:23:22.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.468+0000 7fb1805ff700 1 -- 172.21.15.8:0/3413263713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb178112100 con 0x7fb17810c8a0 2023-11-27T15:23:22.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.469+0000 7fb1805ff700 1 -- 172.21.15.8:0/3413263713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1781023f0 con 0x7fb17810c8a0 2023-11-27T15:23:22.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.472+0000 7fb16f7fe700 1 -- 172.21.15.8:0/3413263713 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb170003a00 con 0x7fb17810c8a0 2023-11-27T15:23:22.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.474+0000 7fb16f7fe700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb1640a11d0 0x7fb1640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.474+0000 7fb16f7fe700 1 -- 172.21.15.8:0/3413263713 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb1700cc0d0 con 0x7fb17810c8a0 2023-11-27T15:23:22.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.474+0000 7fb17e39b700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb1640a11d0 0x7fb1640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.475+0000 7fb17e39b700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb1640a11d0 0x7fb1640a3680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb17400c010 tx=0x7fb174019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.477+0000 7fb16f7fe700 1 -- 172.21.15.8:0/3413263713 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb170095cb0 con 0x7fb17810c8a0 2023-11-27T15:23:22.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.739+0000 7fb1805ff700 1 -- 172.21.15.8:0/3413263713 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb17802cfe0 con 0x7fb17810c8a0 2023-11-27T15:23:22.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.740+0000 7fb16f7fe700 1 -- 172.21.15.8:0/3413263713 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9562 (secure 0 0 0) 0x7fb1700992d0 con 0x7fb17810c8a0 2023-11-27T15:23:22.744 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:22.744 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24593,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.743+0000 7fb1805ff700 1 -- 172.21.15.8:0/3413263713 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb1640a11d0 msgr2=0x7fb1640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb1640a11d0 0x7fb1640a3680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb17400c010 tx=0x7fb174019040 comp rx=0 tx=0).stop 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 -- 172.21.15.8:0/3413263713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb17810c8a0 msgr2=0x7fb1781a3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb17810c8a0 0x7fb1781a3c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb17000efc0 tx=0x7fb17000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 -- 172.21.15.8:0/3413263713 shutdown_connections 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb1640a11d0 0x7fb1640a3680 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb178073550 0x7fb17819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb17810c8a0 0x7fb1781a3c60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 --2- 172.21.15.8:0/3413263713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb178074b30 0x7fb17819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 -- 172.21.15.8:0/3413263713 >> 172.21.15.8:0/3413263713 conn(0x7fb1780fb8b0 msgr2=0x7fb1780fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 -- 172.21.15.8:0/3413263713 shutdown_connections 2023-11-27T15:23:22.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.744+0000 7fb1805ff700 1 -- 172.21.15.8:0/3413263713 wait complete. 2023-11-27T15:23:22.749 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 13 2023-11-27T15:23:22.767 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:23:22.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.978+0000 7f277a1ff700 1 -- 172.21.15.8:0/3603683677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2774102530 msgr2=0x7f2774102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.978+0000 7f277a1ff700 1 --2- 172.21.15.8:0/3603683677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2774102530 0x7f2774102940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2764009990 tx=0x7f2764009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.978+0000 7f277a1ff700 1 -- 172.21.15.8:0/3603683677 shutdown_connections 2023-11-27T15:23:22.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.978+0000 7f277a1ff700 1 --2- 172.21.15.8:0/3603683677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27741040b0 0x7f277410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.978+0000 7f277a1ff700 1 --2- 172.21.15.8:0/3603683677 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2774102530 0x7f2774102940 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.978+0000 7f277a1ff700 1 --2- 172.21.15.8:0/3603683677 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2774103720 0x7f2774103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.978+0000 7f277a1ff700 1 -- 172.21.15.8:0/3603683677 >> 172.21.15.8:0/3603683677 conn(0x7f27740fdb30 msgr2=0x7f27740fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.978+0000 7f277a1ff700 1 -- 172.21.15.8:0/3603683677 shutdown_connections 2023-11-27T15:23:22.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.978+0000 7f277a1ff700 1 -- 172.21.15.8:0/3603683677 wait complete. 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.979+0000 7f277a1ff700 1 Processor -- start 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.979+0000 7f277a1ff700 1 -- start start 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f277a1ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2774102530 0x7f2774198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f277a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2774103720 0x7f27741987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f277a1ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27741040b0 0x7f277419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f277a1ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277419fdf0 con 0x7f27741040b0 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f277a1ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277419ff30 con 0x7f2774102530 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f277a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27741a00c0 con 0x7f2774103720 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f2772ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2774103720 0x7f27741987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f2772ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2774103720 0x7f27741987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:55010/0 (socket says 172.21.15.8:55010) 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f2772ffd700 1 -- 172.21.15.8:0/1629686476 learned_addr learned my addr 172.21.15.8:0/1629686476 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f27737fe700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2774102530 0x7f2774198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f2773fff700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27741040b0 0x7f277419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f2772ffd700 1 -- 172.21.15.8:0/1629686476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2774102530 msgr2=0x7f2774198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f2772ffd700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2774102530 0x7f2774198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f2772ffd700 1 -- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27741040b0 msgr2=0x7f277419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f2772ffd700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27741040b0 0x7f277419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f2772ffd700 1 -- 172.21.15.8:0/1629686476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27640095f0 con 0x7f2774103720 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.980+0000 7f2773fff700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27741040b0 0x7f277419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.981+0000 7f2772ffd700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2774103720 0x7f27741987b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f276800e9a0 tx=0x7f276800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.981+0000 7f2770ff9700 1 -- 172.21.15.8:0/1629686476 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768009790 con 0x7f2774103720 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.981+0000 7f2770ff9700 1 -- 172.21.15.8:0/1629686476 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276800cbd0 con 0x7f2774103720 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.981+0000 7f277a1ff700 1 -- 172.21.15.8:0/1629686476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277410def0 con 0x7f2774103720 2023-11-27T15:23:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.981+0000 7f2770ff9700 1 -- 172.21.15.8:0/1629686476 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768018680 con 0x7f2774103720 2023-11-27T15:23:22.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.981+0000 7f277a1ff700 1 -- 172.21.15.8:0/1629686476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277410dd60 con 0x7f2774103720 2023-11-27T15:23:22.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.982+0000 7f277a1ff700 1 -- 172.21.15.8:0/1629686476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27741a0800 con 0x7f2774103720 2023-11-27T15:23:22.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.984+0000 7f2770ff9700 1 -- 172.21.15.8:0/1629686476 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f276800cd40 con 0x7f2774103720 2023-11-27T15:23:22.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.985+0000 7f2770ff9700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f275c0a11d0 0x7f275c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.985+0000 7f2770ff9700 1 -- 172.21.15.8:0/1629686476 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2768014070 con 0x7f2774103720 2023-11-27T15:23:22.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.986+0000 7f27737fe700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f275c0a11d0 0x7f275c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.986+0000 7f27737fe700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f275c0a11d0 0x7f275c0a3680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f276400b2e0 tx=0x7f2764005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:22.989+0000 7f2770ff9700 1 -- 172.21.15.8:0/1629686476 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2768051730 con 0x7f2774103720 2023-11-27T15:23:23.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.224+0000 7f277a1ff700 1 -- 172.21.15.8:0/1629686476 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27741093e0 con 0x7f2774103720 2023-11-27T15:23:23.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.225+0000 7f2770ff9700 1 -- 172.21.15.8:0/1629686476 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18279 (secure 0 0 0) 0x7f2768096230 con 0x7f2774103720 2023-11-27T15:23:23.229 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:23.230 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":80,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:22:36.503805+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:23:23.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 -- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f275c0a11d0 msgr2=0x7f275c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f275c0a11d0 0x7f275c0a3680 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f276400b2e0 tx=0x7f2764005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 -- 172.21.15.8:0/1629686476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2774103720 msgr2=0x7f27741987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2774103720 0x7f27741987b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f276800e9a0 tx=0x7f276800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 -- 172.21.15.8:0/1629686476 shutdown_connections 2023-11-27T15:23:23.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27741040b0 0x7f277419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2774102530 0x7f2774198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f275c0a11d0 0x7f275c0a3680 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 --2- 172.21.15.8:0/1629686476 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2774103720 0x7f27741987b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 -- 172.21.15.8:0/1629686476 >> 172.21.15.8:0/1629686476 conn(0x7f27740fdb30 msgr2=0x7f27740ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:23.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 -- 172.21.15.8:0/1629686476 shutdown_connections 2023-11-27T15:23:23.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.228+0000 7f277a1ff700 1 -- 172.21.15.8:0/1629686476 wait complete. 2023-11-27T15:23:23.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:23:23.254 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:23:23.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.470+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/797293450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d68103680 msgr2=0x7f0d68105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.470+0000 7f0d6c9ff700 1 --2- 172.21.15.8:0/797293450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d68103680 0x7f0d68105a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009990 tx=0x7f0d5c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.470+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/797293450 shutdown_connections 2023-11-27T15:23:23.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.470+0000 7f0d6c9ff700 1 --2- 172.21.15.8:0/797293450 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d68105fa0 0x7f0d68108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.470+0000 7f0d6c9ff700 1 --2- 172.21.15.8:0/797293450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d68100d60 0x7f0d68103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.470+0000 7f0d6c9ff700 1 --2- 172.21.15.8:0/797293450 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d68103680 0x7f0d68105a60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.470+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/797293450 >> 172.21.15.8:0/797293450 conn(0x7f0d680fa820 msgr2=0x7f0d680fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:23.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.470+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/797293450 shutdown_connections 2023-11-27T15:23:23.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.470+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/797293450 wait complete. 2023-11-27T15:23:23.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.472+0000 7f0d6c9ff700 1 Processor -- start 2023-11-27T15:23:23.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.472+0000 7f0d6c9ff700 1 -- start start 2023-11-27T15:23:23.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.472+0000 7f0d6c9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d68100d60 0x7f0d68195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.472+0000 7f0d6c9ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d68103680 0x7f0d681964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.472+0000 7f0d6c9ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d68105fa0 0x7f0d6819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.472+0000 7f0d6c9ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6819dae0 con 0x7f0d68105fa0 2023-11-27T15:23:23.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.472+0000 7f0d6c9ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6819dc20 con 0x7f0d68103680 2023-11-27T15:23:23.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.472+0000 7f0d6c9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6819ddb0 con 0x7f0d68100d60 2023-11-27T15:23:23.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d6659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d68100d60 0x7f0d68195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d6659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d68100d60 0x7f0d68195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:55022/0 (socket says 172.21.15.8:55022) 2023-11-27T15:23:23.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d6659c700 1 -- 172.21.15.8:0/1217009159 learned_addr learned my addr 172.21.15.8:0/1217009159 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:23.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d65d9b700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d68103680 0x7f0d681964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d66d9d700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d68105fa0 0x7f0d6819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d6659c700 1 -- 172.21.15.8:0/1217009159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d68103680 msgr2=0x7f0d681964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d6659c700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d68103680 0x7f0d681964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d6659c700 1 -- 172.21.15.8:0/1217009159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d68105fa0 msgr2=0x7f0d6819d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.478 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d6659c700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d68105fa0 0x7f0d6819d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d6659c700 1 -- 172.21.15.8:0/1217009159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d58009520 con 0x7f0d68100d60 2023-11-27T15:23:23.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.473+0000 7f0d6659c700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d68100d60 0x7f0d68195f60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800eb30 tx=0x7f0d5800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:23.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.474+0000 7f0d577fe700 1 -- 172.21.15.8:0/1217009159 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58009880 con 0x7f0d68100d60 2023-11-27T15:23:23.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.474+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/1217009159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c0095f0 con 0x7f0d68100d60 2023-11-27T15:23:23.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.474+0000 7f0d577fe700 1 -- 172.21.15.8:0/1217009159 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d580044e0 con 0x7f0d68100d60 2023-11-27T15:23:23.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.474+0000 7f0d577fe700 1 -- 172.21.15.8:0/1217009159 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58003d80 con 0x7f0d68100d60 2023-11-27T15:23:23.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.474+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/1217009159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6810bcc0 con 0x7f0d68100d60 2023-11-27T15:23:23.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.476+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/1217009159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6810be50 con 0x7f0d68100d60 2023-11-27T15:23:23.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.477+0000 7f0d577fe700 1 -- 172.21.15.8:0/1217009159 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0d58004020 con 0x7f0d68100d60 2023-11-27T15:23:23.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.480+0000 7f0d577fe700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0d500a11d0 0x7f0d500a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.480+0000 7f0d577fe700 1 -- 172.21.15.8:0/1217009159 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0d58014070 con 0x7f0d68100d60 2023-11-27T15:23:23.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.480+0000 7f0d65d9b700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0d500a11d0 0x7f0d500a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.481+0000 7f0d65d9b700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0d500a11d0 0x7f0d500a3680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00b2e0 tx=0x7f0d5c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:23.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.483+0000 7f0d577fe700 1 -- 172.21.15.8:0/1217009159 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0d580965a0 con 0x7f0d68100d60 2023-11-27T15:23:23.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:23 smithi008 ceph-mon[128788]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:23.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:23 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3413263713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:23.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:23 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1629686476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:23.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:23 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:23.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.712+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/1217009159 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d68061860 con 0x7f0d68100d60 2023-11-27T15:23:23.716 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.713+0000 7f0d577fe700 1 -- 172.21.15.8:0/1217009159 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d58099bc0 con 0x7f0d68100d60 2023-11-27T15:23:23.717 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:23:23.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/1217009159 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0d500a11d0 msgr2=0x7f0d500a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0d500a11d0 0x7f0d500a3680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00b2e0 tx=0x7f0d5c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/1217009159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d68100d60 msgr2=0x7f0d68195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d68100d60 0x7f0d68195f60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800eb30 tx=0x7f0d5800ee40 comp rx=0 tx=0).stop 2023-11-27T15:23:23.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/1217009159 shutdown_connections 2023-11-27T15:23:23.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0d500a11d0 0x7f0d500a3680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0d68100d60 0x7f0d68195f60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0d68105fa0 0x7f0d6819d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 --2- 172.21.15.8:0/1217009159 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0d68103680 0x7f0d681964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/1217009159 >> 172.21.15.8:0/1217009159 conn(0x7f0d680fa820 msgr2=0x7f0d681019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:23.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/1217009159 shutdown_connections 2023-11-27T15:23:23.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.715+0000 7f0d6c9ff700 1 -- 172.21.15.8:0/1217009159 wait complete. 2023-11-27T15:23:23.737 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-27T15:23:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:23 smithi060 ceph-mon[131518]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:23 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3413263713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:23 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1629686476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:23 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:23 smithi029 ceph-mon[131166]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:23 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3413263713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:23 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1629686476' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:23 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:23.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.945+0000 7f0984bff700 1 -- 172.21.15.8:0/378333637 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980073550 msgr2=0x7f09800739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.945+0000 7f0984bff700 1 --2- 172.21.15.8:0/378333637 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980073550 0x7f09800739c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0974009990 tx=0x7f0974009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.945+0000 7f0984bff700 1 -- 172.21.15.8:0/378333637 shutdown_connections 2023-11-27T15:23:23.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.945+0000 7f0984bff700 1 --2- 172.21.15.8:0/378333637 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0980074b30 0x7f0980072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.945+0000 7f0984bff700 1 --2- 172.21.15.8:0/378333637 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0980108190 0x7f098010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.945+0000 7f0984bff700 1 --2- 172.21.15.8:0/378333637 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980073550 0x7f09800739c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.945+0000 7f0984bff700 1 -- 172.21.15.8:0/378333637 >> 172.21.15.8:0/378333637 conn(0x7f09800fb8b0 msgr2=0x7f09800fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:23.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.945+0000 7f0984bff700 1 -- 172.21.15.8:0/378333637 shutdown_connections 2023-11-27T15:23:23.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.946+0000 7f0984bff700 1 -- 172.21.15.8:0/378333637 wait complete. 2023-11-27T15:23:23.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.946+0000 7f0984bff700 1 Processor -- start 2023-11-27T15:23:23.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f0984bff700 1 -- start start 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f0984bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0980073550 0x7f09801982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f0984bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980074b30 0x7f09801987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f0984bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0980108190 0x7f098019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f0984bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098019fe20 con 0x7f0980074b30 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f0984bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098019ff60 con 0x7f0980073550 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f0984bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801a00f0 con 0x7f0980108190 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f097e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0980073550 0x7f09801982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f097e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0980073550 0x7f09801982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:53520/0 (socket says 172.21.15.8:53520) 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f097e59c700 1 -- 172.21.15.8:0/844748107 learned_addr learned my addr 172.21.15.8:0/844748107 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f097dd9b700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980074b30 0x7f09801987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.947+0000 7f097ed9d700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0980108190 0x7f098019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f097e59c700 1 -- 172.21.15.8:0/844748107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0980108190 msgr2=0x7f098019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f097e59c700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0980108190 0x7f098019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f097e59c700 1 -- 172.21.15.8:0/844748107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980074b30 msgr2=0x7f09801987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f097e59c700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980074b30 0x7f09801987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f097e59c700 1 -- 172.21.15.8:0/844748107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09740095f0 con 0x7f0980073550 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f097e59c700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0980073550 0x7f09801982a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f097000e970 tx=0x7f097000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f096f7fe700 1 -- 172.21.15.8:0/844748107 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970009790 con 0x7f0980073550 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f0984bff700 1 -- 172.21.15.8:0/844748107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098010deb0 con 0x7f0980073550 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f096f7fe700 1 -- 172.21.15.8:0/844748107 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f097000eec0 con 0x7f0980073550 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f096f7fe700 1 -- 172.21.15.8:0/844748107 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970005590 con 0x7f0980073550 2023-11-27T15:23:23.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.948+0000 7f0984bff700 1 -- 172.21.15.8:0/844748107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098010dd20 con 0x7f0980073550 2023-11-27T15:23:23.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.949+0000 7f0984bff700 1 -- 172.21.15.8:0/844748107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0980106840 con 0x7f0980073550 2023-11-27T15:23:23.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.951+0000 7f096f7fe700 1 -- 172.21.15.8:0/844748107 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f097000cbd0 con 0x7f0980073550 2023-11-27T15:23:23.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.952+0000 7f096f7fe700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f09680a1150 0x7f09680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.953+0000 7f096f7fe700 1 -- 172.21.15.8:0/844748107 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0970014070 con 0x7f0980073550 2023-11-27T15:23:23.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.953+0000 7f097dd9b700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f09680a1150 0x7f09680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.953+0000 7f097dd9b700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f09680a1150 0x7f09680a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f097400b2e0 tx=0x7f0974005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:23.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:23.957+0000 7f096f7fe700 1 -- 172.21.15.8:0/844748107 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f09700962a0 con 0x7f0980073550 2023-11-27T15:23:24.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.174+0000 7f0984bff700 1 -- 172.21.15.8:0/844748107 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f098002cfe0 con 0x7f0980073550 2023-11-27T15:23:24.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.175+0000 7f096f7fe700 1 -- 172.21.15.8:0/844748107 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0970004ef0 con 0x7f0980073550 2023-11-27T15:23:24.179 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:24.179 INFO:teuthology.orchestra.run.smithi008.stdout:[ 2023-11-27T15:23:24.179 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:23:24.179 INFO:teuthology.orchestra.run.smithi008.stdout: "poolnum": 1, 2023-11-27T15:23:24.179 INFO:teuthology.orchestra.run.smithi008.stdout: "poolname": ".mgr" 2023-11-27T15:23:24.179 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:23:24.179 INFO:teuthology.orchestra.run.smithi008.stdout:] 2023-11-27T15:23:24.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 -- 172.21.15.8:0/844748107 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f09680a1150 msgr2=0x7f09680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f09680a1150 0x7f09680a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f097400b2e0 tx=0x7f0974005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 -- 172.21.15.8:0/844748107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0980073550 msgr2=0x7f09801982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0980073550 0x7f09801982a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f097000e970 tx=0x7f097000ec80 comp rx=0 tx=0).stop 2023-11-27T15:23:24.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 -- 172.21.15.8:0/844748107 shutdown_connections 2023-11-27T15:23:24.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f09680a1150 0x7f09680a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0980108190 0x7f098019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0980073550 0x7f09801982a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 --2- 172.21.15.8:0/844748107 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980074b30 0x7f09801987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 -- 172.21.15.8:0/844748107 >> 172.21.15.8:0/844748107 conn(0x7f09800fb8b0 msgr2=0x7f09800fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 -- 172.21.15.8:0/844748107 shutdown_connections 2023-11-27T15:23:24.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.177+0000 7f0984bff700 1 -- 172.21.15.8:0/844748107 wait complete. 2023-11-27T15:23:24.200 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-27T15:23:24.200 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-27T15:23:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.417+0000 7f6d795ff700 1 -- 172.21.15.8:0/227562749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d74074b30 msgr2=0x7f6d74072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.417+0000 7f6d795ff700 1 --2- 172.21.15.8:0/227562749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d74074b30 0x7f6d74072f80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009990 tx=0x7f6d64009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.417+0000 7f6d795ff700 1 -- 172.21.15.8:0/227562749 shutdown_connections 2023-11-27T15:23:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.417+0000 7f6d795ff700 1 --2- 172.21.15.8:0/227562749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d7410c870 0x7f6d7410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.417+0000 7f6d795ff700 1 --2- 172.21.15.8:0/227562749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d74074b30 0x7f6d74072f80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.417+0000 7f6d795ff700 1 --2- 172.21.15.8:0/227562749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74073550 0x7f6d740739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.417+0000 7f6d795ff700 1 -- 172.21.15.8:0/227562749 >> 172.21.15.8:0/227562749 conn(0x7f6d740fb880 msgr2=0x7f6d740fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.417+0000 7f6d795ff700 1 -- 172.21.15.8:0/227562749 shutdown_connections 2023-11-27T15:23:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.417+0000 7f6d795ff700 1 -- 172.21.15.8:0/227562749 wait complete. 2023-11-27T15:23:24.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.418+0000 7f6d795ff700 1 Processor -- start 2023-11-27T15:23:24.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.418+0000 7f6d795ff700 1 -- start start 2023-11-27T15:23:24.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d795ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74073550 0x7f6d7419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d795ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d74074b30 0x7f6d7419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d795ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d7410c870 0x7f6d741a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d795ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d741a4180 con 0x7f6d74074b30 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d795ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d741a42c0 con 0x7f6d74073550 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d795ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d741a4450 con 0x7f6d7410c870 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d74074b30 0x7f6d7419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d72ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74073550 0x7f6d7419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d727fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d74074b30 0x7f6d7419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49558/0 (socket says 172.21.15.8:49558) 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d727fc700 1 -- 172.21.15.8:0/3268743181 learned_addr learned my addr 172.21.15.8:0/3268743181 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d737fe700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d7410c870 0x7f6d741a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d72ffd700 1 -- 172.21.15.8:0/3268743181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d7410c870 msgr2=0x7f6d741a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.425 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d72ffd700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d7410c870 0x7f6d741a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d72ffd700 1 -- 172.21.15.8:0/3268743181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d74074b30 msgr2=0x7f6d7419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d72ffd700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d74074b30 0x7f6d7419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.419+0000 7f6d72ffd700 1 -- 172.21.15.8:0/3268743181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640095f0 con 0x7f6d74073550 2023-11-27T15:23:24.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.420+0000 7f6d72ffd700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74073550 0x7f6d7419c600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6d640098e0 tx=0x7f6d64009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.420+0000 7f6d5bfff700 1 -- 172.21.15.8:0/3268743181 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6401d070 con 0x7f6d74073550 2023-11-27T15:23:24.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.420+0000 7f6d5bfff700 1 -- 172.21.15.8:0/3268743181 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d64021410 con 0x7f6d74073550 2023-11-27T15:23:24.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.420+0000 7f6d5bfff700 1 -- 172.21.15.8:0/3268743181 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64005720 con 0x7f6d74073550 2023-11-27T15:23:24.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.420+0000 7f6d795ff700 1 -- 172.21.15.8:0/3268743181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d74112260 con 0x7f6d74073550 2023-11-27T15:23:24.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.420+0000 7f6d795ff700 1 -- 172.21.15.8:0/3268743181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d741120d0 con 0x7f6d74073550 2023-11-27T15:23:24.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.421+0000 7f6d795ff700 1 -- 172.21.15.8:0/3268743181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d741023c0 con 0x7f6d74073550 2023-11-27T15:23:24.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.423+0000 7f6d5bfff700 1 -- 172.21.15.8:0/3268743181 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6d64021580 con 0x7f6d74073550 2023-11-27T15:23:24.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.425+0000 7f6d5bfff700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6d5c0a11d0 0x7f6d5c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.425+0000 7f6d5bfff700 1 -- 172.21.15.8:0/3268743181 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6d640cdd70 con 0x7f6d74073550 2023-11-27T15:23:24.429 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.425+0000 7f6d727fc700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6d5c0a11d0 0x7f6d5c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.429 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.425+0000 7f6d727fc700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6d5c0a11d0 0x7f6d5c0a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d68005950 tx=0x7f6d68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.427+0000 7f6d5bfff700 1 -- 172.21.15.8:0/3268743181 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6d64097950 con 0x7f6d74073550 2023-11-27T15:23:24.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:24.645+0000 7f6d795ff700 1 -- 172.21.15.8:0/3268743181 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6d74061700 con 0x7f6d74073550 2023-11-27T15:23:24.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1217009159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:23:24.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/844748107' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-27T15:23:24.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:24.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:23:24.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:24 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1217009159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:23:24.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/844748107' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-27T15:23:24.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:24.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:23:24.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:24 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1217009159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/844748107' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-27T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:24.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:23:24.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:24 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:25.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.572+0000 7f6d5bfff700 1 -- 172.21.15.8:0/3268743181 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6d6409af70 con 0x7f6d74073550 2023-11-27T15:23:25.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.579+0000 7f6d795ff700 1 -- 172.21.15.8:0/3268743181 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6d5c0a11d0 msgr2=0x7f6d5c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.579+0000 7f6d795ff700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6d5c0a11d0 0x7f6d5c0a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6d68005950 tx=0x7f6d68009260 comp rx=0 tx=0).stop 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.579+0000 7f6d795ff700 1 -- 172.21.15.8:0/3268743181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74073550 msgr2=0x7f6d7419c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.579+0000 7f6d795ff700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74073550 0x7f6d7419c600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6d640098e0 tx=0x7f6d64009580 comp rx=0 tx=0).stop 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.580+0000 7f6d795ff700 1 -- 172.21.15.8:0/3268743181 shutdown_connections 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.580+0000 7f6d795ff700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6d5c0a11d0 0x7f6d5c0a3680 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.580+0000 7f6d795ff700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6d7410c870 0x7f6d741a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.580+0000 7f6d795ff700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6d74073550 0x7f6d7419c600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.580+0000 7f6d795ff700 1 --2- 172.21.15.8:0/3268743181 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6d74074b30 0x7f6d7419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.580+0000 7f6d795ff700 1 -- 172.21.15.8:0/3268743181 >> 172.21.15.8:0/3268743181 conn(0x7f6d740fb880 msgr2=0x7f6d740fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.580+0000 7f6d795ff700 1 -- 172.21.15.8:0/3268743181 shutdown_connections 2023-11-27T15:23:25.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.580+0000 7f6d795ff700 1 -- 172.21.15.8:0/3268743181 wait complete. 2023-11-27T15:23:25.585 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_metadata' created 2023-11-27T15:23:25.602 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-27T15:23:25.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.816+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3931559697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd4100290 msgr2=0x7f5fd41006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:25.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.816+0000 7f5fdbdff700 1 --2- 172.21.15.8:0/3931559697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd4100290 0x7f5fd41006a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009990 tx=0x7f5fd0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:25.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.816+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3931559697 shutdown_connections 2023-11-27T15:23:25.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.816+0000 7f5fdbdff700 1 --2- 172.21.15.8:0/3931559697 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fd4101e10 0x7f5fd41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:25.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.816+0000 7f5fdbdff700 1 --2- 172.21.15.8:0/3931559697 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd4100290 0x7f5fd41006a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:25.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.816+0000 7f5fdbdff700 1 --2- 172.21.15.8:0/3931559697 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fd4101480 0x7f5fd41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:25.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.816+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3931559697 >> 172.21.15.8:0/3931559697 conn(0x7f5fd40fb870 msgr2=0x7f5fd40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:25.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.816+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3931559697 shutdown_connections 2023-11-27T15:23:25.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.816+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3931559697 wait complete. 2023-11-27T15:23:25.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.817+0000 7f5fdbdff700 1 Processor -- start 2023-11-27T15:23:25.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.817+0000 7f5fdbdff700 1 -- start start 2023-11-27T15:23:25.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fdbdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fd4100290 0x7f5fd4198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:25.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fdbdff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd4101480 0x7f5fd4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:25.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fdbdff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fd4101e10 0x7f5fd419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:25.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fdbdff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd419fd80 con 0x7f5fd4101e10 2023-11-27T15:23:25.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fdbdff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd419fec0 con 0x7f5fd4101480 2023-11-27T15:23:25.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fdbdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41a0050 con 0x7f5fd4100290 2023-11-27T15:23:25.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd9b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fd4100290 0x7f5fd4198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:25.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd9b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fd4100290 0x7f5fd4198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:55066/0 (socket says 172.21.15.8:55066) 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd9b9b700 1 -- 172.21.15.8:0/3346361327 learned_addr learned my addr 172.21.15.8:0/3346361327 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fda39c700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fd4101e10 0x7f5fd419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd939a700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd4101480 0x7f5fd4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd939a700 1 -- 172.21.15.8:0/3346361327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fd4100290 msgr2=0x7f5fd4198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd939a700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fd4100290 0x7f5fd4198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd939a700 1 -- 172.21.15.8:0/3346361327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fd4101e10 msgr2=0x7f5fd419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd939a700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fd4101e10 0x7f5fd419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd939a700 1 -- 172.21.15.8:0/3346361327 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd00095f0 con 0x7f5fd4101480 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd9b9b700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fd4100290 0x7f5fd4198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.818+0000 7f5fd939a700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd4101480 0x7f5fd4198740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400e9f0 tx=0x7f5fc400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:25.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.819+0000 7f5fcaffd700 1 -- 172.21.15.8:0/3346361327 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc400c9c0 con 0x7f5fd4101480 2023-11-27T15:23:25.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.819+0000 7f5fcaffd700 1 -- 172.21.15.8:0/3346361327 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fc4010430 con 0x7f5fd4101480 2023-11-27T15:23:25.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.819+0000 7f5fcaffd700 1 -- 172.21.15.8:0/3346361327 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4018680 con 0x7f5fd4101480 2023-11-27T15:23:25.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:25 smithi060 ceph-mon[131518]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:25.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:25 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:23:25.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:25 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:23:25.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:25 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:23:25.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:25 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:23:25.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:25 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3268743181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:23:25.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:25 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:23:25.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:25 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:25.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.819+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3346361327 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd410bc40 con 0x7f5fd4101480 2023-11-27T15:23:25.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.819+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3346361327 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd410bab0 con 0x7f5fd4101480 2023-11-27T15:23:25.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.821+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3346361327 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb8005310 con 0x7f5fd4101480 2023-11-27T15:23:25.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.822+0000 7f5fcaffd700 1 -- 172.21.15.8:0/3346361327 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5fc400cb20 con 0x7f5fd4101480 2023-11-27T15:23:25.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.824+0000 7f5fcaffd700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5fc00a11d0 0x7f5fc00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:25.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.824+0000 7f5fcaffd700 1 -- 172.21.15.8:0/3346361327 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f5fc4014070 con 0x7f5fd4101480 2023-11-27T15:23:25.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.824+0000 7f5fd9b9b700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5fc00a11d0 0x7f5fc00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:25.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.825+0000 7f5fd9b9b700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5fc00a11d0 0x7f5fc00a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000b2e0 tx=0x7f5fd0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:25.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:25.827+0000 7f5fcaffd700 1 -- 172.21.15.8:0/3346361327 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5fc4051730 con 0x7f5fd4101480 2023-11-27T15:23:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:25 smithi008 ceph-mon[128788]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:25 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:23:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:25 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:23:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:25 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:23:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:25 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:23:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:25 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3268743181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:23:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:25 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:23:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:25 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:25 smithi029 ceph-mon[131166]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:25 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:23:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:25 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:23:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:25 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:23:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:25 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:23:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:25 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3268743181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:23:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:25 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:23:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:25 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:26.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.056+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3346361327 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f5fb8005180 con 0x7f5fd4101480 2023-11-27T15:23:26.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.593+0000 7f5fcaffd700 1 -- 172.21.15.8:0/3346361327 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7f5fc4099a90 con 0x7f5fd4101480 2023-11-27T15:23:26.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.606+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3346361327 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5fc00a11d0 msgr2=0x7f5fc00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:26.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.606+0000 7f5fdbdff700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5fc00a11d0 0x7f5fc00a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000b2e0 tx=0x7f5fd0005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:26.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.607+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3346361327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd4101480 msgr2=0x7f5fd4198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:26.612 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.607+0000 7f5fdbdff700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd4101480 0x7f5fd4198740 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400e9f0 tx=0x7f5fc400ed00 comp rx=0 tx=0).stop 2023-11-27T15:23:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.608+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3346361327 shutdown_connections 2023-11-27T15:23:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.608+0000 7f5fdbdff700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5fc00a11d0 0x7f5fc00a3680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.608+0000 7f5fdbdff700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5fd4100290 0x7f5fd4198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.608+0000 7f5fdbdff700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fd4101e10 0x7f5fd419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.608+0000 7f5fdbdff700 1 --2- 172.21.15.8:0/3346361327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5fd4101480 0x7f5fd4198740 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.608+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3346361327 >> 172.21.15.8:0/3346361327 conn(0x7f5fd40fb870 msgr2=0x7f5fd40fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:26.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.609+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3346361327 shutdown_connections 2023-11-27T15:23:26.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.609+0000 7f5fdbdff700 1 -- 172.21.15.8:0/3346361327 wait complete. 2023-11-27T15:23:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_data' created 2023-11-27T15:23:26.638 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-27T15:23:26.855 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.850+0000 7fdacebff700 1 -- 172.21.15.8:0/2039331092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdac8101480 msgr2=0x7fdac81018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:26.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.850+0000 7fdacebff700 1 --2- 172.21.15.8:0/2039331092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdac8101480 0x7fdac81018d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0098f0 tx=0x7fdabc009c00 comp rx=0 tx=0).stop 2023-11-27T15:23:26.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.850+0000 7fdacebff700 1 -- 172.21.15.8:0/2039331092 shutdown_connections 2023-11-27T15:23:26.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.850+0000 7fdacebff700 1 --2- 172.21.15.8:0/2039331092 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac8101e10 0x7fdac81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:26.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.850+0000 7fdacebff700 1 --2- 172.21.15.8:0/2039331092 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac8100290 0x7fdac81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:26.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.850+0000 7fdacebff700 1 --2- 172.21.15.8:0/2039331092 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdac8101480 0x7fdac81018d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:26.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.850+0000 7fdacebff700 1 -- 172.21.15.8:0/2039331092 >> 172.21.15.8:0/2039331092 conn(0x7fdac80fb870 msgr2=0x7fdac80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:26.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.851+0000 7fdacebff700 1 -- 172.21.15.8:0/2039331092 shutdown_connections 2023-11-27T15:23:26.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.851+0000 7fdacebff700 1 -- 172.21.15.8:0/2039331092 wait complete. 2023-11-27T15:23:26.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacebff700 1 Processor -- start 2023-11-27T15:23:26.858 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacebff700 1 -- start start 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacebff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac8100290 0x7fdac81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacebff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdac8101480 0x7fdac8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacebff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac8101e10 0x7fdac819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacebff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac819fd70 con 0x7fdac8101480 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacebff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac819feb0 con 0x7fdac8100290 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacebff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac81a0040 con 0x7fdac8101e10 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacc99b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac8100290 0x7fdac81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdac7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdac8101480 0x7fdac8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacc99b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac8100290 0x7fdac81981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:53576/0 (socket says 172.21.15.8:53576) 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.852+0000 7fdacc99b700 1 -- 172.21.15.8:0/2694092192 learned_addr learned my addr 172.21.15.8:0/2694092192 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdacd19c700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac8101e10 0x7fdac819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdacc99b700 1 -- 172.21.15.8:0/2694092192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac8101e10 msgr2=0x7fdac819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdacc99b700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac8101e10 0x7fdac819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:26.859 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdacc99b700 1 -- 172.21.15.8:0/2694092192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdac8101480 msgr2=0x7fdac8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:26.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdacc99b700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdac8101480 0x7fdac8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:26.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdacc99b700 1 -- 172.21.15.8:0/2694092192 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc009520 con 0x7fdac8100290 2023-11-27T15:23:26.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdacc99b700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac8100290 0x7fdac81981f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdab400e9f0 tx=0x7fdab400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:26.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdac5ffb700 1 -- 172.21.15.8:0/2694092192 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4004220 con 0x7fdac8100290 2023-11-27T15:23:26.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdac5ffb700 1 -- 172.21.15.8:0/2694092192 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdab4004380 con 0x7fdac8100290 2023-11-27T15:23:26.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdacebff700 1 -- 172.21.15.8:0/2694092192 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac810bc00 con 0x7fdac8100290 2023-11-27T15:23:26.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdac5ffb700 1 -- 172.21.15.8:0/2694092192 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4003c00 con 0x7fdac8100290 2023-11-27T15:23:26.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.853+0000 7fdacebff700 1 -- 172.21.15.8:0/2694092192 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac810ba70 con 0x7fdac8100290 2023-11-27T15:23:26.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.854+0000 7fdacebff700 1 -- 172.21.15.8:0/2694092192 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac81a07e0 con 0x7fdac8100290 2023-11-27T15:23:26.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.856+0000 7fdac5ffb700 1 -- 172.21.15.8:0/2694092192 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdab400c7c0 con 0x7fdac8100290 2023-11-27T15:23:26.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.858+0000 7fdac5ffb700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdab80a11d0 0x7fdab80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:26.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.858+0000 7fdac5ffb700 1 -- 172.21.15.8:0/2694092192 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fdab4014070 con 0x7fdac8100290 2023-11-27T15:23:26.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.858+0000 7fdac7fff700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdab80a11d0 0x7fdab80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:26.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:26 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-27T15:23:26.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:26 smithi008 ceph-mon[128788]: osdmap e81: 12 total, 12 up, 12 in 2023-11-27T15:23:26.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.859+0000 7fdac7fff700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdab80a11d0 0x7fdab80a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00f9e0 tx=0x7fdabc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:26.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:26.861+0000 7fdac5ffb700 1 -- 172.21.15.8:0/2694092192 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdab4096380 con 0x7fdac8100290 2023-11-27T15:23:26.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:26 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-27T15:23:26.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:26 smithi029 ceph-mon[131166]: osdmap e81: 12 total, 12 up, 12 in 2023-11-27T15:23:27.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:26 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-27T15:23:27.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:26 smithi060 ceph-mon[131518]: osdmap e81: 12 total, 12 up, 12 in 2023-11-27T15:23:27.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.164+0000 7fdacebff700 1 -- 172.21.15.8:0/2694092192 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdac81a0d90 con 0x7fdac8100290 2023-11-27T15:23:27.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.173+0000 7fdac5ffb700 1 -- 172.21.15.8:0/2694092192 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdab40999a0 con 0x7fdac8100290 2023-11-27T15:23:27.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.177+0000 7fdacebff700 1 -- 172.21.15.8:0/2694092192 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdab80a11d0 msgr2=0x7fdab80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:27.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.177+0000 7fdacebff700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdab80a11d0 0x7fdab80a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdabc00f9e0 tx=0x7fdabc019040 comp rx=0 tx=0).stop 2023-11-27T15:23:27.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.177+0000 7fdacebff700 1 -- 172.21.15.8:0/2694092192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac8100290 msgr2=0x7fdac81981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:27.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.177+0000 7fdacebff700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac8100290 0x7fdac81981f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdab400e9f0 tx=0x7fdab400ed00 comp rx=0 tx=0).stop 2023-11-27T15:23:27.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.178+0000 7fdacebff700 1 -- 172.21.15.8:0/2694092192 shutdown_connections 2023-11-27T15:23:27.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.178+0000 7fdacebff700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdab80a11d0 0x7fdab80a3680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:27.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.178+0000 7fdacebff700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdac8101e10 0x7fdac819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:27.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.178+0000 7fdacebff700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdac8100290 0x7fdac81981f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:27.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.178+0000 7fdacebff700 1 --2- 172.21.15.8:0/2694092192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdac8101480 0x7fdac8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:27.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.178+0000 7fdacebff700 1 -- 172.21.15.8:0/2694092192 >> 172.21.15.8:0/2694092192 conn(0x7fdac80fb870 msgr2=0x7fdac80fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:27.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.179+0000 7fdacebff700 1 -- 172.21.15.8:0/2694092192 shutdown_connections 2023-11-27T15:23:27.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.180+0000 7fdacebff700 1 -- 172.21.15.8:0/2694092192 wait complete. 2023-11-27T15:23:27.187 INFO:teuthology.orchestra.run.smithi008.stderr:new fs with metadata pool 2 and data pool 3 2023-11-27T15:23:27.220 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-27T15:23:27.220 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-27T15:23:27.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.438+0000 7ff3c95ff700 1 -- 172.21.15.8:0/2316257259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3c41036e0 msgr2=0x7ff3c4105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:27.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.438+0000 7ff3c95ff700 1 --2- 172.21.15.8:0/2316257259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3c41036e0 0x7ff3c4105ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009990 tx=0x7ff3b8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:27.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.438+0000 7ff3c95ff700 1 -- 172.21.15.8:0/2316257259 shutdown_connections 2023-11-27T15:23:27.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.438+0000 7ff3c95ff700 1 --2- 172.21.15.8:0/2316257259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff3c4106000 0x7ff3c4108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:27.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.438+0000 7ff3c95ff700 1 --2- 172.21.15.8:0/2316257259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3c4100dc0 0x7ff3c41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:27.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.438+0000 7ff3c95ff700 1 --2- 172.21.15.8:0/2316257259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3c41036e0 0x7ff3c4105ac0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:27.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.438+0000 7ff3c95ff700 1 -- 172.21.15.8:0/2316257259 >> 172.21.15.8:0/2316257259 conn(0x7ff3c40fa800 msgr2=0x7ff3c40fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:27.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.438+0000 7ff3c95ff700 1 -- 172.21.15.8:0/2316257259 shutdown_connections 2023-11-27T15:23:27.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.438+0000 7ff3c95ff700 1 -- 172.21.15.8:0/2316257259 wait complete. 2023-11-27T15:23:27.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.439+0000 7ff3c95ff700 1 Processor -- start 2023-11-27T15:23:27.445 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.439+0000 7ff3c95ff700 1 -- start start 2023-11-27T15:23:27.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.439+0000 7ff3c95ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3c4100dc0 0x7ff3c4100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:27.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.439+0000 7ff3c95ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff3c41036e0 0x7ff3c40fed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:27.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.439+0000 7ff3c95ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3c4106000 0x7ff3c40ff2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:27.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.439+0000 7ff3c95ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40ffa50 con 0x7ff3c41036e0 2023-11-27T15:23:27.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.439+0000 7ff3c95ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40ffbc0 con 0x7ff3c4100dc0 2023-11-27T15:23:27.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.439+0000 7ff3c95ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c419ffd0 con 0x7ff3c4106000 2023-11-27T15:23:27.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c2ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3c4100dc0 0x7ff3c4100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:27.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c2ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3c4100dc0 0x7ff3c4100730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:53592/0 (socket says 172.21.15.8:53592) 2023-11-27T15:23:27.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3c4106000 0x7ff3c40ff2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:27.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c2ffd700 1 -- 172.21.15.8:0/4069062800 learned_addr learned my addr 172.21.15.8:0/4069062800 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c27fc700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff3c41036e0 0x7ff3c40fed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c2ffd700 1 -- 172.21.15.8:0/4069062800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3c4106000 msgr2=0x7ff3c40ff2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c2ffd700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3c4106000 0x7ff3c40ff2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c2ffd700 1 -- 172.21.15.8:0/4069062800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff3c41036e0 msgr2=0x7ff3c40fed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c2ffd700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff3c41036e0 0x7ff3c40fed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c2ffd700 1 -- 172.21.15.8:0/4069062800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b80095f0 con 0x7ff3c4100dc0 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c37fe700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3c4106000 0x7ff3c40ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.440+0000 7ff3c2ffd700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3c4100dc0 0x7ff3c4100730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400e9a0 tx=0x7ff3b400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.441+0000 7ff3abfff700 1 -- 172.21.15.8:0/4069062800 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4009790 con 0x7ff3c4100dc0 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.441+0000 7ff3abfff700 1 -- 172.21.15.8:0/4069062800 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3b4010430 con 0x7ff3c4100dc0 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.441+0000 7ff3c95ff700 1 -- 172.21.15.8:0/4069062800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c410bcf0 con 0x7ff3c4100dc0 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.441+0000 7ff3abfff700 1 -- 172.21.15.8:0/4069062800 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4018680 con 0x7ff3c4100dc0 2023-11-27T15:23:27.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.441+0000 7ff3c95ff700 1 -- 172.21.15.8:0/4069062800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c410bb60 con 0x7ff3c4100dc0 2023-11-27T15:23:27.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.442+0000 7ff3c95ff700 1 -- 172.21.15.8:0/4069062800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c40fc370 con 0x7ff3c4100dc0 2023-11-27T15:23:27.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.445+0000 7ff3abfff700 1 -- 172.21.15.8:0/4069062800 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff3b40105a0 con 0x7ff3c4100dc0 2023-11-27T15:23:27.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.447+0000 7ff3abfff700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff3ac0a1150 0x7ff3ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:27.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.447+0000 7ff3abfff700 1 -- 172.21.15.8:0/4069062800 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3b4014070 con 0x7ff3c4100dc0 2023-11-27T15:23:27.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.447+0000 7ff3c27fc700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff3ac0a1150 0x7ff3ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:27.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.448+0000 7ff3abfff700 1 -- 172.21.15.8:0/4069062800 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff3b4099380 con 0x7ff3c4100dc0 2023-11-27T15:23:27.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.448+0000 7ff3c27fc700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff3ac0a1150 0x7ff3ac0a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800b2e0 tx=0x7ff3b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:27.711 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:27.706+0000 7ff3c95ff700 1 -- 172.21.15.8:0/4069062800 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7ff3c4061700 con 0x7ff3c4100dc0 2023-11-27T15:23:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3346361327' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:23:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:23:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: pgmap v275: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-27T15:23:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: osdmap e82: 12 total, 12 up, 12 in 2023-11-27T15:23:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2694092192' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:23:27.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:23:27.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: daemon mds.a assigned to filesystem cephfs as rank 0 2023-11-27T15:23:27.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: osdmap e83: 12 total, 12 up, 12 in 2023-11-27T15:23:27.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-27T15:23:27.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2023-11-27T15:23:27.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:27 smithi008 ceph-mon[128788]: daemon mds.a is now active in filesystem cephfs as rank 0 2023-11-27T15:23:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3346361327' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:23:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:23:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: pgmap v275: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-27T15:23:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: osdmap e82: 12 total, 12 up, 12 in 2023-11-27T15:23:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2694092192' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:23:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:23:27.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: daemon mds.a assigned to filesystem cephfs as rank 0 2023-11-27T15:23:27.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: osdmap e83: 12 total, 12 up, 12 in 2023-11-27T15:23:27.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-27T15:23:27.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2023-11-27T15:23:27.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:27 smithi029 ceph-mon[131166]: daemon mds.a is now active in filesystem cephfs as rank 0 2023-11-27T15:23:28.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3346361327' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:23:28.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:23:28.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: pgmap v275: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:28.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-27T15:23:28.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: osdmap e82: 12 total, 12 up, 12 in 2023-11-27T15:23:28.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2694092192' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:23:28.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:23:28.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: daemon mds.a assigned to filesystem cephfs as rank 0 2023-11-27T15:23:28.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: osdmap e83: 12 total, 12 up, 12 in 2023-11-27T15:23:28.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-27T15:23:28.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: fsmap cephfs:1 {0=a=up:creating} 11 up:standby 2023-11-27T15:23:28.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:27 smithi060 ceph-mon[131518]: daemon mds.a is now active in filesystem cephfs as rank 0 2023-11-27T15:23:28.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.176+0000 7ff3abfff700 1 -- 172.21.15.8:0/4069062800 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7ff3b4099610 con 0x7ff3c4100dc0 2023-11-27T15:23:28.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 -- 172.21.15.8:0/4069062800 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff3ac0a1150 msgr2=0x7ff3ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:28.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff3ac0a1150 0x7ff3ac0a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800b2e0 tx=0x7ff3b8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 -- 172.21.15.8:0/4069062800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3c4100dc0 msgr2=0x7ff3c4100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:28.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3c4100dc0 0x7ff3c4100730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400e9a0 tx=0x7ff3b400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 -- 172.21.15.8:0/4069062800 shutdown_connections 2023-11-27T15:23:28.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff3ac0a1150 0x7ff3ac0a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff3c4106000 0x7ff3c40ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff3c4100dc0 0x7ff3c4100730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 --2- 172.21.15.8:0/4069062800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff3c41036e0 0x7ff3c40fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 -- 172.21.15.8:0/4069062800 >> 172.21.15.8:0/4069062800 conn(0x7ff3c40fa800 msgr2=0x7ff3c4101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:28.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 -- 172.21.15.8:0/4069062800 shutdown_connections 2023-11-27T15:23:28.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.178+0000 7ff3c95ff700 1 -- 172.21.15.8:0/4069062800 wait complete. 2023-11-27T15:23:28.202 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-27T15:23:28.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.427+0000 7f5aca5ff700 1 -- 172.21.15.8:0/2905656560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ac41040d0 msgr2=0x7f5ac410a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:28.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.427+0000 7f5aca5ff700 1 --2- 172.21.15.8:0/2905656560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ac41040d0 0x7f5ac410a990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00b0a0 tx=0x7f5abc00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.427+0000 7f5aca5ff700 1 -- 172.21.15.8:0/2905656560 shutdown_connections 2023-11-27T15:23:28.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.427+0000 7f5aca5ff700 1 --2- 172.21.15.8:0/2905656560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ac41040d0 0x7f5ac410a990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.427+0000 7f5aca5ff700 1 --2- 172.21.15.8:0/2905656560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ac4102550 0x7f5ac4102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.427+0000 7f5aca5ff700 1 --2- 172.21.15.8:0/2905656560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ac4103740 0x7f5ac4103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.432 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.427+0000 7f5aca5ff700 1 -- 172.21.15.8:0/2905656560 >> 172.21.15.8:0/2905656560 conn(0x7f5ac40fdb30 msgr2=0x7f5ac40fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:28.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.427+0000 7f5aca5ff700 1 -- 172.21.15.8:0/2905656560 shutdown_connections 2023-11-27T15:23:28.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.427+0000 7f5aca5ff700 1 -- 172.21.15.8:0/2905656560 wait complete. 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.428+0000 7f5aca5ff700 1 Processor -- start 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.428+0000 7f5aca5ff700 1 -- start start 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.428+0000 7f5aca5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ac4102550 0x7f5ac41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.428+0000 7f5aca5ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ac4103740 0x7f5ac41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.428+0000 7f5aca5ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ac41040d0 0x7f5ac419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.428+0000 7f5aca5ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac419fe20 con 0x7f5ac4103740 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.428+0000 7f5aca5ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac419ff60 con 0x7f5ac41040d0 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.428+0000 7f5aca5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac41a00f0 con 0x7f5ac4102550 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ac4103740 0x7f5ac41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ac4103740 0x7f5ac41987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49632/0 (socket says 172.21.15.8:49632) 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac37fe700 1 -- 172.21.15.8:0/3739725108 learned_addr learned my addr 172.21.15.8:0/3739725108 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:28.435 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac3fff700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ac4102550 0x7f5ac41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac8b9c700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ac41040d0 0x7f5ac419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac37fe700 1 -- 172.21.15.8:0/3739725108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ac4102550 msgr2=0x7f5ac41982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac37fe700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ac4102550 0x7f5ac41982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac37fe700 1 -- 172.21.15.8:0/3739725108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ac41040d0 msgr2=0x7f5ac419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac37fe700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ac41040d0 0x7f5ac419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac37fe700 1 -- 172.21.15.8:0/3739725108 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc009d00 con 0x7f5ac4103740 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac37fe700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ac4103740 0x7f5ac41987e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800d980 tx=0x7f5ab800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.429+0000 7f5ac17fa700 1 -- 172.21.15.8:0/3739725108 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8004220 con 0x7f5ac4103740 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.430+0000 7f5ac17fa700 1 -- 172.21.15.8:0/3739725108 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ab8007640 con 0x7f5ac4103740 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.430+0000 7f5aca5ff700 1 -- 172.21.15.8:0/3739725108 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac410deb0 con 0x7f5ac4103740 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.430+0000 7f5ac17fa700 1 -- 172.21.15.8:0/3739725108 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8005690 con 0x7f5ac4103740 2023-11-27T15:23:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.430+0000 7f5aca5ff700 1 -- 172.21.15.8:0/3739725108 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac410dd20 con 0x7f5ac4103740 2023-11-27T15:23:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.431+0000 7f5aca5ff700 1 -- 172.21.15.8:0/3739725108 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac41a0830 con 0x7f5ac4103740 2023-11-27T15:23:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.433+0000 7f5ac17fa700 1 -- 172.21.15.8:0/3739725108 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5ab8004750 con 0x7f5ac4103740 2023-11-27T15:23:28.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.434+0000 7f5ac17fa700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5aac0a11d0 0x7f5aac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:28.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.434+0000 7f5ac17fa700 1 -- 172.21.15.8:0/3739725108 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f5ab80ccbb0 con 0x7f5ac4103740 2023-11-27T15:23:28.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.434+0000 7f5ac3fff700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5aac0a11d0 0x7f5aac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:28.439 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.435+0000 7f5ac3fff700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5aac0a11d0 0x7f5aac0a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ac41035a0 tx=0x7f5ab4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:28.441 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.437+0000 7f5ac17fa700 1 -- 172.21.15.8:0/3739725108 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5ab80962c0 con 0x7f5ac4103740 2023-11-27T15:23:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:28.660+0000 7f5aca5ff700 1 -- 172.21.15.8:0/3739725108 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5ac41093a0 con 0x7f5ac4103740 2023-11-27T15:23:28.863 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-a[163286]: 2023-11-27T15:23:28.764+0000 7f4f5d3c2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4069062800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] up:active 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: osdmap e84: 12 total, 12 up, 12 in 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:28.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:28 smithi008 ceph-mon[128788]: pgmap v279: 97 pgs: 15 creating+peering, 25 active+clean, 57 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:28.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/4069062800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:23:28.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:23:28.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:28.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:28.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:28.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:28.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] up:active 2023-11-27T15:23:28.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:28.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-27T15:23:28.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: osdmap e84: 12 total, 12 up, 12 in 2023-11-27T15:23:28.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:28.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:28 smithi029 ceph-mon[131166]: pgmap v279: 97 pgs: 15 creating+peering, 25 active+clean, 57 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:29.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/4069062800' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] up:active 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: osdmap e84: 12 total, 12 up, 12 in 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:29.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:28 smithi060 ceph-mon[131518]: pgmap v279: 97 pgs: 15 creating+peering, 25 active+clean, 57 unknown; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:23:29.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:29 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3739725108' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:23:29.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:29 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:29.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:29 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:29.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:29 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:29.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:29 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:23:29.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:29 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:23:29.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:29 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:29.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:29 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:23:29.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:29 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:23:29.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:29 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:23:29.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:29 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3739725108' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:23:29.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:29 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:29.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:29 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:29.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:29 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:29.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:29 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:23:29.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:29 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:23:29.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:29 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:29.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:29 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:23:29.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:29 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:23:29.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:29 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:23:30.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:29 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3739725108' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:23:30.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:29 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:30.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:29 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:30.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:29 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:23:30.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:29 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:23:30.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:29 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:23:30.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:29 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:23:30.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:29 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:23:30.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:29 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:23:30.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:29 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:23:30.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.621+0000 7f5ac17fa700 1 -- 172.21.15.8:0/3739725108 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7f5ab8016070 con 0x7f5ac4103740 2023-11-27T15:23:30.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.627+0000 7f5aca5ff700 1 -- 172.21.15.8:0/3739725108 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5aac0a11d0 msgr2=0x7f5aac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.627+0000 7f5aca5ff700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5aac0a11d0 0x7f5aac0a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5ac41035a0 tx=0x7f5ab4009260 comp rx=0 tx=0).stop 2023-11-27T15:23:30.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.627+0000 7f5aca5ff700 1 -- 172.21.15.8:0/3739725108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ac4103740 msgr2=0x7f5ac41987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.627+0000 7f5aca5ff700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ac4103740 0x7f5ac41987e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800d980 tx=0x7f5ab800dc90 comp rx=0 tx=0).stop 2023-11-27T15:23:30.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.628+0000 7f5aca5ff700 1 -- 172.21.15.8:0/3739725108 shutdown_connections 2023-11-27T15:23:30.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.628+0000 7f5aca5ff700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5aac0a11d0 0x7f5aac0a3680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.628+0000 7f5aca5ff700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ac4102550 0x7f5ac41982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.628+0000 7f5aca5ff700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ac41040d0 0x7f5ac419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.628+0000 7f5aca5ff700 1 --2- 172.21.15.8:0/3739725108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ac4103740 0x7f5ac41987e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.628+0000 7f5aca5ff700 1 -- 172.21.15.8:0/3739725108 >> 172.21.15.8:0/3739725108 conn(0x7f5ac40fdb30 msgr2=0x7f5ac40ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:30.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.628+0000 7f5aca5ff700 1 -- 172.21.15.8:0/3739725108 shutdown_connections 2023-11-27T15:23:30.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.628+0000 7f5aca5ff700 1 -- 172.21.15.8:0/3739725108 wait complete. 2023-11-27T15:23:30.636 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_data_ec' created 2023-11-27T15:23:30.655 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-27T15:23:30.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.859+0000 7f349a9ff700 1 -- 172.21.15.8:0/2580056968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3494100290 msgr2=0x7f34941006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.859+0000 7f349a9ff700 1 --2- 172.21.15.8:0/2580056968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3494100290 0x7f34941006a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3484009990 tx=0x7f3484009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.859+0000 7f349a9ff700 1 -- 172.21.15.8:0/2580056968 shutdown_connections 2023-11-27T15:23:30.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.859+0000 7f349a9ff700 1 --2- 172.21.15.8:0/2580056968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3494101e10 0x7f34941086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.859+0000 7f349a9ff700 1 --2- 172.21.15.8:0/2580056968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3494100290 0x7f34941006a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.859+0000 7f349a9ff700 1 --2- 172.21.15.8:0/2580056968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3494101480 0x7f34941018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.859+0000 7f349a9ff700 1 -- 172.21.15.8:0/2580056968 >> 172.21.15.8:0/2580056968 conn(0x7f34940fb870 msgr2=0x7f34940fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:30.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.859+0000 7f349a9ff700 1 -- 172.21.15.8:0/2580056968 shutdown_connections 2023-11-27T15:23:30.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.859+0000 7f349a9ff700 1 -- 172.21.15.8:0/2580056968 wait complete. 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.860+0000 7f349a9ff700 1 Processor -- start 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.860+0000 7f349a9ff700 1 -- start start 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f349a9ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3494100290 0x7f34940748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f349a9ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3494101480 0x7f3494072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f349a9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3494101e10 0x7f3494073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f349a9ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494073bc0 con 0x7f3494101480 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f349a9ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494073d30 con 0x7f3494100290 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f349a9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34941a00f0 con 0x7f3494101e10 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f34937fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3494101480 0x7f3494072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f34937fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3494101480 0x7f3494072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37192/0 (socket says 172.21.15.8:37192) 2023-11-27T15:23:30.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f34937fe700 1 -- 172.21.15.8:0/18696041 learned_addr learned my addr 172.21.15.8:0/18696041 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f3493fff700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3494100290 0x7f34940748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f3498f9c700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3494101e10 0x7f3494073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f34937fe700 1 -- 172.21.15.8:0/18696041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3494101e10 msgr2=0x7f3494073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f34937fe700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3494101e10 0x7f3494073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f34937fe700 1 -- 172.21.15.8:0/18696041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3494100290 msgr2=0x7f34940748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f34937fe700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3494100290 0x7f34940748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.861+0000 7f34937fe700 1 -- 172.21.15.8:0/18696041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c009c40 con 0x7f3494101480 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.862+0000 7f34937fe700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3494101480 0x7f3494072ee0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f348800e9a0 tx=0x7f348800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.862+0000 7f34917fa700 1 -- 172.21.15.8:0/18696041 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488009790 con 0x7f3494101480 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.862+0000 7f34917fa700 1 -- 172.21.15.8:0/18696041 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f348800ce80 con 0x7f3494101480 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.862+0000 7f349a9ff700 1 -- 172.21.15.8:0/18696041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34840095f0 con 0x7f3494101480 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.862+0000 7f34917fa700 1 -- 172.21.15.8:0/18696041 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488018680 con 0x7f3494101480 2023-11-27T15:23:30.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.862+0000 7f349a9ff700 1 -- 172.21.15.8:0/18696041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349410bc40 con 0x7f3494101480 2023-11-27T15:23:30.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.863+0000 7f349a9ff700 1 -- 172.21.15.8:0/18696041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3494067a50 con 0x7f3494101480 2023-11-27T15:23:30.871 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.867+0000 7f34917fa700 1 -- 172.21.15.8:0/18696041 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3488010430 con 0x7f3494101480 2023-11-27T15:23:30.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.868+0000 7f34917fa700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f347c0a11d0 0x7f347c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.868+0000 7f34917fa700 1 -- 172.21.15.8:0/18696041 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f3488014070 con 0x7f3494101480 2023-11-27T15:23:30.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.868+0000 7f3493fff700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f347c0a11d0 0x7f347c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.869+0000 7f3493fff700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f347c0a11d0 0x7f347c0a3680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f348400b2e0 tx=0x7f3484005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:30.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:30.871+0000 7f34917fa700 1 -- 172.21.15.8:0/18696041 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3488096570 con 0x7f3494101480 2023-11-27T15:23:30.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:30 smithi029 ceph-mon[131166]: osdmap e85: 12 total, 12 up, 12 in 2023-11-27T15:23:30.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:30 smithi029 ceph-mon[131166]: pgmap v281: 97 pgs: 15 creating+peering, 58 active+clean, 24 unknown; 578 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 7 op/s 2023-11-27T15:23:31.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:30 smithi060 ceph-mon[131518]: osdmap e85: 12 total, 12 up, 12 in 2023-11-27T15:23:31.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:30 smithi060 ceph-mon[131518]: pgmap v281: 97 pgs: 15 creating+peering, 58 active+clean, 24 unknown; 578 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 7 op/s 2023-11-27T15:23:31.084 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.080+0000 7f349a9ff700 1 -- 172.21.15.8:0/18696041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f34941a0ee0 con 0x7f3494101480 2023-11-27T15:23:31.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:30 smithi008 ceph-mon[128788]: osdmap e85: 12 total, 12 up, 12 in 2023-11-27T15:23:31.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:30 smithi008 ceph-mon[128788]: pgmap v281: 97 pgs: 15 creating+peering, 58 active+clean, 24 unknown; 578 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 7 op/s 2023-11-27T15:23:31.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.622+0000 7f34917fa700 1 -- 172.21.15.8:0/18696041 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f34880260c0 con 0x7f3494101480 2023-11-27T15:23:31.635 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.630+0000 7f349a9ff700 1 -- 172.21.15.8:0/18696041 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f347c0a11d0 msgr2=0x7f347c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:31.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.630+0000 7f349a9ff700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f347c0a11d0 0x7f347c0a3680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f348400b2e0 tx=0x7f3484005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.630+0000 7f349a9ff700 1 -- 172.21.15.8:0/18696041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3494101480 msgr2=0x7f3494072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:31.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.630+0000 7f349a9ff700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3494101480 0x7f3494072ee0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f348800e9a0 tx=0x7f348800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.631+0000 7f349a9ff700 1 -- 172.21.15.8:0/18696041 shutdown_connections 2023-11-27T15:23:31.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.631+0000 7f349a9ff700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f347c0a11d0 0x7f347c0a3680 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.631+0000 7f349a9ff700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3494101e10 0x7f3494073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.631+0000 7f349a9ff700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3494100290 0x7f34940748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.631+0000 7f349a9ff700 1 --2- 172.21.15.8:0/18696041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3494101480 0x7f3494072ee0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.631+0000 7f349a9ff700 1 -- 172.21.15.8:0/18696041 >> 172.21.15.8:0/18696041 conn(0x7f34940fb870 msgr2=0x7f34940fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:31.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.631+0000 7f349a9ff700 1 -- 172.21.15.8:0/18696041 shutdown_connections 2023-11-27T15:23:31.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.632+0000 7f349a9ff700 1 -- 172.21.15.8:0/18696041 wait complete. 2023-11-27T15:23:31.639 INFO:teuthology.orchestra.run.smithi008.stderr:set pool 4 allow_ec_overwrites to true 2023-11-27T15:23:31.669 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-27T15:23:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.870+0000 7f9d22bff700 1 -- 172.21.15.8:0/1443885825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d1c100290 msgr2=0x7f9d1c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.870+0000 7f9d22bff700 1 --2- 172.21.15.8:0/1443885825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d1c100290 0x7f9d1c1006a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009990 tx=0x7f9d10009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.870+0000 7f9d22bff700 1 -- 172.21.15.8:0/1443885825 shutdown_connections 2023-11-27T15:23:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.870+0000 7f9d22bff700 1 --2- 172.21.15.8:0/1443885825 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d1c101e10 0x7f9d1c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.870+0000 7f9d22bff700 1 --2- 172.21.15.8:0/1443885825 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d1c100290 0x7f9d1c1006a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.870+0000 7f9d22bff700 1 --2- 172.21.15.8:0/1443885825 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c101480 0x7f9d1c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.870+0000 7f9d22bff700 1 -- 172.21.15.8:0/1443885825 >> 172.21.15.8:0/1443885825 conn(0x7f9d1c0fb870 msgr2=0x7f9d1c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.871+0000 7f9d22bff700 1 -- 172.21.15.8:0/1443885825 shutdown_connections 2023-11-27T15:23:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.871+0000 7f9d22bff700 1 -- 172.21.15.8:0/1443885825 wait complete. 2023-11-27T15:23:31.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.871+0000 7f9d22bff700 1 Processor -- start 2023-11-27T15:23:31.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d22bff700 1 -- start start 2023-11-27T15:23:31.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d22bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d1c100290 0x7f9d1c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:31.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d22bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d1c101480 0x7f9d1c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:31.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d22bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c101e10 0x7f9d1c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:31.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d22bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c073bc0 con 0x7f9d1c100290 2023-11-27T15:23:31.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d22bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c073d30 con 0x7f9d1c101e10 2023-11-27T15:23:31.878 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d22bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1a0060 con 0x7f9d1c101480 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d2099b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d1c100290 0x7f9d1c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d2099b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d1c100290 0x7f9d1c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37220/0 (socket says 172.21.15.8:37220) 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d2099b700 1 -- 172.21.15.8:0/930916663 learned_addr learned my addr 172.21.15.8:0/930916663 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d2119c700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c101e10 0x7f9d1c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.872+0000 7f9d1bfff700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d1c101480 0x7f9d1c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d2099b700 1 -- 172.21.15.8:0/930916663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d1c101480 msgr2=0x7f9d1c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d2099b700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d1c101480 0x7f9d1c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d2099b700 1 -- 172.21.15.8:0/930916663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c101e10 msgr2=0x7f9d1c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d2099b700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c101e10 0x7f9d1c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d2099b700 1 -- 172.21.15.8:0/930916663 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100095f0 con 0x7f9d1c100290 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d2099b700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d1c100290 0x7f9d1c0748a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d10000c00 tx=0x7f9d10005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d19ffb700 1 -- 172.21.15.8:0/930916663 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1001d070 con 0x7f9d1c100290 2023-11-27T15:23:31.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d22bff700 1 -- 172.21.15.8:0/930916663 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c10bc40 con 0x7f9d1c100290 2023-11-27T15:23:31.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d19ffb700 1 -- 172.21.15.8:0/930916663 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d10003f60 con 0x7f9d1c100290 2023-11-27T15:23:31.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d19ffb700 1 -- 172.21.15.8:0/930916663 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10022c70 con 0x7f9d1c100290 2023-11-27T15:23:31.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.873+0000 7f9d22bff700 1 -- 172.21.15.8:0/930916663 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c10bab0 con 0x7f9d1c100290 2023-11-27T15:23:31.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.874+0000 7f9d22bff700 1 -- 172.21.15.8:0/930916663 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c067a50 con 0x7f9d1c100290 2023-11-27T15:23:31.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.876+0000 7f9d19ffb700 1 -- 172.21.15.8:0/930916663 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9d100040d0 con 0x7f9d1c100290 2023-11-27T15:23:31.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.877+0000 7f9d19ffb700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9d0c0a1150 0x7f9d0c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:31.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.878+0000 7f9d19ffb700 1 -- 172.21.15.8:0/930916663 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f9d100ce0e0 con 0x7f9d1c100290 2023-11-27T15:23:31.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.878+0000 7f9d1bfff700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9d0c0a1150 0x7f9d0c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:31.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.878+0000 7f9d1bfff700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9d0c0a1150 0x7f9d0c0a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9d08005950 tx=0x7f9d080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:31.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:31.881+0000 7f9d19ffb700 1 -- 172.21.15.8:0/930916663 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9d10097690 con 0x7f9d1c100290 2023-11-27T15:23:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:31 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3739725108' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-27T15:23:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:31 smithi029 ceph-mon[131166]: osdmap e86: 12 total, 12 up, 12 in 2023-11-27T15:23:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:31 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/18696041' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-27T15:23:32.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:31 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3739725108' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-27T15:23:32.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:31 smithi060 ceph-mon[131518]: osdmap e86: 12 total, 12 up, 12 in 2023-11-27T15:23:32.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:31 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/18696041' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-27T15:23:32.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:31 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3739725108' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-27T15:23:32.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:31 smithi008 ceph-mon[128788]: osdmap e86: 12 total, 12 up, 12 in 2023-11-27T15:23:32.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:31 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/18696041' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-27T15:23:32.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.143+0000 7f9d22bff700 1 -- 172.21.15.8:0/930916663 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f9d1c107200 con 0x7f9d1c100290 2023-11-27T15:23:32.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.149+0000 7f9d19ffb700 1 -- 172.21.15.8:0/930916663 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f9d1009acb0 con 0x7f9d1c100290 2023-11-27T15:23:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 -- 172.21.15.8:0/930916663 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9d0c0a1150 msgr2=0x7f9d0c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9d0c0a1150 0x7f9d0c0a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9d08005950 tx=0x7f9d080058e0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 -- 172.21.15.8:0/930916663 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d1c100290 msgr2=0x7f9d1c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d1c100290 0x7f9d1c0748a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9d10000c00 tx=0x7f9d10005740 comp rx=0 tx=0).stop 2023-11-27T15:23:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 -- 172.21.15.8:0/930916663 shutdown_connections 2023-11-27T15:23:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9d1c101e10 0x7f9d1c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d1c100290 0x7f9d1c0748a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9d0c0a1150 0x7f9d0c0a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 --2- 172.21.15.8:0/930916663 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9d1c101480 0x7f9d1c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 -- 172.21.15.8:0/930916663 >> 172.21.15.8:0/930916663 conn(0x7f9d1c0fb870 msgr2=0x7f9d1c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 -- 172.21.15.8:0/930916663 shutdown_connections 2023-11-27T15:23:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.152+0000 7f9d22bff700 1 -- 172.21.15.8:0/930916663 wait complete. 2023-11-27T15:23:32.158 INFO:teuthology.orchestra.run.smithi008.stderr:added data pool 4 to fsmap 2023-11-27T15:23:32.176 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:32.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.374+0000 7f35275ff700 1 -- 172.21.15.8:0/3892473197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3520101e50 msgr2=0x7f3520108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:32.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.374+0000 7f35275ff700 1 --2- 172.21.15.8:0/3892473197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3520101e50 0x7f3520108710 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f351800b0a0 tx=0x7f351800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.374+0000 7f35275ff700 1 -- 172.21.15.8:0/3892473197 shutdown_connections 2023-11-27T15:23:32.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.374+0000 7f35275ff700 1 --2- 172.21.15.8:0/3892473197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3520101e50 0x7f3520108710 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.374+0000 7f35275ff700 1 --2- 172.21.15.8:0/3892473197 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35201002d0 0x7f35201006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.379 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.374+0000 7f35275ff700 1 --2- 172.21.15.8:0/3892473197 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f35201014c0 0x7f3520101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.374+0000 7f35275ff700 1 -- 172.21.15.8:0/3892473197 >> 172.21.15.8:0/3892473197 conn(0x7f35200fb8b0 msgr2=0x7f35200fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:32.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.374+0000 7f35275ff700 1 -- 172.21.15.8:0/3892473197 shutdown_connections 2023-11-27T15:23:32.380 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.374+0000 7f35275ff700 1 -- 172.21.15.8:0/3892473197 wait complete. 2023-11-27T15:23:32.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.375+0000 7f35275ff700 1 Processor -- start 2023-11-27T15:23:32.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.375+0000 7f35275ff700 1 -- start start 2023-11-27T15:23:32.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f35275ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35201002d0 0x7f3520198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:32.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f35275ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35201014c0 0x7f3520198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:32.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f35275ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3520101e50 0x7f352019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:32.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f35275ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352019fec0 con 0x7f35201002d0 2023-11-27T15:23:32.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f35275ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35201a0000 con 0x7f35201014c0 2023-11-27T15:23:32.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f35275ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35201a0190 con 0x7f3520101e50 2023-11-27T15:23:32.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f352539b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35201002d0 0x7f3520198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:32.382 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f352539b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35201002d0 0x7f3520198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37242/0 (socket says 172.21.15.8:37242) 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f352539b700 1 -- 172.21.15.8:0/3909786496 learned_addr learned my addr 172.21.15.8:0/3909786496 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f3525b9c700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3520101e50 0x7f352019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f3524b9a700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35201014c0 0x7f3520198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f352539b700 1 -- 172.21.15.8:0/3909786496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3520101e50 msgr2=0x7f352019f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f352539b700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3520101e50 0x7f352019f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f352539b700 1 -- 172.21.15.8:0/3909786496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35201014c0 msgr2=0x7f3520198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f352539b700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35201014c0 0x7f3520198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f352539b700 1 -- 172.21.15.8:0/3909786496 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009d00 con 0x7f35201002d0 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.376+0000 7f3525b9c700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3520101e50 0x7f352019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.377+0000 7f352539b700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35201002d0 0x7f3520198340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f351c00e970 tx=0x7f351c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.377+0000 7f35167fc700 1 -- 172.21.15.8:0/3909786496 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c009790 con 0x7f35201002d0 2023-11-27T15:23:32.383 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.377+0000 7f35167fc700 1 -- 172.21.15.8:0/3909786496 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f351c00eec0 con 0x7f35201002d0 2023-11-27T15:23:32.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.377+0000 7f35275ff700 1 -- 172.21.15.8:0/3909786496 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352010bca0 con 0x7f35201002d0 2023-11-27T15:23:32.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.377+0000 7f35167fc700 1 -- 172.21.15.8:0/3909786496 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c003cf0 con 0x7f35201002d0 2023-11-27T15:23:32.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.377+0000 7f35275ff700 1 -- 172.21.15.8:0/3909786496 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352010bb10 con 0x7f35201002d0 2023-11-27T15:23:32.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.378+0000 7f35275ff700 1 -- 172.21.15.8:0/3909786496 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35201a08e0 con 0x7f35201002d0 2023-11-27T15:23:32.384 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.380+0000 7f35167fc700 1 -- 172.21.15.8:0/3909786496 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f351c00ccd0 con 0x7f35201002d0 2023-11-27T15:23:32.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.381+0000 7f35167fc700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f350c0a1150 0x7f350c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:32.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.381+0000 7f35167fc700 1 -- 172.21.15.8:0/3909786496 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f351c014070 con 0x7f35201002d0 2023-11-27T15:23:32.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.381+0000 7f3524b9a700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f350c0a1150 0x7f350c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:32.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.382+0000 7f3524b9a700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f350c0a1150 0x7f350c0a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3510009c60 tx=0x7f3510009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:32.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.385+0000 7f35167fc700 1 -- 172.21.15.8:0/3909786496 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f351c0962b0 con 0x7f35201002d0 2023-11-27T15:23:32.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.647+0000 7f35275ff700 1 -- 172.21.15.8:0/3909786496 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35201a0e90 con 0x7f35201002d0 2023-11-27T15:23:32.652 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.648+0000 7f35167fc700 1 -- 172.21.15.8:0/3909786496 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10708 (secure 0 0 0) 0x7f351c0998d0 con 0x7f35201002d0 2023-11-27T15:23:32.653 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:32.653 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24593,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-27T15:23:27.167939+0000","modified":"2023-11-27T15:23:32.146351+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24595":{"gid":24595,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-27T15:23:32.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 -- 172.21.15.8:0/3909786496 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f350c0a1150 msgr2=0x7f350c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:32.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f350c0a1150 0x7f350c0a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3510009c60 tx=0x7f3510009260 comp rx=0 tx=0).stop 2023-11-27T15:23:32.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 -- 172.21.15.8:0/3909786496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35201002d0 msgr2=0x7f3520198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:32.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35201002d0 0x7f3520198340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f351c00e970 tx=0x7f351c00ec80 comp rx=0 tx=0).stop 2023-11-27T15:23:32.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 -- 172.21.15.8:0/3909786496 shutdown_connections 2023-11-27T15:23:32.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f350c0a1150 0x7f350c0a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3520101e50 0x7f352019f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f35201002d0 0x7f3520198340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 --2- 172.21.15.8:0/3909786496 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f35201014c0 0x7f3520198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 -- 172.21.15.8:0/3909786496 >> 172.21.15.8:0/3909786496 conn(0x7f35200fb8b0 msgr2=0x7f3520106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:32.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.650+0000 7f35275ff700 1 -- 172.21.15.8:0/3909786496 shutdown_connections 2023-11-27T15:23:32.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.651+0000 7f35275ff700 1 -- 172.21.15.8:0/3909786496 wait complete. 2023-11-27T15:23:32.657 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 17 2023-11-27T15:23:32.674 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:23:32.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.874+0000 7feb56dff700 1 -- 172.21.15.8:0/2619297522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb50105f80 msgr2=0x7feb501083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:32.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.874+0000 7feb56dff700 1 --2- 172.21.15.8:0/2619297522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb50105f80 0x7feb501083e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb48009f90 tx=0x7feb4800b330 comp rx=0 tx=0).stop 2023-11-27T15:23:32.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.874+0000 7feb56dff700 1 -- 172.21.15.8:0/2619297522 shutdown_connections 2023-11-27T15:23:32.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.874+0000 7feb56dff700 1 --2- 172.21.15.8:0/2619297522 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb50100d40 0x7feb50103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.874+0000 7feb56dff700 1 --2- 172.21.15.8:0/2619297522 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb50105f80 0x7feb501083e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.874+0000 7feb56dff700 1 --2- 172.21.15.8:0/2619297522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb50103660 0x7feb50105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.874+0000 7feb56dff700 1 -- 172.21.15.8:0/2619297522 >> 172.21.15.8:0/2619297522 conn(0x7feb500fa7c0 msgr2=0x7feb500fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:32.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.874+0000 7feb56dff700 1 -- 172.21.15.8:0/2619297522 shutdown_connections 2023-11-27T15:23:32.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.874+0000 7feb56dff700 1 -- 172.21.15.8:0/2619297522 wait complete. 2023-11-27T15:23:32.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.875+0000 7feb56dff700 1 Processor -- start 2023-11-27T15:23:32.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.875+0000 7feb56dff700 1 -- start start 2023-11-27T15:23:32.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb56dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb50100d40 0x7feb501006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:32.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb56dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb50103660 0x7feb500fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:32.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb56dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb50105f80 0x7feb500ff230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:32.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb56dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb500ff9d0 con 0x7feb50105f80 2023-11-27T15:23:32.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb56dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb500ffb40 con 0x7feb50103660 2023-11-27T15:23:32.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb56dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb501a01e0 con 0x7feb50100d40 2023-11-27T15:23:32.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb54b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb50100d40 0x7feb501006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb5539c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb50105f80 0x7feb500ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb54b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb50100d40 0x7feb501006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:47428/0 (socket says 172.21.15.8:47428) 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb54b9b700 1 -- 172.21.15.8:0/3701840029 learned_addr learned my addr 172.21.15.8:0/3701840029 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb4ffff700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb50103660 0x7feb500fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb5539c700 1 -- 172.21.15.8:0/3701840029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb50100d40 msgr2=0x7feb501006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb5539c700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb50100d40 0x7feb501006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb5539c700 1 -- 172.21.15.8:0/3701840029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb50103660 msgr2=0x7feb500fecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb5539c700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb50103660 0x7feb500fecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb5539c700 1 -- 172.21.15.8:0/3701840029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb48009c40 con 0x7feb50105f80 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.876+0000 7feb54b9b700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb50100d40 0x7feb501006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.877+0000 7feb5539c700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb50105f80 0x7feb500ff230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feb4800b6c0 tx=0x7feb48009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:32.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.877+0000 7feb4dffb700 1 -- 172.21.15.8:0/3701840029 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800e040 con 0x7feb50105f80 2023-11-27T15:23:32.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.877+0000 7feb56dff700 1 -- 172.21.15.8:0/3701840029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb5010bcd0 con 0x7feb50105f80 2023-11-27T15:23:32.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.877+0000 7feb4dffb700 1 -- 172.21.15.8:0/3701840029 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb48004e70 con 0x7feb50105f80 2023-11-27T15:23:32.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.877+0000 7feb4dffb700 1 -- 172.21.15.8:0/3701840029 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48005040 con 0x7feb50105f80 2023-11-27T15:23:32.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.877+0000 7feb56dff700 1 -- 172.21.15.8:0/3701840029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb5010bb40 con 0x7feb50105f80 2023-11-27T15:23:32.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.878+0000 7feb56dff700 1 -- 172.21.15.8:0/3701840029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb500fc330 con 0x7feb50105f80 2023-11-27T15:23:32.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.880+0000 7feb4dffb700 1 -- 172.21.15.8:0/3701840029 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7feb4801d470 con 0x7feb50105f80 2023-11-27T15:23:32.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.881+0000 7feb4dffb700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feb400a1150 0x7feb400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:32.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.881+0000 7feb4dffb700 1 -- 172.21.15.8:0/3701840029 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7feb480cde80 con 0x7feb50105f80 2023-11-27T15:23:32.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.881+0000 7feb54b9b700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feb400a1150 0x7feb400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:32.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.882+0000 7feb54b9b700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feb400a1150 0x7feb400a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feb44011550 tx=0x7feb44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:32.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:32.885+0000 7feb4dffb700 1 -- 172.21.15.8:0/3701840029 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7feb48097410 con 0x7feb50105f80 2023-11-27T15:23:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:32 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/18696041' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-27T15:23:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:32 smithi029 ceph-mon[131166]: osdmap e87: 12 total, 12 up, 12 in 2023-11-27T15:23:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:32 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/930916663' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-27T15:23:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:32 smithi029 ceph-mon[131166]: osdmap e88: 12 total, 12 up, 12 in 2023-11-27T15:23:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:32 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/930916663' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-27T15:23:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:32 smithi029 ceph-mon[131166]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:32.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:32 smithi029 ceph-mon[131166]: pgmap v285: 129 pgs: 17 creating+peering, 102 active+clean, 10 unknown; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-11-27T15:23:33.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:32 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/18696041' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-27T15:23:33.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:32 smithi060 ceph-mon[131518]: osdmap e87: 12 total, 12 up, 12 in 2023-11-27T15:23:33.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:32 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/930916663' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-27T15:23:33.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:32 smithi060 ceph-mon[131518]: osdmap e88: 12 total, 12 up, 12 in 2023-11-27T15:23:33.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:32 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/930916663' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-27T15:23:33.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:32 smithi060 ceph-mon[131518]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:33.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:32 smithi060 ceph-mon[131518]: pgmap v285: 129 pgs: 17 creating+peering, 102 active+clean, 10 unknown; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-11-27T15:23:33.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.095+0000 7feb56dff700 1 -- 172.21.15.8:0/3701840029 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb50061700 con 0x7feb50105f80 2023-11-27T15:23:33.101 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.097+0000 7feb4dffb700 1 -- 172.21.15.8:0/3701840029 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23817 (secure 0 0 0) 0x7feb4809aa30 con 0x7feb50105f80 2023-11-27T15:23:33.102 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:33.102 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":88,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:23:32.146115+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 -- 172.21.15.8:0/3701840029 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feb400a1150 msgr2=0x7feb400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feb400a1150 0x7feb400a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feb44011550 tx=0x7feb44009260 comp rx=0 tx=0).stop 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 -- 172.21.15.8:0/3701840029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb50105f80 msgr2=0x7feb500ff230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb50105f80 0x7feb500ff230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feb4800b6c0 tx=0x7feb48009510 comp rx=0 tx=0).stop 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 -- 172.21.15.8:0/3701840029 shutdown_connections 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feb400a1150 0x7feb400a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feb50100d40 0x7feb501006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb50105f80 0x7feb500ff230 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 --2- 172.21.15.8:0/3701840029 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feb50103660 0x7feb500fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 -- 172.21.15.8:0/3701840029 >> 172.21.15.8:0/3701840029 conn(0x7feb500fa7c0 msgr2=0x7feb501019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 -- 172.21.15.8:0/3701840029 shutdown_connections 2023-11-27T15:23:33.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.099+0000 7feb56dff700 1 -- 172.21.15.8:0/3701840029 wait complete. 2023-11-27T15:23:33.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:32 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/18696041' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-27T15:23:33.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:32 smithi008 ceph-mon[128788]: osdmap e87: 12 total, 12 up, 12 in 2023-11-27T15:23:33.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:32 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/930916663' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-27T15:23:33.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:32 smithi008 ceph-mon[128788]: osdmap e88: 12 total, 12 up, 12 in 2023-11-27T15:23:33.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:32 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/930916663' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-27T15:23:33.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:32 smithi008 ceph-mon[128788]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:33.113 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:32 smithi008 ceph-mon[128788]: pgmap v285: 129 pgs: 17 creating+peering, 102 active+clean, 10 unknown; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-11-27T15:23:33.123 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:23:33.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.324+0000 7faa6adff700 1 -- 172.21.15.8:0/356419179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa64074b30 msgr2=0x7faa64072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:33.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.324+0000 7faa6adff700 1 --2- 172.21.15.8:0/356419179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa64074b30 0x7faa64072f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faa58009990 tx=0x7faa58009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.324+0000 7faa6adff700 1 -- 172.21.15.8:0/356419179 shutdown_connections 2023-11-27T15:23:33.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.324+0000 7faa6adff700 1 --2- 172.21.15.8:0/356419179 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa64108190 0x7faa6410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.324+0000 7faa6adff700 1 --2- 172.21.15.8:0/356419179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa64074b30 0x7faa64072f80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.324+0000 7faa6adff700 1 --2- 172.21.15.8:0/356419179 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa64073550 0x7faa640739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.324+0000 7faa6adff700 1 -- 172.21.15.8:0/356419179 >> 172.21.15.8:0/356419179 conn(0x7faa640fb8b0 msgr2=0x7faa640fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:33.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.324+0000 7faa6adff700 1 -- 172.21.15.8:0/356419179 shutdown_connections 2023-11-27T15:23:33.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.325+0000 7faa6adff700 1 -- 172.21.15.8:0/356419179 wait complete. 2023-11-27T15:23:33.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.325+0000 7faa6adff700 1 Processor -- start 2023-11-27T15:23:33.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa6adff700 1 -- start start 2023-11-27T15:23:33.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa6adff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa64073550 0x7faa64198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:33.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa6adff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa64074b30 0x7faa64198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:33.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa6adff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa64108190 0x7faa6419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:33.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa6adff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6419fe90 con 0x7faa64074b30 2023-11-27T15:23:33.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa6adff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6419ffd0 con 0x7faa64073550 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa6adff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa641a0160 con 0x7faa64108190 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa63fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa64074b30 0x7faa64198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa68b9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa64073550 0x7faa64198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa63fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa64074b30 0x7faa64198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37278/0 (socket says 172.21.15.8:37278) 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa63fff700 1 -- 172.21.15.8:0/3928131776 learned_addr learned my addr 172.21.15.8:0/3928131776 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.326+0000 7faa6939c700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa64108190 0x7faa6419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa68b9b700 1 -- 172.21.15.8:0/3928131776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa64108190 msgr2=0x7faa6419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa68b9b700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa64108190 0x7faa6419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa68b9b700 1 -- 172.21.15.8:0/3928131776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa64074b30 msgr2=0x7faa64198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa68b9b700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa64074b30 0x7faa64198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa68b9b700 1 -- 172.21.15.8:0/3928131776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c009c40 con 0x7faa64073550 2023-11-27T15:23:33.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa6939c700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa64108190 0x7faa6419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:33.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa63fff700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa64074b30 0x7faa64198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:33.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa68b9b700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa64073550 0x7faa64198310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa580098e0 tx=0x7faa58009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:33.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa61ffb700 1 -- 172.21.15.8:0/3928131776 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5801d070 con 0x7faa64073550 2023-11-27T15:23:33.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa6adff700 1 -- 172.21.15.8:0/3928131776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa580095f0 con 0x7faa64073550 2023-11-27T15:23:33.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa61ffb700 1 -- 172.21.15.8:0/3928131776 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa58021410 con 0x7faa64073550 2023-11-27T15:23:33.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa61ffb700 1 -- 172.21.15.8:0/3928131776 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58017650 con 0x7faa64073550 2023-11-27T15:23:33.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.327+0000 7faa6adff700 1 -- 172.21.15.8:0/3928131776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6410de90 con 0x7faa64073550 2023-11-27T15:23:33.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.328+0000 7faa6adff700 1 -- 172.21.15.8:0/3928131776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa64106840 con 0x7faa64073550 2023-11-27T15:23:33.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.330+0000 7faa61ffb700 1 -- 172.21.15.8:0/3928131776 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7faa58021580 con 0x7faa64073550 2023-11-27T15:23:33.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.331+0000 7faa61ffb700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa540a1150 0x7faa540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:33.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.332+0000 7faa61ffb700 1 -- 172.21.15.8:0/3928131776 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faa580cde30 con 0x7faa64073550 2023-11-27T15:23:33.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.332+0000 7faa63fff700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa540a1150 0x7faa540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:33.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.332+0000 7faa63fff700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa540a1150 0x7faa540a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa50005950 tx=0x7faa50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:33.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.336+0000 7faa61ffb700 1 -- 172.21.15.8:0/3928131776 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7faa580973c0 con 0x7faa64073550 2023-11-27T15:23:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.546+0000 7faa6adff700 1 -- 172.21.15.8:0/3928131776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa6402cfe0 con 0x7faa64073550 2023-11-27T15:23:33.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.549+0000 7faa61ffb700 1 -- 172.21.15.8:0/3928131776 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23817 (secure 0 0 0) 0x7faa5809a9e0 con 0x7faa64073550 2023-11-27T15:23:33.553 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:33.553 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":88,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:23:32.146115+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:23:33.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.550+0000 7faa6adff700 1 -- 172.21.15.8:0/3928131776 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa540a1150 msgr2=0x7faa540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:33.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.550+0000 7faa6adff700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa540a1150 0x7faa540a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa50005950 tx=0x7faa50009260 comp rx=0 tx=0).stop 2023-11-27T15:23:33.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.550+0000 7faa6adff700 1 -- 172.21.15.8:0/3928131776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa64073550 msgr2=0x7faa64198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:33.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.550+0000 7faa6adff700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa64073550 0x7faa64198310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa580098e0 tx=0x7faa58009580 comp rx=0 tx=0).stop 2023-11-27T15:23:33.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.551+0000 7faa6adff700 1 -- 172.21.15.8:0/3928131776 shutdown_connections 2023-11-27T15:23:33.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.551+0000 7faa6adff700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa540a1150 0x7faa540a3600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.551+0000 7faa6adff700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faa64108190 0x7faa6419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.551+0000 7faa6adff700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faa64073550 0x7faa64198310 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.551+0000 7faa6adff700 1 --2- 172.21.15.8:0/3928131776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa64074b30 0x7faa64198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.551+0000 7faa6adff700 1 -- 172.21.15.8:0/3928131776 >> 172.21.15.8:0/3928131776 conn(0x7faa640fb8b0 msgr2=0x7faa640fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:33.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.551+0000 7faa6adff700 1 -- 172.21.15.8:0/3928131776 shutdown_connections 2023-11-27T15:23:33.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.551+0000 7faa6adff700 1 -- 172.21.15.8:0/3928131776 wait complete. 2023-11-27T15:23:33.587 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:33.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.787+0000 7f59e69ff700 1 -- 172.21.15.8:0/647810841 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59e01002d0 msgr2=0x7f59e01006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:33.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.787+0000 7f59e69ff700 1 --2- 172.21.15.8:0/647810841 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59e01002d0 0x7f59e01006e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009990 tx=0x7f59d0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.787+0000 7f59e69ff700 1 -- 172.21.15.8:0/647810841 shutdown_connections 2023-11-27T15:23:33.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.787+0000 7f59e69ff700 1 --2- 172.21.15.8:0/647810841 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e0101e50 0x7f59e0108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.787+0000 7f59e69ff700 1 --2- 172.21.15.8:0/647810841 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59e01002d0 0x7f59e01006e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.787+0000 7f59e69ff700 1 --2- 172.21.15.8:0/647810841 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59e01014c0 0x7f59e0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.787+0000 7f59e69ff700 1 -- 172.21.15.8:0/647810841 >> 172.21.15.8:0/647810841 conn(0x7f59e00fb8b0 msgr2=0x7f59e00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:33.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.787+0000 7f59e69ff700 1 -- 172.21.15.8:0/647810841 shutdown_connections 2023-11-27T15:23:33.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.787+0000 7f59e69ff700 1 -- 172.21.15.8:0/647810841 wait complete. 2023-11-27T15:23:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.788+0000 7f59e69ff700 1 Processor -- start 2023-11-27T15:23:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.788+0000 7f59e69ff700 1 -- start start 2023-11-27T15:23:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.788+0000 7f59e69ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59e01002d0 0x7f59e0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.788+0000 7f59e69ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59e01014c0 0x7f59e01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.788+0000 7f59e69ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e0101e50 0x7f59e019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.788+0000 7f59e69ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e019fde0 con 0x7f59e01014c0 2023-11-27T15:23:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.788+0000 7f59e69ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e019ff20 con 0x7f59e01002d0 2023-11-27T15:23:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.788+0000 7f59e69ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01a00b0 con 0x7f59e0101e50 2023-11-27T15:23:33.795 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59d7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59e01014c0 0x7f59e01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59d7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59e01014c0 0x7f59e01987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37302/0 (socket says 172.21.15.8:37302) 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59d7fff700 1 -- 172.21.15.8:0/3129342884 learned_addr learned my addr 172.21.15.8:0/3129342884 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59dffff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59e01002d0 0x7f59e0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59e4f9c700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e0101e50 0x7f59e019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59d7fff700 1 -- 172.21.15.8:0/3129342884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e0101e50 msgr2=0x7f59e019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59d7fff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e0101e50 0x7f59e019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59d7fff700 1 -- 172.21.15.8:0/3129342884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59e01002d0 msgr2=0x7f59e0198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59d7fff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59e01002d0 0x7f59e0198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59d7fff700 1 -- 172.21.15.8:0/3129342884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d00095f0 con 0x7f59e01014c0 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59dffff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59e01002d0 0x7f59e0198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59e4f9c700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e0101e50 0x7f59e019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59d7fff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59e01014c0 0x7f59e01987a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59c800d950 tx=0x7f59c800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:33.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.789+0000 7f59ddffb700 1 -- 172.21.15.8:0/3129342884 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8009b40 con 0x7f59e01014c0 2023-11-27T15:23:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.790+0000 7f59ddffb700 1 -- 172.21.15.8:0/3129342884 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59c8009ca0 con 0x7f59e01014c0 2023-11-27T15:23:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.790+0000 7f59e69ff700 1 -- 172.21.15.8:0/3129342884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e010bca0 con 0x7f59e01014c0 2023-11-27T15:23:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.790+0000 7f59ddffb700 1 -- 172.21.15.8:0/3129342884 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c800f620 con 0x7f59e01014c0 2023-11-27T15:23:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.790+0000 7f59e69ff700 1 -- 172.21.15.8:0/3129342884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e010bb10 con 0x7f59e01014c0 2023-11-27T15:23:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.791+0000 7f59e69ff700 1 -- 172.21.15.8:0/3129342884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e01a07f0 con 0x7f59e01014c0 2023-11-27T15:23:33.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.793+0000 7f59ddffb700 1 -- 172.21.15.8:0/3129342884 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f59c8010460 con 0x7f59e01014c0 2023-11-27T15:23:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.794+0000 7f59ddffb700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59cc0a11d0 0x7f59cc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.794+0000 7f59ddffb700 1 -- 172.21.15.8:0/3129342884 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f59c80cce60 con 0x7f59e01014c0 2023-11-27T15:23:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.794+0000 7f59dffff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59cc0a11d0 0x7f59cc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:33.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.795+0000 7f59dffff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59cc0a11d0 0x7f59cc0a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f59d000b5c0 tx=0x7f59d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:33.802 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:33.798+0000 7f59ddffb700 1 -- 172.21.15.8:0/3129342884 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f59c8096360 con 0x7f59e01014c0 2023-11-27T15:23:33.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:33 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3909786496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:33.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:33 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3701840029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:33.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:33 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3928131776' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:33 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3909786496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:33 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3701840029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:33 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3928131776' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:34.060 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.056+0000 7f59e69ff700 1 -- 172.21.15.8:0/3129342884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59e01a0da0 con 0x7f59e01014c0 2023-11-27T15:23:34.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.056+0000 7f59ddffb700 1 -- 172.21.15.8:0/3129342884 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10708 (secure 0 0 0) 0x7f59c8020090 con 0x7f59e01014c0 2023-11-27T15:23:34.061 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:34.061 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24535,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24565,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24593,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-27T15:23:27.167939+0000","modified":"2023-11-27T15:23:32.146351+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24595":{"gid":24595,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-27T15:23:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 -- 172.21.15.8:0/3129342884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59cc0a11d0 msgr2=0x7f59cc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59cc0a11d0 0x7f59cc0a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f59d000b5c0 tx=0x7f59d0005fd0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 -- 172.21.15.8:0/3129342884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59e01014c0 msgr2=0x7f59e01987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59e01014c0 0x7f59e01987a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59c800d950 tx=0x7f59c800dd10 comp rx=0 tx=0).stop 2023-11-27T15:23:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 -- 172.21.15.8:0/3129342884 shutdown_connections 2023-11-27T15:23:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59cc0a11d0 0x7f59cc0a3680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f59e0101e50 0x7f59e019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f59e01002d0 0x7f59e0198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 --2- 172.21.15.8:0/3129342884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59e01014c0 0x7f59e01987a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 -- 172.21.15.8:0/3129342884 >> 172.21.15.8:0/3129342884 conn(0x7f59e00fb8b0 msgr2=0x7f59e0106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 -- 172.21.15.8:0/3129342884 shutdown_connections 2023-11-27T15:23:34.064 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.059+0000 7f59e69ff700 1 -- 172.21.15.8:0/3129342884 wait complete. 2023-11-27T15:23:34.065 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 17 2023-11-27T15:23:34.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:33 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3909786496' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:34.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:33 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3701840029' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:34.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:33 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3928131776' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:34.083 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:23:34.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.281+0000 7f78799ff700 1 -- 172.21.15.8:0/3417540658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78741014c0 msgr2=0x7f7874101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.281+0000 7f78799ff700 1 --2- 172.21.15.8:0/3417540658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78741014c0 0x7f7874101910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7868009990 tx=0x7f7868009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.281+0000 7f78799ff700 1 -- 172.21.15.8:0/3417540658 shutdown_connections 2023-11-27T15:23:34.289 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.281+0000 7f78799ff700 1 --2- 172.21.15.8:0/3417540658 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78741002d0 0x7f78741006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.281+0000 7f78799ff700 1 --2- 172.21.15.8:0/3417540658 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7874101e50 0x7f7874108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.281+0000 7f78799ff700 1 --2- 172.21.15.8:0/3417540658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78741014c0 0x7f7874101910 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.281+0000 7f78799ff700 1 -- 172.21.15.8:0/3417540658 >> 172.21.15.8:0/3417540658 conn(0x7f78740fb8b0 msgr2=0x7f78740fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:34.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.281+0000 7f78799ff700 1 -- 172.21.15.8:0/3417540658 shutdown_connections 2023-11-27T15:23:34.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.281+0000 7f78799ff700 1 -- 172.21.15.8:0/3417540658 wait complete. 2023-11-27T15:23:34.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.282+0000 7f78799ff700 1 Processor -- start 2023-11-27T15:23:34.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.282+0000 7f78799ff700 1 -- start start 2023-11-27T15:23:34.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78799ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78741002d0 0x7f7874198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78799ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78741014c0 0x7f7874198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78799ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7874101e50 0x7f787419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.292 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78799ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787419fdc0 con 0x7f7874101e50 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78799ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787419ff00 con 0x7f78741002d0 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78799ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78741a0090 con 0x7f78741014c0 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78727fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78741014c0 0x7f7874198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78737fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7874101e50 0x7f787419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78727fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78741014c0 0x7f7874198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:47464/0 (socket says 172.21.15.8:47464) 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78727fc700 1 -- 172.21.15.8:0/4156351521 learned_addr learned my addr 172.21.15.8:0/4156351521 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78737fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7874101e50 0x7f787419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37314/0 (socket says 172.21.15.8:37314) 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f7872ffd700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78741002d0 0x7f7874198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78737fe700 1 -- 172.21.15.8:0/4156351521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78741014c0 msgr2=0x7f7874198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78737fe700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78741014c0 0x7f7874198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78737fe700 1 -- 172.21.15.8:0/4156351521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78741002d0 msgr2=0x7f7874198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.293 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78737fe700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78741002d0 0x7f7874198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.283+0000 7f78737fe700 1 -- 172.21.15.8:0/4156351521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680095f0 con 0x7f7874101e50 2023-11-27T15:23:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.284+0000 7f78737fe700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7874101e50 0x7f787419f7f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f786c00efc0 tx=0x7f786c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.284+0000 7f78789fd700 1 -- 172.21.15.8:0/4156351521 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c018070 con 0x7f7874101e50 2023-11-27T15:23:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.284+0000 7f78789fd700 1 -- 172.21.15.8:0/4156351521 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f786c00f040 con 0x7f7874101e50 2023-11-27T15:23:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.284+0000 7f78789fd700 1 -- 172.21.15.8:0/4156351521 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00cd40 con 0x7f7874101e50 2023-11-27T15:23:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.284+0000 7f78799ff700 1 -- 172.21.15.8:0/4156351521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787410bc80 con 0x7f7874101e50 2023-11-27T15:23:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.284+0000 7f78799ff700 1 -- 172.21.15.8:0/4156351521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787410baf0 con 0x7f7874101e50 2023-11-27T15:23:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.285+0000 7f78799ff700 1 -- 172.21.15.8:0/4156351521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78741a06d0 con 0x7f7874101e50 2023-11-27T15:23:34.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.287+0000 7f78789fd700 1 -- 172.21.15.8:0/4156351521 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f786c003a00 con 0x7f7874101e50 2023-11-27T15:23:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.288+0000 7f78789fd700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f785c0a11d0 0x7f785c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.289+0000 7f78789fd700 1 -- 172.21.15.8:0/4156351521 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f786c0cc630 con 0x7f7874101e50 2023-11-27T15:23:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.289+0000 7f7872ffd700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f785c0a11d0 0x7f785c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.289+0000 7f7872ffd700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f785c0a11d0 0x7f785c0a3680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7874101320 tx=0x7f786400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:34.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.292+0000 7f78789fd700 1 -- 172.21.15.8:0/4156351521 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f786c095bc0 con 0x7f7874101e50 2023-11-27T15:23:34.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.503+0000 7f78799ff700 1 -- 172.21.15.8:0/4156351521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78741a0d70 con 0x7f7874101e50 2023-11-27T15:23:34.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.505+0000 7f78789fd700 1 -- 172.21.15.8:0/4156351521 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23817 (secure 0 0 0) 0x7f786c0991e0 con 0x7f7874101e50 2023-11-27T15:23:34.510 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:34.510 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":88,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:23:32.146115+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:23:34.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.507+0000 7f78799ff700 1 -- 172.21.15.8:0/4156351521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f785c0a11d0 msgr2=0x7f785c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.507+0000 7f78799ff700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f785c0a11d0 0x7f785c0a3680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7874101320 tx=0x7f786400b410 comp rx=0 tx=0).stop 2023-11-27T15:23:34.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.507+0000 7f78799ff700 1 -- 172.21.15.8:0/4156351521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7874101e50 msgr2=0x7f787419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.507+0000 7f78799ff700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7874101e50 0x7f787419f7f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f786c00efc0 tx=0x7f786c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.507+0000 7f78799ff700 1 -- 172.21.15.8:0/4156351521 shutdown_connections 2023-11-27T15:23:34.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.508+0000 7f78799ff700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7874101e50 0x7f787419f7f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.508+0000 7f78799ff700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78741002d0 0x7f7874198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.508+0000 7f78799ff700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f785c0a11d0 0x7f785c0a3680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.508+0000 7f78799ff700 1 --2- 172.21.15.8:0/4156351521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78741014c0 0x7f7874198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.508+0000 7f78799ff700 1 -- 172.21.15.8:0/4156351521 >> 172.21.15.8:0/4156351521 conn(0x7f78740fb8b0 msgr2=0x7f7874106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:34.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.508+0000 7f78799ff700 1 -- 172.21.15.8:0/4156351521 shutdown_connections 2023-11-27T15:23:34.513 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.508+0000 7f78799ff700 1 -- 172.21.15.8:0/4156351521 wait complete. 2023-11-27T15:23:34.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-27T15:23:27.167939+0000', 'modified': '2023-11-27T15:23:32.146351+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24595}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24595': {'gid': 24595, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.8:6841/110161953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.8:6840', 'nonce': 110161953}, {'type': 'v1', 'addr': '172.21.15.8:6841', 'nonce': 110161953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-27T15:23:34.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-27T15:23:34.532 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-27T15:23:34.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.736+0000 7f7bc65ff700 1 -- 172.21.15.8:0/159799446 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bc0100240 msgr2=0x7f7bc0100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.736+0000 7f7bc65ff700 1 --2- 172.21.15.8:0/159799446 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bc0100240 0x7f7bc0100650 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009990 tx=0x7f7bb0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.736+0000 7f7bc65ff700 1 -- 172.21.15.8:0/159799446 shutdown_connections 2023-11-27T15:23:34.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.736+0000 7f7bc65ff700 1 --2- 172.21.15.8:0/159799446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc0101dc0 0x7f7bc0108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.736+0000 7f7bc65ff700 1 --2- 172.21.15.8:0/159799446 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bc0100240 0x7f7bc0100650 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.736+0000 7f7bc65ff700 1 --2- 172.21.15.8:0/159799446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc0101430 0x7f7bc0101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.736+0000 7f7bc65ff700 1 -- 172.21.15.8:0/159799446 >> 172.21.15.8:0/159799446 conn(0x7f7bc00fb820 msgr2=0x7f7bc00fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:34.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.736+0000 7f7bc65ff700 1 -- 172.21.15.8:0/159799446 shutdown_connections 2023-11-27T15:23:34.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.736+0000 7f7bc65ff700 1 -- 172.21.15.8:0/159799446 wait complete. 2023-11-27T15:23:34.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.737+0000 7f7bc65ff700 1 Processor -- start 2023-11-27T15:23:34.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.737+0000 7f7bc65ff700 1 -- start start 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.737+0000 7f7bc65ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc0100240 0x7f7bc01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.737+0000 7f7bc65ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bc0101430 0x7f7bc0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.737+0000 7f7bc65ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc0101dc0 0x7f7bc019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.737+0000 7f7bc65ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc019fd60 con 0x7f7bc0101430 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.737+0000 7f7bc65ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc019fea0 con 0x7f7bc0100240 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.737+0000 7f7bc65ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc01a0030 con 0x7f7bc0101dc0 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc0100240 0x7f7bc01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bc0101430 0x7f7bc0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc0100240 0x7f7bc01981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:35728/0 (socket says 172.21.15.8:35728) 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbffff700 1 -- 172.21.15.8:0/173881576 learned_addr learned my addr 172.21.15.8:0/173881576 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bc4b9c700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc0101dc0 0x7f7bc019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbffff700 1 -- 172.21.15.8:0/173881576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc0101dc0 msgr2=0x7f7bc019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbffff700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc0101dc0 0x7f7bc019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbffff700 1 -- 172.21.15.8:0/173881576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bc0101430 msgr2=0x7f7bc0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbffff700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bc0101430 0x7f7bc0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbffff700 1 -- 172.21.15.8:0/173881576 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb00095f0 con 0x7f7bc0100240 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbf7fe700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bc0101430 0x7f7bc0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.738+0000 7f7bbffff700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc0100240 0x7f7bc01981e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0005740 tx=0x7f7bb0005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.739+0000 7f7bbd7fa700 1 -- 172.21.15.8:0/173881576 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb001d070 con 0x7f7bc0100240 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.739+0000 7f7bbd7fa700 1 -- 172.21.15.8:0/173881576 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bb0021410 con 0x7f7bc0100240 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.739+0000 7f7bc65ff700 1 -- 172.21.15.8:0/173881576 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc010bbf0 con 0x7f7bc0100240 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.739+0000 7f7bbd7fa700 1 -- 172.21.15.8:0/173881576 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0017650 con 0x7f7bc0100240 2023-11-27T15:23:34.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.739+0000 7f7bc65ff700 1 -- 172.21.15.8:0/173881576 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc010ba60 con 0x7f7bc0100240 2023-11-27T15:23:34.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.743+0000 7f7bbd7fa700 1 -- 172.21.15.8:0/173881576 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7bb0021580 con 0x7f7bc0100240 2023-11-27T15:23:34.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.744+0000 7f7bd9ec1b80 1 Processor -- start 2023-11-27T15:23:34.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.744+0000 7f7bd9ec1b80 1 -- start start 2023-11-27T15:23:34.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.744+0000 7f7bbd7fa700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ba80a1150 0x7f7ba80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3aabb0 0x55810b3aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 0x55810b3ac530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b2b0010 con 0x55810b3ac0c0 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b302850 con 0x55810b3aabb0 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbd7fa700 1 -- 172.21.15.8:0/173881576 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7bb00cdc80 con 0x7f7bc0100240 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b2fbce0 con 0x55810b3ab500 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3aabb0 0x55810b3aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3aabb0 0x55810b3aafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:35738/0 (socket says 172.21.15.8:35738) 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:47486/0 (socket says 172.21.15.8:47486) 2023-11-27T15:23:34.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbffff700 1 -- 172.21.15.8:0/556741369 learned_addr learned my addr 172.21.15.8:0/556741369 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bc4b9c700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ba80a1150 0x7f7ba80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bc4b9c700 1 --2- 172.21.15.8:0/556741369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 0x55810b3ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbffff700 1 -- 172.21.15.8:0/556741369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 msgr2=0x55810b3abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbffff700 1 --2- 172.21.15.8:0/556741369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbffff700 1 -- 172.21.15.8:0/556741369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 msgr2=0x55810b3ac530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbffff700 1 --2- 172.21.15.8:0/556741369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 0x55810b3ac530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbffff700 1 -- 172.21.15.8:0/556741369 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55810b17d5e0 con 0x55810b3aabb0 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.745+0000 7f7bbf7fe700 1 --2- 172.21.15.8:0/556741369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.746+0000 7f7bbffff700 1 --2- 172.21.15.8:0/556741369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3aabb0 0x55810b3aafc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0000c00 tx=0x7f7bb0097450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8de38866756e95 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.746+0000 7f7bc4b9c700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ba80a1150 0x7f7ba80a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009ec0 tx=0x7f7bb8009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.746+0000 7f7ba67fc700 1 -- 172.21.15.8:0/556741369 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb009a950 con 0x55810b3aabb0 2023-11-27T15:23:34.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.746+0000 7f7ba67fc700 1 -- 172.21.15.8:0/556741369 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bb009aff0 con 0x55810b3aabb0 2023-11-27T15:23:34.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.746+0000 7f7ba67fc700 1 -- 172.21.15.8:0/556741369 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb009b250 con 0x55810b3aabb0 2023-11-27T15:23:34.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.747+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/556741369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3aabb0 msgr2=0x55810b3aafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.747+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/556741369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3aabb0 0x55810b3aafc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0000c00 tx=0x7f7bb0097450 comp rx=0 tx=0).stop 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.747+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/556741369 shutdown_connections 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.747+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/556741369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 0x55810b3ac530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.747+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/556741369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3aabb0 0x55810b3aafc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.747+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/556741369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.747+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/556741369 >> 172.21.15.8:0/556741369 conn(0x55810b3a0310 msgr2=0x55810b3a5620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.747+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/556741369 shutdown_connections 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.747+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/556741369 wait complete. 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.747+0000 7f7bd9ec1b80 1 -- start start 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.748+0000 7f7bd9ec1b80 1 Processor -- start 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.748+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 0x55810b3b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ab500 0x55810b3b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 0x55810b3bb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b29d890 con 0x55810b3ac0c0 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b29d530 con 0x55810b3ab500 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b17f8d0 con 0x55810b3aabb0 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bd9ec1b80 10 client.0 _ll_register_callbacks cb 0x55810b2bb730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bc4b9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 0x55810b3bb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bbf7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ab500 0x55810b3b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bc4b9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 0x55810b3bb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37352/0 (socket says 172.21.15.8:37352) 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bbf7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ab500 0x55810b3b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:35752/0 (socket says 172.21.15.8:35752) 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bc4b9c700 1 -- 172.21.15.8:0/1459601036 learned_addr learned my addr 172.21.15.8:0/1459601036 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bbffff700 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 0x55810b3b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bbf7fe700 1 -- 172.21.15.8:0/1459601036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 msgr2=0x55810b3b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bbf7fe700 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 0x55810b3b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bbf7fe700 1 -- 172.21.15.8:0/1459601036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 msgr2=0x55810b3bb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bbf7fe700 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 0x55810b3bb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bbf7fe700 1 -- 172.21.15.8:0/1459601036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc010ba60 con 0x55810b3ab500 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bbffff700 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 0x55810b3b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7bbf7fe700 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ab500 0x55810b3b4750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4012cd0 tx=0x7f7bb4009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d3b6da23c49f33d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.749+0000 7f7b9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.750+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1459601036 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4017070 con 0x55810b3ab500 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.750+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1459601036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55810b39dad0 con 0x55810b3ab500 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.750+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1459601036 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7bb4004030 con 0x55810b3ab500 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.750+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1459601036 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb40041f0 con 0x55810b3ab500 2023-11-27T15:23:34.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.750+0000 7f7bd9ec1b80 10 client.24701 fetch_fsmap learned FSMap version 17 2023-11-27T15:23:34.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.750+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1459601036 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55810b2c2140 con 0x55810b3ab500 2023-11-27T15:23:34.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.750+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1459601036 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55810b39dad0 con 0x55810b3ab500 2023-11-27T15:23:34.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.750+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1459601036 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7bb40039c0 con 0x55810b3ab500 2023-11-27T15:23:34.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.750+0000 7f7bd9ec1b80 10 client.24701 fetch_fsmap finished waiting for FSMap version 17 2023-11-27T15:23:34.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.750+0000 7f7bd9ec1b80 10 client.24701 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-27T15:23:34.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.751+0000 7f7bd9ec1b80 20 client.24701 populate_metadata read hostname 'smithi008' 2023-11-27T15:23:34.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.751+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55810b3bf570 0x55810b3be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.751+0000 7f7bd9ec1b80 4 client.24701 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:34.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.751+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1459601036 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55810b2be050 con 0x55810b3bf570 2023-11-27T15:23:34.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.751+0000 7f7bbffff700 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55810b3bf570 0x55810b3be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.751+0000 7f7bbffff700 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55810b3bf570 0x55810b3be940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead2338ef128d757 server_cookie=16e52d08e44af8bd in_seq=0 out_seq=0 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.751+0000 7f7b9f7fe700 10 client.24701 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.752+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1459601036 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55810b2be050 con 0x55810b3bf570 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.752+0000 7f7b9f7fe700 10 client.24701 handle_command_reply: tid=0 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.752+0000 7f7bd9ec1b80 1 client.24701 shutdown 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.752+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1459601036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ab500 msgr2=0x55810b3b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.752+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ab500 0x55810b3b4750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4012cd0 tx=0x7f7bb4009e20 comp rx=0 tx=0).stop 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.753+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1459601036 shutdown_connections 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.753+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55810b3bf570 0x55810b3be940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.753+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 0x55810b3b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.753+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ac0c0 0x55810b3bb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.753+0000 7f7b9f7fe700 0 client.24701 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.753+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1459601036 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ab500 0x55810b3b4750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.753+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1459601036 >> 172.21.15.8:0/1459601036 conn(0x55810b3a0310 msgr2=0x55810b3a4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.753+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1459601036 shutdown_connections 2023-11-27T15:23:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.753+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1459601036 wait complete. 2023-11-27T15:23:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.753+0000 7f7bd9ec1b80 20 client.24701 trim_cache size 0 max 16384 2023-11-27T15:23:34.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bd9ec1b80 1 Processor -- start 2023-11-27T15:23:34.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bd9ec1b80 1 -- start start 2023-11-27T15:23:34.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3aabb0 0x55810b3c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3bb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 0x55810b3be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b2b0010 con 0x55810b3aabb0 2023-11-27T15:23:34.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b2be050 con 0x55810b3ac0c0 2023-11-27T15:23:34.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b207ed0 con 0x55810b3ab500 2023-11-27T15:23:34.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3aabb0 0x55810b3c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bbffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3aabb0 0x55810b3c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37368/0 (socket says 172.21.15.8:37368) 2023-11-27T15:23:34.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bbffff700 1 -- 172.21.15.8:0/2855430133 learned_addr learned my addr 172.21.15.8:0/2855430133 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:34.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bc4b9c700 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 0x55810b3be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.772+0000 7f7bbf7fe700 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3bb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7bbffff700 1 -- 172.21.15.8:0/2855430133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 msgr2=0x55810b3bb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:34.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7bbffff700 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3bb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7bbffff700 1 -- 172.21.15.8:0/2855430133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 msgr2=0x55810b3be940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:34.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7bbffff700 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 0x55810b3be940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7bbffff700 1 -- 172.21.15.8:0/2855430133 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55810b2c1e90 con 0x55810b3aabb0 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7bc4b9c700 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 0x55810b3be940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7bbf7fe700 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7bbffff700 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3aabb0 0x55810b3c82e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0000c00 tx=0x7f7bb0021c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f4ee46f6c03ff server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7ba6ffd700 1 -- 172.21.15.8:0/2855430133 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00b2070 con 0x55810b3aabb0 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7ba6ffd700 1 -- 172.21.15.8:0/2855430133 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bb009f0a0 con 0x55810b3aabb0 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.773+0000 7f7ba6ffd700 1 -- 172.21.15.8:0/2855430133 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb009a4e0 con 0x55810b3aabb0 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/2855430133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3aabb0 msgr2=0x55810b3c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3aabb0 0x55810b3c82e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0000c00 tx=0x7f7bb0021c20 comp rx=0 tx=0).stop 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/2855430133 shutdown_connections 2023-11-27T15:23:34.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 0x55810b3be940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3aabb0 0x55810b3c82e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/2855430133 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3ab500 0x55810b3bb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/2855430133 >> 172.21.15.8:0/2855430133 conn(0x55810b3a0310 msgr2=0x55810b3a55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/2855430133 shutdown_connections 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/2855430133 wait complete. 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 -- start start 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.774+0000 7f7bd9ec1b80 1 Processor -- start 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 0x55810b3b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ab500 0x55810b3bb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bd9ec1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 0x55810b3b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b3b6c10 con 0x55810b3ab500 2023-11-27T15:23:34.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b413410 con 0x55810b3ac0c0 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bd9ec1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810b413550 con 0x55810b3aabb0 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bd9ec1b80 10 client.0 _ll_register_callbacks cb 0x7f7bb40127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bbf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ab500 0x55810b3bb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bbf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ab500 0x55810b3bb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37382/0 (socket says 172.21.15.8:37382) 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bbf7fe700 1 -- 172.21.15.8:0/1253288028 learned_addr learned my addr 172.21.15.8:0/1253288028 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bbffff700 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 0x55810b3b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bc4b9c700 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 0x55810b3b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bbf7fe700 1 -- 172.21.15.8:0/1253288028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 msgr2=0x55810b3b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bbf7fe700 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 0x55810b3b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bbf7fe700 1 -- 172.21.15.8:0/1253288028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 msgr2=0x55810b3b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bbf7fe700 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 0x55810b3b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bbf7fe700 1 -- 172.21.15.8:0/1253288028 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55810b17d5e0 con 0x55810b3ab500 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.775+0000 7f7bbffff700 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 0x55810b3b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:34.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7bbf7fe700 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ab500 0x55810b3bb560 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4000c00 tx=0x7f7bb40104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c9a2432e53e9aa server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7b9f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1253288028 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4017040 con 0x55810b3ab500 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1253288028 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7bb4003680 con 0x55810b3ab500 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1253288028 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bb009aff0 con 0x55810b3ab500 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1253288028 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb4010a60 con 0x55810b3ab500 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7bd9ec1b80 10 client.14679 fetch_fsmap learned FSMap version 17 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1253288028 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55810b112c10 con 0x55810b3ab500 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1253288028 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bb009aff0 con 0x55810b3ab500 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1253288028 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7bb4005150 con 0x55810b3ab500 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7bd9ec1b80 10 client.14679 fetch_fsmap finished waiting for FSMap version 17 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7bd9ec1b80 10 client.14679 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-27T15:23:34.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.776+0000 7f7bd9ec1b80 20 client.14679 populate_metadata read hostname 'smithi008' 2023-11-27T15:23:34.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.777+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55810b4182d0 0x55810b414c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:34.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.777+0000 7f7bd9ec1b80 4 client.14679 mds_command: new command op to 24595 tid=0[{"prefix": "status"}] 2023-11-27T15:23:34.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.777+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1253288028 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55810b2ca610 con 0x55810b4182d0 2023-11-27T15:23:34.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.777+0000 7f7bbffff700 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55810b4182d0 0x55810b414c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.777+0000 7f7bbffff700 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55810b4182d0 0x55810b414c40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70bfb7f9b2919684 server_cookie=2496cdcd3ee80d16 in_seq=0 out_seq=0 2023-11-27T15:23:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.777+0000 7f7b9f7fe700 10 client.14679 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:23:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.777+0000 7f7b9f7fe700 1 -- 172.21.15.8:0/1253288028 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55810b2ca610 con 0x55810b4182d0 2023-11-27T15:23:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.777+0000 7f7b9f7fe700 10 client.14679 handle_command_reply: tid=0 2023-11-27T15:23:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.777+0000 7f7bd9ec1b80 1 client.14679 shutdown 2023-11-27T15:23:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1253288028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ab500 msgr2=0x55810b3bb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ab500 0x55810b3bb560 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4000c00 tx=0x7f7bb40104a0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1253288028 shutdown_connections 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55810b4182d0 0x55810b414c40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55810b3aabb0 0x55810b3b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55810b3ac0c0 0x55810b3b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 1 --2- 172.21.15.8:0/1253288028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55810b3ab500 0x55810b3bb560 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7b9f7fe700 0 client.14679 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1253288028 >> 172.21.15.8:0/1253288028 conn(0x55810b3a0310 msgr2=0x55810b3c82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1253288028 shutdown_connections 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 1 -- 172.21.15.8:0/1253288028 wait complete. 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bd9ec1b80 20 client.14679 trim_cache size 0 max 16384 2023-11-27T15:23:34.788 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 0, 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24595, 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 7.6078385559999999, 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 17, 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 88, 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 87, 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 16.772102481000001 2023-11-27T15:23:34.789 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:23:34.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bc65ff700 1 -- 172.21.15.8:0/173881576 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ba80a1150 msgr2=0x7f7ba80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bc65ff700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ba80a1150 0x7f7ba80a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009ec0 tx=0x7f7bb8009920 comp rx=0 tx=0).stop 2023-11-27T15:23:34.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bc65ff700 1 -- 172.21.15.8:0/173881576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc0100240 msgr2=0x7f7bc01981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:34.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.778+0000 7f7bc65ff700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc0100240 0x7f7bc01981e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0005740 tx=0x7f7bb0005770 comp rx=0 tx=0).stop 2023-11-27T15:23:34.790 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.779+0000 7f7bc65ff700 1 -- 172.21.15.8:0/173881576 shutdown_connections 2023-11-27T15:23:34.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.779+0000 7f7bc65ff700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ba80a1150 0x7f7ba80a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.779+0000 7f7bc65ff700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7bc0101dc0 0x7f7bc019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.779+0000 7f7bc65ff700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7bc0100240 0x7f7bc01981e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.779+0000 7f7bc65ff700 1 --2- 172.21.15.8:0/173881576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bc0101430 0x7f7bc0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:34.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.779+0000 7f7bc65ff700 1 -- 172.21.15.8:0/173881576 >> 172.21.15.8:0/173881576 conn(0x7f7bc00fb820 msgr2=0x7f7bc00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:34.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.779+0000 7f7bc65ff700 1 -- 172.21.15.8:0/173881576 shutdown_connections 2023-11-27T15:23:34.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:34.779+0000 7f7bc65ff700 1 -- 172.21.15.8:0/173881576 wait complete. 2023-11-27T15:23:34.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:34 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3129342884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:34.799 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:34 smithi008 ceph-mon[128788]: pgmap v286: 129 pgs: 17 creating+peering, 109 active+clean, 3 unknown; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 4 op/s 2023-11-27T15:23:34.800 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:34 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4156351521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:34.802 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-27T15:23:34.802 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:23:34.802 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph fs ls 2023-11-27T15:23:34.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:34 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3129342884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:34.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:34 smithi029 ceph-mon[131166]: pgmap v286: 129 pgs: 17 creating+peering, 109 active+clean, 3 unknown; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 4 op/s 2023-11-27T15:23:34.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:34 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/4156351521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:35.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:34 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3129342884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:35.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:34 smithi060 ceph-mon[131518]: pgmap v286: 129 pgs: 17 creating+peering, 109 active+clean, 3 unknown; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 4 op/s 2023-11-27T15:23:35.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:34 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/4156351521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:35.657 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:23:37.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.145+0000 7f17957e1700 1 -- 172.21.15.8:0/3621462022 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1790100ef0 msgr2=0x7f1790104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:37.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.145+0000 7f17957e1700 1 --2- 172.21.15.8:0/3621462022 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1790100ef0 0x7f1790104ec0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f17840099c0 tx=0x7f1784009cd0 comp rx=0 tx=0).stop 2023-11-27T15:23:37.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.145+0000 7f17957e1700 1 -- 172.21.15.8:0/3621462022 shutdown_connections 2023-11-27T15:23:37.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.145+0000 7f17957e1700 1 --2- 172.21.15.8:0/3621462022 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17901055f0 0x7f179010d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:37.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.145+0000 7f17957e1700 1 --2- 172.21.15.8:0/3621462022 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1790100550 0x7f1790100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:37.151 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.145+0000 7f17957e1700 1 --2- 172.21.15.8:0/3621462022 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1790100ef0 0x7f1790104ec0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:37.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.145+0000 7f17957e1700 1 -- 172.21.15.8:0/3621462022 >> 172.21.15.8:0/3621462022 conn(0x7f17900fbdf0 msgr2=0x7f17900fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:37.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.145+0000 7f17957e1700 1 -- 172.21.15.8:0/3621462022 shutdown_connections 2023-11-27T15:23:37.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.145+0000 7f17957e1700 1 -- 172.21.15.8:0/3621462022 wait complete. 2023-11-27T15:23:37.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.146+0000 7f17957e1700 1 Processor -- start 2023-11-27T15:23:37.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f17957e1700 1 -- start start 2023-11-27T15:23:37.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f17957e1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1790100550 0x7f17901963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:37.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f17957e1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1790100ef0 0x7f17901968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:37.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f17957e1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17901055f0 0x7f179019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:37.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f17957e1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179019b370 con 0x7f1790100ef0 2023-11-27T15:23:37.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f17957e1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179019b4e0 con 0x7f1790100550 2023-11-27T15:23:37.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f17957e1700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179019b6a0 con 0x7f17901055f0 2023-11-27T15:23:37.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f178e7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1790100ef0 0x7f17901968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:37.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f178e7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1790100ef0 0x7f17901968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37396/0 (socket says 172.21.15.8:37396) 2023-11-27T15:23:37.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f178e7fc700 1 -- 172.21.15.8:0/3042457294 learned_addr learned my addr 172.21.15.8:0/3042457294 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:37.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f178effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1790100550 0x7f17901963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:37.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.147+0000 7f178f7fe700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17901055f0 0x7f179019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:37.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f178e7fc700 1 -- 172.21.15.8:0/3042457294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17901055f0 msgr2=0x7f179019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:37.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f178e7fc700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17901055f0 0x7f179019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:37.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f178e7fc700 1 -- 172.21.15.8:0/3042457294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1790100550 msgr2=0x7f17901963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:37.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f178e7fc700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1790100550 0x7f17901963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:37.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f178e7fc700 1 -- 172.21.15.8:0/3042457294 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840095f0 con 0x7f1790100ef0 2023-11-27T15:23:37.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f178effd700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1790100550 0x7f17901963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:37.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f178e7fc700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1790100ef0 0x7f17901968f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f17840098e0 tx=0x7f178400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:37.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f1777fff700 1 -- 172.21.15.8:0/3042457294 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178401d070 con 0x7f1790100ef0 2023-11-27T15:23:37.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f1777fff700 1 -- 172.21.15.8:0/3042457294 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1784005510 con 0x7f1790100ef0 2023-11-27T15:23:37.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f1777fff700 1 -- 172.21.15.8:0/3042457294 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784004a00 con 0x7f1790100ef0 2023-11-27T15:23:37.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f17957e1700 1 -- 172.21.15.8:0/3042457294 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1790110370 con 0x7f1790100ef0 2023-11-27T15:23:37.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.148+0000 7f17957e1700 1 -- 172.21.15.8:0/3042457294 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901101e0 con 0x7f1790100ef0 2023-11-27T15:23:37.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.150+0000 7f17957e1700 1 -- 172.21.15.8:0/3042457294 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1790110500 con 0x7f1790100ef0 2023-11-27T15:23:37.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.151+0000 7f1777fff700 1 -- 172.21.15.8:0/3042457294 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1784004be0 con 0x7f1790100ef0 2023-11-27T15:23:37.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.154+0000 7f1777fff700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f17780a11d0 0x7f17780a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:37.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.154+0000 7f1777fff700 1 -- 172.21.15.8:0/3042457294 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f17840cd830 con 0x7f1790100ef0 2023-11-27T15:23:37.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.154+0000 7f178effd700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f17780a11d0 0x7f17780a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:37.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.154+0000 7f178effd700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f17780a11d0 0x7f17780a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1780011490 tx=0x7f1780009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:37.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.158+0000 7f1777fff700 1 -- 172.21.15.8:0/3042457294 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1784096dc0 con 0x7f1790100ef0 2023-11-27T15:23:37.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.413+0000 7f17957e1700 1 -- 172.21.15.8:0/3042457294 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f17901a3430 con 0x7f1790100ef0 2023-11-27T15:23:37.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.413+0000 7f1777fff700 1 -- 172.21.15.8:0/3042457294 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f178409a3e0 con 0x7f1790100ef0 2023-11-27T15:23:37.419 INFO:teuthology.orchestra.run.smithi008.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-27T15:23:37.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.415+0000 7f17957e1700 1 -- 172.21.15.8:0/3042457294 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f17780a11d0 msgr2=0x7f17780a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:37.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.415+0000 7f17957e1700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f17780a11d0 0x7f17780a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1780011490 tx=0x7f1780009260 comp rx=0 tx=0).stop 2023-11-27T15:23:37.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.416+0000 7f17957e1700 1 -- 172.21.15.8:0/3042457294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1790100ef0 msgr2=0x7f17901968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:37.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.416+0000 7f17957e1700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1790100ef0 0x7f17901968f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f17840098e0 tx=0x7f178400f690 comp rx=0 tx=0).stop 2023-11-27T15:23:37.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.416+0000 7f17957e1700 1 -- 172.21.15.8:0/3042457294 shutdown_connections 2023-11-27T15:23:37.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.416+0000 7f17957e1700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f17780a11d0 0x7f17780a3680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:37.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.416+0000 7f17957e1700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f17901055f0 0x7f179019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:37.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.416+0000 7f17957e1700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1790100550 0x7f17901963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:37.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.416+0000 7f17957e1700 1 --2- 172.21.15.8:0/3042457294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1790100ef0 0x7f17901968f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:37.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.416+0000 7f17957e1700 1 -- 172.21.15.8:0/3042457294 >> 172.21.15.8:0/3042457294 conn(0x7f17900fbdf0 msgr2=0x7f17900fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:37.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.416+0000 7f17957e1700 1 -- 172.21.15.8:0/3042457294 shutdown_connections 2023-11-27T15:23:37.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:37.416+0000 7f17957e1700 1 -- 172.21.15.8:0/3042457294 wait complete. 2023-11-27T15:23:37.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:37 smithi029 ceph-mon[131166]: pgmap v287: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2023-11-27T15:23:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:37 smithi060 ceph-mon[131518]: pgmap v287: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2023-11-27T15:23:37.836 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:37 smithi008 ceph-mon[128788]: pgmap v287: 129 pgs: 6 creating+peering, 123 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2023-11-27T15:23:37.839 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-27T15:23:37.839 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-27T15:23:37.839 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi008.front.sepia.ceph.com 2023-11-27T15:23:37.839 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-27T15:23:37.839 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:23:37.839 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-27T15:23:37.839 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-27T15:23:37.839 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-27T15:23:37.839 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-27T15:23:37.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:37.840 DEBUG:teuthology.orchestra.run.smithi008:> ip addr 2023-11-27T15:23:37.886 INFO:teuthology.orchestra.run.smithi008.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-27T15:23:37.886 INFO:teuthology.orchestra.run.smithi008.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-27T15:23:37.886 INFO:teuthology.orchestra.run.smithi008.stdout: inet 127.0.0.1/8 scope host lo 2023-11-27T15:23:37.886 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:23:37.886 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 ::1/128 scope host 2023-11-27T15:23:37.886 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:23:37.886 INFO:teuthology.orchestra.run.smithi008.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:23:37.886 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2e brd ff:ff:ff:ff:ff:ff 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f0 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2f brd ff:ff:ff:ff:ff:ff 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f1 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:fe brd ff:ff:ff:ff:ff:ff 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f0 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout: inet 172.21.15.8/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft 41568sec preferred_lft 41568sec 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 fe80::ae1f:6bff:fef8:25fe/64 scope link 2023-11-27T15:23:37.887 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:23:37.888 INFO:teuthology.orchestra.run.smithi008.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:23:37.888 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:ff brd ff:ff:ff:ff:ff:ff 2023-11-27T15:23:37.888 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f1 2023-11-27T15:23:37.888 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-27T15:23:37.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:37.889 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-11-27T15:23:37.889 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-11-27T15:23:37.889 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add name ceph-brx type bridge 2023-11-27T15:23:37.889 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr flush dev ceph-brx 2023-11-27T15:23:37.889 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set ceph-brx up 2023-11-27T15:23:37.889 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-27T15:23:37.889 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-11-27T15:23:38.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:38.362 DEBUG:teuthology.orchestra.run.smithi008:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-27T15:23:38.437 INFO:teuthology.orchestra.run.smithi008.stdout:1 2023-11-27T15:23:38.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:38.441 DEBUG:teuthology.orchestra.run.smithi008:> route 2023-11-27T15:23:38.512 INFO:teuthology.orchestra.run.smithi008.stdout:Kernel IP routing table 2023-11-27T15:23:38.512 INFO:teuthology.orchestra.run.smithi008.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-27T15:23:38.512 INFO:teuthology.orchestra.run.smithi008.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-27T15:23:38.512 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-27T15:23:38.512 INFO:teuthology.orchestra.run.smithi008.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-27T15:23:38.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:38.513 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-11-27T15:23:38.513 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-11-27T15:23:38.513 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-27T15:23:38.513 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-27T15:23:38.513 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-27T15:23:38.513 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-11-27T15:23:38.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:38 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3042457294' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:23:38.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:38 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3042457294' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:23:38.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:38 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3042457294' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:23:39.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:39.260 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2023-11-27T15:23:39.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:39.305 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list-id 2023-11-27T15:23:39.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:39.389 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-11-27T15:23:39.389 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-11-27T15:23:39.389 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-27T15:23:39.389 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-27T15:23:39.390 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-11-27T15:23:39.532 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:39 smithi008 ceph-mon[128788]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 2 op/s 2023-11-27T15:23:39.576 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-27T15:23:39.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:39.576 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-11-27T15:23:39.576 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-11-27T15:23:39.576 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-27T15:23:39.577 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-27T15:23:39.577 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-27T15:23:39.577 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-27T15:23:39.577 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-27T15:23:39.577 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-11-27T15:23:39.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:39 smithi029 ceph-mon[131166]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 2 op/s 2023-11-27T15:23:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:39 smithi060 ceph-mon[131518]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 2 op/s 2023-11-27T15:23:39.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:39.872 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-11-27T15:23:39.872 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-11-27T15:23:39.872 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set brx.0 up 2023-11-27T15:23:39.872 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set dev brx.0 master ceph-brx 2023-11-27T15:23:39.872 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-11-27T15:23:40.047 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-27T15:23:40.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-27T15:23:40.047 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:23:40.091 INFO:teuthology.orchestra.run.smithi008.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-27T15:23:40.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-27T15:23:40.092 DEBUG:teuthology.orchestra.run.smithi008:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:23:40.174 DEBUG:teuthology.orchestra.run.smithi008:> sudo modprobe fuse 2023-11-27T15:23:40.287 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-27T15:23:40.287 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-27T15:23:40.356 INFO:teuthology.orchestra.run.smithi008.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-27T15:23:40.358 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-27T15:23:40.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:40.359 DEBUG:teuthology.orchestra.run.smithi008:> ls /sys/fs/fuse/connections 2023-11-27T15:23:40.403 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-27T15:23:40.404 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-11-27T15:23:40.448 DEBUG:teuthology.orchestra.run.smithi008:> sudo modprobe fuse 2023-11-27T15:23:40.521 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-27T15:23:40.521 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-27T15:23:40.592 INFO:teuthology.orchestra.run.smithi008.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-27T15:23:40.595 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-27T15:23:40.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:40.595 DEBUG:teuthology.orchestra.run.smithi008:> ls /sys/fs/fuse/connections 2023-11-27T15:23:40.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi008.stderr:2023-11-27T15:23:40.656+0000 7f5fa4f6a5c0 -1 init, newargv = 0x55b4bca3e2d0 newargc=15 2023-11-27T15:23:40.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi008.stderr:ceph-fuse[165782]: starting ceph client 2023-11-27T15:23:40.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi008.stderr:ceph-fuse[165782]: starting fuse 2023-11-27T15:23:41.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:41 smithi029 ceph-mon[131166]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 823 B/s wr, 0 op/s 2023-11-27T15:23:41.741 DEBUG:teuthology.orchestra.run.smithi008:> sudo modprobe fuse 2023-11-27T15:23:41.809 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:41 smithi008 ceph-mon[128788]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 823 B/s wr, 0 op/s 2023-11-27T15:23:41.820 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-27T15:23:41.820 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-27T15:23:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:41 smithi060 ceph-mon[131518]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 823 B/s wr, 0 op/s 2023-11-27T15:23:41.890 INFO:teuthology.orchestra.run.smithi008.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-27T15:23:41.893 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-27T15:23:41.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:41.894 DEBUG:teuthology.orchestra.run.smithi008:> ls /sys/fs/fuse/connections 2023-11-27T15:23:41.938 INFO:teuthology.orchestra.run.smithi008.stdout:121 2023-11-27T15:23:41.938 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> import glob 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> import re 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> import os 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> import subprocess 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> def _find_admin_socket(client_name): 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> files = glob.glob(asok_path) 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-27T15:23:41.939 DEBUG:teuthology.orchestra.run.smithi008:> 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> # Given a non-glob path, it better be there 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> if "*" not in asok_path: 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> assert(len(files) == 1) 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> return files[0] 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> for f in files: 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> contents = proc_f.read() 2023-11-27T15:23:41.940 DEBUG:teuthology.orchestra.run.smithi008:> if mountpoint in contents: 2023-11-27T15:23:41.941 DEBUG:teuthology.orchestra.run.smithi008:> return f 2023-11-27T15:23:41.941 DEBUG:teuthology.orchestra.run.smithi008:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-27T15:23:41.941 DEBUG:teuthology.orchestra.run.smithi008:> 2023-11-27T15:23:41.941 DEBUG:teuthology.orchestra.run.smithi008:> print(_find_admin_socket("client.admin")) 2023-11-27T15:23:41.941 DEBUG:teuthology.orchestra.run.smithi008:> ' 2023-11-27T15:23:42.126 INFO:teuthology.orchestra.run.smithi008.stdout:/var/run/ceph/ceph-client.admin.165782.asok 2023-11-27T15:23:42.302 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.165782.asok 2023-11-27T15:23:42.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:42.303 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.165782.asok status 2023-11-27T15:23:42.536 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:23:42.536 INFO:teuthology.orchestra.run.smithi008.stdout: "metadata": { 2023-11-27T15:23:42.536 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_sha1": "1c9565146c8647d4e00eb6c7ae626dc192680e7b", 2023-11-27T15:23:42.536 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:23:42.536 INFO:teuthology.orchestra.run.smithi008.stdout: "entity_id": "admin", 2023-11-27T15:23:42.536 INFO:teuthology.orchestra.run.smithi008.stdout: "hostname": "smithi008", 2023-11-27T15:23:42.536 INFO:teuthology.orchestra.run.smithi008.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-27T15:23:42.536 INFO:teuthology.orchestra.run.smithi008.stdout: "pid": "165782", 2023-11-27T15:23:42.537 INFO:teuthology.orchestra.run.smithi008.stdout: "root": "/" 2023-11-27T15:23:42.537 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:23:42.537 INFO:teuthology.orchestra.run.smithi008.stdout: "dentry_count": 0, 2023-11-27T15:23:42.537 INFO:teuthology.orchestra.run.smithi008.stdout: "dentry_pinned_count": 0, 2023-11-27T15:23:42.537 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 14691, 2023-11-27T15:23:42.537 INFO:teuthology.orchestra.run.smithi008.stdout: "inst": { 2023-11-27T15:23:42.537 INFO:teuthology.orchestra.run.smithi008.stdout: "name": { 2023-11-27T15:23:42.537 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "client", 2023-11-27T15:23:42.538 INFO:teuthology.orchestra.run.smithi008.stdout: "num": 14691 2023-11-27T15:23:42.538 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:23:42.538 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": { 2023-11-27T15:23:42.538 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v1", 2023-11-27T15:23:42.538 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "192.168.0.1:0", 2023-11-27T15:23:42.538 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 2729948751 2023-11-27T15:23:42.538 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:23:42.538 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:23:42.539 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": { 2023-11-27T15:23:42.539 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v1", 2023-11-27T15:23:42.539 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "192.168.0.1:0", 2023-11-27T15:23:42.539 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 2729948751 2023-11-27T15:23:42.539 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:23:42.539 INFO:teuthology.orchestra.run.smithi008.stdout: "inst_str": "client.14691 192.168.0.1:0/2729948751", 2023-11-27T15:23:42.539 INFO:teuthology.orchestra.run.smithi008.stdout: "addr_str": "192.168.0.1:0/2729948751", 2023-11-27T15:23:42.539 INFO:teuthology.orchestra.run.smithi008.stdout: "inode_count": 1, 2023-11-27T15:23:42.539 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_epoch": 17, 2023-11-27T15:23:42.539 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_epoch": 88, 2023-11-27T15:23:42.540 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_epoch_barrier": 0, 2023-11-27T15:23:42.540 INFO:teuthology.orchestra.run.smithi008.stdout: "blocklisted": false, 2023-11-27T15:23:42.540 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs" 2023-11-27T15:23:42.540 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:23:42.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:42.551 DEBUG:teuthology.orchestra.run.smithi008:> stat --file-system '--printf=%T 2023-11-27T15:23:42.551 DEBUG:teuthology.orchestra.run.smithi008:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:23:42.595 INFO:teuthology.orchestra.run.smithi008.stdout:fuseblk 2023-11-27T15:23:42.595 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:23:42.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:42.596 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:23:42.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:42.707 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-27T15:23:42.832 INFO:teuthology.orchestra.run.smithi008.stdout:# file: . 2023-11-27T15:23:42.832 INFO:teuthology.orchestra.run.smithi008.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-27T15:23:42.832 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:42.833 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:23:42.833 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:23:42.879 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:23:43.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.120+0000 7f66c61ff700 1 -- 172.21.15.8:0/1230140208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0102530 msgr2=0x7f66c0102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:43.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.120+0000 7f66c61ff700 1 --2- 172.21.15.8:0/1230140208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0102530 0x7f66c0102940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009990 tx=0x7f66b0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.120+0000 7f66c61ff700 1 -- 172.21.15.8:0/1230140208 shutdown_connections 2023-11-27T15:23:43.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.120+0000 7f66c61ff700 1 --2- 172.21.15.8:0/1230140208 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c01040b0 0x7f66c010a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.120+0000 7f66c61ff700 1 --2- 172.21.15.8:0/1230140208 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0102530 0x7f66c0102940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.120+0000 7f66c61ff700 1 --2- 172.21.15.8:0/1230140208 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66c0103720 0x7f66c0103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.120+0000 7f66c61ff700 1 -- 172.21.15.8:0/1230140208 >> 172.21.15.8:0/1230140208 conn(0x7f66c00fdb30 msgr2=0x7f66c00fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:43.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.120+0000 7f66c61ff700 1 -- 172.21.15.8:0/1230140208 shutdown_connections 2023-11-27T15:23:43.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.120+0000 7f66c61ff700 1 -- 172.21.15.8:0/1230140208 wait complete. 2023-11-27T15:23:43.128 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.122+0000 7f66c61ff700 1 Processor -- start 2023-11-27T15:23:43.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.122+0000 7f66c61ff700 1 -- start start 2023-11-27T15:23:43.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.122+0000 7f66c61ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66c0102530 0x7f66c01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:43.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66c61ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0103720 0x7f66c0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:43.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66c61ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c01040b0 0x7f66c019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:43.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66c61ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c019fe60 con 0x7f66c0102530 2023-11-27T15:23:43.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66c61ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c019ffa0 con 0x7f66c0103720 2023-11-27T15:23:43.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66c61ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c01a0130 con 0x7f66c01040b0 2023-11-27T15:23:43.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66bf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66c0102530 0x7f66c01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:43.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66bf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66c0102530 0x7f66c01982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49982/0 (socket says 172.21.15.8:49982) 2023-11-27T15:23:43.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66bf7fe700 1 -- 172.21.15.8:0/2827685154 learned_addr learned my addr 172.21.15.8:0/2827685154 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:43.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66b7fff700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0103720 0x7f66c0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:43.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66bffff700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c01040b0 0x7f66c019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:43.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66bf7fe700 1 -- 172.21.15.8:0/2827685154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c01040b0 msgr2=0x7f66c019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:43.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66bf7fe700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c01040b0 0x7f66c019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66bf7fe700 1 -- 172.21.15.8:0/2827685154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0103720 msgr2=0x7f66c0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:43.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66bf7fe700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0103720 0x7f66c0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.123+0000 7f66bf7fe700 1 -- 172.21.15.8:0/2827685154 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b00095f0 con 0x7f66c0102530 2023-11-27T15:23:43.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.124+0000 7f66bf7fe700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66c0102530 0x7f66c01982e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f66b0000c00 tx=0x7f66b00047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:43.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.124+0000 7f66bd7fa700 1 -- 172.21.15.8:0/2827685154 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b001d070 con 0x7f66c0102530 2023-11-27T15:23:43.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.124+0000 7f66bd7fa700 1 -- 172.21.15.8:0/2827685154 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66b0022470 con 0x7f66c0102530 2023-11-27T15:23:43.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.124+0000 7f66c61ff700 1 -- 172.21.15.8:0/2827685154 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c010de90 con 0x7f66c0102530 2023-11-27T15:23:43.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.124+0000 7f66bd7fa700 1 -- 172.21.15.8:0/2827685154 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0017650 con 0x7f66c0102530 2023-11-27T15:23:43.131 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.124+0000 7f66c61ff700 1 -- 172.21.15.8:0/2827685154 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c010dd00 con 0x7f66c0102530 2023-11-27T15:23:43.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.125+0000 7f66c61ff700 1 -- 172.21.15.8:0/2827685154 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c0109ad0 con 0x7f66c0102530 2023-11-27T15:23:43.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.127+0000 7f66bd7fa700 1 -- 172.21.15.8:0/2827685154 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f66b00225e0 con 0x7f66c0102530 2023-11-27T15:23:43.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.128+0000 7f66bd7fa700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66ac0a1150 0x7f66ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:43.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.128+0000 7f66bd7fa700 1 -- 172.21.15.8:0/2827685154 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f66b00ce150 con 0x7f66c0102530 2023-11-27T15:23:43.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.129+0000 7f66b7fff700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66ac0a1150 0x7f66ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:43.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.129+0000 7f66b7fff700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66ac0a1150 0x7f66ac0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009c60 tx=0x7f66a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:43.136 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.132+0000 7f66bd7fa700 1 -- 172.21.15.8:0/2827685154 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f66b00976e0 con 0x7f66c0102530 2023-11-27T15:23:43.354 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.351+0000 7f66c61ff700 1 -- 172.21.15.8:0/2827685154 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66c002cfe0 con 0x7f66c0102530 2023-11-27T15:23:43.356 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.352+0000 7f66bd7fa700 1 -- 172.21.15.8:0/2827685154 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23817 (secure 0 0 0) 0x7f66b009ad00 con 0x7f66c0102530 2023-11-27T15:23:43.357 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:43.357 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":88,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:23:32.146115+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:23:43.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 -- 172.21.15.8:0/2827685154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66ac0a1150 msgr2=0x7f66ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:43.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66ac0a1150 0x7f66ac0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009c60 tx=0x7f66a8009260 comp rx=0 tx=0).stop 2023-11-27T15:23:43.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 -- 172.21.15.8:0/2827685154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66c0102530 msgr2=0x7f66c01982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:43.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66c0102530 0x7f66c01982e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f66b0000c00 tx=0x7f66b00047a0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.359 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 -- 172.21.15.8:0/2827685154 shutdown_connections 2023-11-27T15:23:43.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66ac0a1150 0x7f66ac0a3600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66c01040b0 0x7f66c019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66c0102530 0x7f66c01982e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 --2- 172.21.15.8:0/2827685154 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66c0103720 0x7f66c0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 -- 172.21.15.8:0/2827685154 >> 172.21.15.8:0/2827685154 conn(0x7f66c00fdb30 msgr2=0x7f66c0106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:43.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 -- 172.21.15.8:0/2827685154 shutdown_connections 2023-11-27T15:23:43.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.355+0000 7f66c61ff700 1 -- 172.21.15.8:0/2827685154 wait complete. 2023-11-27T15:23:43.378 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:23:25.166148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:23:26.584539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:23:30.130965+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:23:43.378 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:23:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.583+0000 7f801a1ff700 1 -- 172.21.15.8:0/3777706766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8014101480 msgr2=0x7f80141018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.583+0000 7f801a1ff700 1 --2- 172.21.15.8:0/3777706766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8014101480 0x7f80141018d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0098f0 tx=0x7f7ffc009c00 comp rx=0 tx=0).stop 2023-11-27T15:23:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.583+0000 7f801a1ff700 1 -- 172.21.15.8:0/3777706766 shutdown_connections 2023-11-27T15:23:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.583+0000 7f801a1ff700 1 --2- 172.21.15.8:0/3777706766 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8014100290 0x7f80141006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.583+0000 7f801a1ff700 1 --2- 172.21.15.8:0/3777706766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8014101e10 0x7f80141086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.583+0000 7f801a1ff700 1 --2- 172.21.15.8:0/3777706766 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8014101480 0x7f80141018d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.583+0000 7f801a1ff700 1 -- 172.21.15.8:0/3777706766 >> 172.21.15.8:0/3777706766 conn(0x7f80140fb870 msgr2=0x7f80140fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.583+0000 7f801a1ff700 1 -- 172.21.15.8:0/3777706766 shutdown_connections 2023-11-27T15:23:43.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.583+0000 7f801a1ff700 1 -- 172.21.15.8:0/3777706766 wait complete. 2023-11-27T15:23:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.584+0000 7f801a1ff700 1 Processor -- start 2023-11-27T15:23:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.584+0000 7f801a1ff700 1 -- start start 2023-11-27T15:23:43.590 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.584+0000 7f801a1ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8014100290 0x7f8014198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.584+0000 7f801a1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8014101480 0x7f8014198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f801a1ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8014101e10 0x7f801419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f801a1ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801419fdb0 con 0x7f8014100290 2023-11-27T15:23:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f801a1ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801419fef0 con 0x7f8014101e10 2023-11-27T15:23:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f801a1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80141a0080 con 0x7f8014101480 2023-11-27T15:23:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f80137fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8014100290 0x7f8014198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f800bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8014101480 0x7f8014198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f8013fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8014101e10 0x7f801419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:43.591 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f80137fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8014100290 0x7f8014198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49996/0 (socket says 172.21.15.8:49996) 2023-11-27T15:23:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f800bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8014101480 0x7f8014198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:45614/0 (socket says 172.21.15.8:45614) 2023-11-27T15:23:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f80137fe700 1 -- 172.21.15.8:0/3753172564 learned_addr learned my addr 172.21.15.8:0/3753172564 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f800bfff700 1 -- 172.21.15.8:0/3753172564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8014101e10 msgr2=0x7f801419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f800bfff700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8014101e10 0x7f801419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f800bfff700 1 -- 172.21.15.8:0/3753172564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8014100290 msgr2=0x7f8014198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f800bfff700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8014100290 0x7f8014198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f800bfff700 1 -- 172.21.15.8:0/3753172564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c009d00 con 0x7f8014101480 2023-11-27T15:23:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f80137fe700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8014100290 0x7f8014198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:43.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.585+0000 7f800bfff700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8014101480 0x7f8014198770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009230 tx=0x7f7ffc00b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.586+0000 7f80117fa700 1 -- 172.21.15.8:0/3753172564 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc01c070 con 0x7f8014101480 2023-11-27T15:23:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.586+0000 7f80117fa700 1 -- 172.21.15.8:0/3753172564 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ffc00f7f0 con 0x7f8014101480 2023-11-27T15:23:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.586+0000 7f801a1ff700 1 -- 172.21.15.8:0/3753172564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ffc009520 con 0x7f8014101480 2023-11-27T15:23:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.586+0000 7f80117fa700 1 -- 172.21.15.8:0/3753172564 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc003990 con 0x7f8014101480 2023-11-27T15:23:43.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.586+0000 7f801a1ff700 1 -- 172.21.15.8:0/3753172564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801410bc50 con 0x7f8014101480 2023-11-27T15:23:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.587+0000 7f801a1ff700 1 -- 172.21.15.8:0/3753172564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8014067a50 con 0x7f8014101480 2023-11-27T15:23:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.589+0000 7f80117fa700 1 -- 172.21.15.8:0/3753172564 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7ffc020030 con 0x7f8014101480 2023-11-27T15:23:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.590+0000 7f80117fa700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80000a1150 0x7f80000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:43.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.590+0000 7f80117fa700 1 -- 172.21.15.8:0/3753172564 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7ffc0cd9b0 con 0x7f8014101480 2023-11-27T15:23:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.590+0000 7f80137fe700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80000a1150 0x7f80000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.591+0000 7f80137fe700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80000a1150 0x7f80000a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8004005950 tx=0x7f8004009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:43.598 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.594+0000 7f80117fa700 1 -- 172.21.15.8:0/3753172564 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7ffc096f40 con 0x7f8014101480 2023-11-27T15:23:43.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:43 smithi029 ceph-mon[131166]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:23:43.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:43 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2827685154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:43.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.805+0000 7f801a1ff700 1 -- 172.21.15.8:0/3753172564 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80141a0a10 con 0x7f8014101480 2023-11-27T15:23:43.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.806+0000 7f80117fa700 1 -- 172.21.15.8:0/3753172564 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ffc09a560 con 0x7f8014101480 2023-11-27T15:23:43.810 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:23:43.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 -- 172.21.15.8:0/3753172564 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80000a1150 msgr2=0x7f80000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80000a1150 0x7f80000a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8004005950 tx=0x7f8004009260 comp rx=0 tx=0).stop 2023-11-27T15:23:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 -- 172.21.15.8:0/3753172564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8014101480 msgr2=0x7f8014198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8014101480 0x7f8014198770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009230 tx=0x7f7ffc00b730 comp rx=0 tx=0).stop 2023-11-27T15:23:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 -- 172.21.15.8:0/3753172564 shutdown_connections 2023-11-27T15:23:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8014101e10 0x7f801419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8014100290 0x7f8014198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80000a1150 0x7f80000a3600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 --2- 172.21.15.8:0/3753172564 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8014101480 0x7f8014198770 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:43.814 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 -- 172.21.15.8:0/3753172564 >> 172.21.15.8:0/3753172564 conn(0x7f80140fb870 msgr2=0x7f80140fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:43.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 -- 172.21.15.8:0/3753172564 shutdown_connections 2023-11-27T15:23:43.815 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:43.809+0000 7f801a1ff700 1 -- 172.21.15.8:0/3753172564 wait complete. 2023-11-27T15:23:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:43 smithi060 ceph-mon[131518]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:23:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:43 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2827685154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:43.829 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:43 smithi008 ceph-mon[128788]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:23:43.829 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:43 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2827685154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:43.831 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:23:44.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.034+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/1983756647 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a08101480 msgr2=0x7f4a081018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.034+0000 7f4a0d1ff700 1 --2- 172.21.15.8:0/1983756647 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a08101480 0x7f4a081018d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009990 tx=0x7f49fc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.035+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/1983756647 shutdown_connections 2023-11-27T15:23:44.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.035+0000 7f4a0d1ff700 1 --2- 172.21.15.8:0/1983756647 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a08100290 0x7f4a081006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.035+0000 7f4a0d1ff700 1 --2- 172.21.15.8:0/1983756647 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a08101e10 0x7f4a081086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.035+0000 7f4a0d1ff700 1 --2- 172.21.15.8:0/1983756647 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a08101480 0x7f4a081018d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.035+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/1983756647 >> 172.21.15.8:0/1983756647 conn(0x7f4a080fb870 msgr2=0x7f4a080fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:44.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.035+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/1983756647 shutdown_connections 2023-11-27T15:23:44.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.035+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/1983756647 wait complete. 2023-11-27T15:23:44.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a0d1ff700 1 Processor -- start 2023-11-27T15:23:44.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a0d1ff700 1 -- start start 2023-11-27T15:23:44.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a0d1ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a08100290 0x7f4a080748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a0d1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a08101480 0x7f4a08072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a0d1ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a08101e10 0x7f4a08073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a0d1ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a08073bc0 con 0x7f4a08100290 2023-11-27T15:23:44.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a0d1ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a08073d30 con 0x7f4a08101e10 2023-11-27T15:23:44.042 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a0d1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a081a0040 con 0x7f4a08101480 2023-11-27T15:23:44.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a06d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a08100290 0x7f4a080748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a06d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a08100290 0x7f4a080748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50012/0 (socket says 172.21.15.8:50012) 2023-11-27T15:23:44.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a06d9d700 1 -- 172.21.15.8:0/4084005277 learned_addr learned my addr 172.21.15.8:0/4084005277 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:44.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a0759e700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a08101e10 0x7f4a08073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.036+0000 7f4a0659c700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a08101480 0x7f4a08072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f4a06d9d700 1 -- 172.21.15.8:0/4084005277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a08101480 msgr2=0x7f4a08072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f4a06d9d700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a08101480 0x7f4a08072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f4a06d9d700 1 -- 172.21.15.8:0/4084005277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a08101e10 msgr2=0x7f4a08073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f4a06d9d700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a08101e10 0x7f4a08073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.043 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f4a06d9d700 1 -- 172.21.15.8:0/4084005277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc0095f0 con 0x7f4a08100290 2023-11-27T15:23:44.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f4a06d9d700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a08100290 0x7f4a080748a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f49f800e970 tx=0x7f49f800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:44.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f49effff700 1 -- 172.21.15.8:0/4084005277 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8009790 con 0x7f4a08100290 2023-11-27T15:23:44.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f49effff700 1 -- 172.21.15.8:0/4084005277 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f800c7e0 con 0x7f4a08100290 2023-11-27T15:23:44.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f49effff700 1 -- 172.21.15.8:0/4084005277 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8003c70 con 0x7f4a08100290 2023-11-27T15:23:44.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/4084005277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0810bc50 con 0x7f4a08100290 2023-11-27T15:23:44.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.037+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/4084005277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0810bac0 con 0x7f4a08100290 2023-11-27T15:23:44.044 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.038+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/4084005277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a081a0790 con 0x7f4a08100290 2023-11-27T15:23:44.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.040+0000 7f49effff700 1 -- 172.21.15.8:0/4084005277 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f49f800cc50 con 0x7f4a08100290 2023-11-27T15:23:44.045 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.042+0000 7f49effff700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f49f00a11d0 0x7f49f00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.042+0000 7f49effff700 1 -- 172.21.15.8:0/4084005277 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f49f8014070 con 0x7f4a08100290 2023-11-27T15:23:44.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.042+0000 7f4a0659c700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f49f00a11d0 0x7f49f00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.046 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.043+0000 7f4a0659c700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f49f00a11d0 0x7f49f00a3680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b2e0 tx=0x7f49fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:44.049 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.046+0000 7f49effff700 1 -- 172.21.15.8:0/4084005277 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f49f8096170 con 0x7f4a08100290 2023-11-27T15:23:44.258 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.255+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/4084005277 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a08107210 con 0x7f4a08100290 2023-11-27T15:23:44.259 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.255+0000 7f49effff700 1 -- 172.21.15.8:0/4084005277 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49f8003dd0 con 0x7f4a08100290 2023-11-27T15:23:44.259 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:23:44.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/4084005277 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f49f00a11d0 msgr2=0x7f49f00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f49f00a11d0 0x7f49f00a3680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00b2e0 tx=0x7f49fc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/4084005277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a08100290 msgr2=0x7f4a080748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a08100290 0x7f4a080748a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f49f800e970 tx=0x7f49f800ec80 comp rx=0 tx=0).stop 2023-11-27T15:23:44.262 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/4084005277 shutdown_connections 2023-11-27T15:23:44.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4a08101e10 0x7f4a08073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a08100290 0x7f4a080748a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f49f00a11d0 0x7f49f00a3680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 --2- 172.21.15.8:0/4084005277 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4a08101480 0x7f4a08072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/4084005277 >> 172.21.15.8:0/4084005277 conn(0x7f4a080fb870 msgr2=0x7f4a080fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:44.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/4084005277 shutdown_connections 2023-11-27T15:23:44.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.258+0000 7f4a0d1ff700 1 -- 172.21.15.8:0/4084005277 wait complete. 2023-11-27T15:23:44.280 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:23:44.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.484+0000 7fb296bff700 1 -- 172.21.15.8:0/3304550850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2900ff170 msgr2=0x7fb2900ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.484+0000 7fb296bff700 1 --2- 172.21.15.8:0/3304550850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2900ff170 0x7fb2900ff580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb284009990 tx=0x7fb284009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.484+0000 7fb296bff700 1 -- 172.21.15.8:0/3304550850 shutdown_connections 2023-11-27T15:23:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.484+0000 7fb296bff700 1 --2- 172.21.15.8:0/3304550850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb290100db0 0x7fb290107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.484+0000 7fb296bff700 1 --2- 172.21.15.8:0/3304550850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2900ff170 0x7fb2900ff580 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.484+0000 7fb296bff700 1 --2- 172.21.15.8:0/3304550850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb290100400 0x7fb290100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.484+0000 7fb296bff700 1 -- 172.21.15.8:0/3304550850 >> 172.21.15.8:0/3304550850 conn(0x7fb2900fa750 msgr2=0x7fb2900fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.484+0000 7fb296bff700 1 -- 172.21.15.8:0/3304550850 shutdown_connections 2023-11-27T15:23:44.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.484+0000 7fb296bff700 1 -- 172.21.15.8:0/3304550850 wait complete. 2023-11-27T15:23:44.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.485+0000 7fb296bff700 1 Processor -- start 2023-11-27T15:23:44.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.485+0000 7fb296bff700 1 -- start start 2023-11-27T15:23:44.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb296bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2900ff170 0x7fb2901031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb296bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb290100400 0x7fb2901017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb296bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb290100db0 0x7fb290101d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb296bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901024c0 con 0x7fb290100db0 2023-11-27T15:23:44.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb296bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290102630 con 0x7fb2900ff170 2023-11-27T15:23:44.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb296bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2901a0040 con 0x7fb290100400 2023-11-27T15:23:44.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb29519c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb290100db0 0x7fb290101d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb29519c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb290100db0 0x7fb290101d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50038/0 (socket says 172.21.15.8:50038) 2023-11-27T15:23:44.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb29519c700 1 -- 172.21.15.8:0/3654372126 learned_addr learned my addr 172.21.15.8:0/3654372126 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:44.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb29499b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2900ff170 0x7fb2901031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb28ffff700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb290100400 0x7fb2901017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb29519c700 1 -- 172.21.15.8:0/3654372126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb290100400 msgr2=0x7fb2901017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb29519c700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb290100400 0x7fb2901017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb29519c700 1 -- 172.21.15.8:0/3654372126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2900ff170 msgr2=0x7fb2901031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb29519c700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2900ff170 0x7fb2901031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.486+0000 7fb29519c700 1 -- 172.21.15.8:0/3654372126 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb288009c40 con 0x7fb290100db0 2023-11-27T15:23:44.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.487+0000 7fb29519c700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb290100db0 0x7fb290101d20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb28800c370 tx=0x7fb28800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:44.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.487+0000 7fb28dffb700 1 -- 172.21.15.8:0/3654372126 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288017070 con 0x7fb290100db0 2023-11-27T15:23:44.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.487+0000 7fb28dffb700 1 -- 172.21.15.8:0/3654372126 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28800f040 con 0x7fb290100db0 2023-11-27T15:23:44.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.487+0000 7fb28dffb700 1 -- 172.21.15.8:0/3654372126 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28800e050 con 0x7fb290100db0 2023-11-27T15:23:44.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.487+0000 7fb296bff700 1 -- 172.21.15.8:0/3654372126 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2840095f0 con 0x7fb290100db0 2023-11-27T15:23:44.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.487+0000 7fb296bff700 1 -- 172.21.15.8:0/3654372126 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29010ab60 con 0x7fb290100db0 2023-11-27T15:23:44.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.488+0000 7fb296bff700 1 -- 172.21.15.8:0/3654372126 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb290067a50 con 0x7fb290100db0 2023-11-27T15:23:44.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.490+0000 7fb28dffb700 1 -- 172.21.15.8:0/3654372126 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb28800cb60 con 0x7fb290100db0 2023-11-27T15:23:44.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.492+0000 7fb28dffb700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb2800a1150 0x7fb2800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.493+0000 7fb28dffb700 1 -- 172.21.15.8:0/3654372126 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb2880cc4e0 con 0x7fb290100db0 2023-11-27T15:23:44.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.493+0000 7fb29499b700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb2800a1150 0x7fb2800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.493+0000 7fb29499b700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb2800a1150 0x7fb2800a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb28400b2e0 tx=0x7fb284005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:44.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.496+0000 7fb28dffb700 1 -- 172.21.15.8:0/3654372126 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb2880959e0 con 0x7fb290100db0 2023-11-27T15:23:44.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:44 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3753172564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:23:44.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:44 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4084005277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:23:44.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.706+0000 7fb296bff700 1 -- 172.21.15.8:0/3654372126 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2901a0aa0 con 0x7fb290100db0 2023-11-27T15:23:44.710 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.706+0000 7fb28dffb700 1 -- 172.21.15.8:0/3654372126 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb288099000 con 0x7fb290100db0 2023-11-27T15:23:44.710 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-11-27T15:23:44.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 -- 172.21.15.8:0/3654372126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb2800a1150 msgr2=0x7fb2800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb2800a1150 0x7fb2800a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb28400b2e0 tx=0x7fb284005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 -- 172.21.15.8:0/3654372126 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb290100db0 msgr2=0x7fb290101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb290100db0 0x7fb290101d20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb28800c370 tx=0x7fb28800c680 comp rx=0 tx=0).stop 2023-11-27T15:23:44.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 -- 172.21.15.8:0/3654372126 shutdown_connections 2023-11-27T15:23:44.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb290100db0 0x7fb290101d20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.713 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb2900ff170 0x7fb2901031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb2800a1150 0x7fb2800a3600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 --2- 172.21.15.8:0/3654372126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb290100400 0x7fb2901017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 -- 172.21.15.8:0/3654372126 >> 172.21.15.8:0/3654372126 conn(0x7fb2900fa750 msgr2=0x7fb290067dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:44.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 -- 172.21.15.8:0/3654372126 shutdown_connections 2023-11-27T15:23:44.714 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.709+0000 7fb296bff700 1 -- 172.21.15.8:0/3654372126 wait complete. 2023-11-27T15:23:44.731 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:23:44.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:44 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3753172564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:23:44.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:44 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/4084005277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:23:44.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:44 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3753172564' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:23:44.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:44 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/4084005277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:23:44.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.934+0000 7fd08ffff700 1 -- 172.21.15.8:0/2614590250 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd088101480 msgr2=0x7fd0881018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.934+0000 7fd08ffff700 1 --2- 172.21.15.8:0/2614590250 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd088101480 0x7fd0881018d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd078009990 tx=0x7fd078009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.935+0000 7fd08ffff700 1 -- 172.21.15.8:0/2614590250 shutdown_connections 2023-11-27T15:23:44.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.935+0000 7fd08ffff700 1 --2- 172.21.15.8:0/2614590250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd088101e10 0x7fd0881086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.935+0000 7fd08ffff700 1 --2- 172.21.15.8:0/2614590250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd088100290 0x7fd0881006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.935+0000 7fd08ffff700 1 --2- 172.21.15.8:0/2614590250 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd088101480 0x7fd0881018d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.935+0000 7fd08ffff700 1 -- 172.21.15.8:0/2614590250 >> 172.21.15.8:0/2614590250 conn(0x7fd0880fb870 msgr2=0x7fd0880fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:44.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.935+0000 7fd08ffff700 1 -- 172.21.15.8:0/2614590250 shutdown_connections 2023-11-27T15:23:44.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.935+0000 7fd08ffff700 1 -- 172.21.15.8:0/2614590250 wait complete. 2023-11-27T15:23:44.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.936+0000 7fd08ffff700 1 Processor -- start 2023-11-27T15:23:44.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.936+0000 7fd08ffff700 1 -- start start 2023-11-27T15:23:44.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.936+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd088100290 0x7fd0881981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.936+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd088101480 0x7fd088198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.936+0000 7fd08ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd088101e10 0x7fd08819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.936+0000 7fd08ffff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08819fd70 con 0x7fd088101e10 2023-11-27T15:23:44.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.936+0000 7fd08ffff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08819feb0 con 0x7fd088101480 2023-11-27T15:23:44.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.936+0000 7fd08ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0881a0040 con 0x7fd088100290 2023-11-27T15:23:44.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08d59a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd088101480 0x7fd088198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd088101e10 0x7fd08819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08dd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd088100290 0x7fd0881981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08d59a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd088101480 0x7fd088198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45146/0 (socket says 172.21.15.8:45146) 2023-11-27T15:23:44.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08d59a700 1 -- 172.21.15.8:0/2034920996 learned_addr learned my addr 172.21.15.8:0/2034920996 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:44.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08d59a700 1 -- 172.21.15.8:0/2034920996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd088100290 msgr2=0x7fd0881981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08d59a700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd088100290 0x7fd0881981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08d59a700 1 -- 172.21.15.8:0/2034920996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd088101e10 msgr2=0x7fd08819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:44.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08d59a700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd088101e10 0x7fd08819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:44.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08d59a700 1 -- 172.21.15.8:0/2034920996 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0780095f0 con 0x7fd088101480 2023-11-27T15:23:44.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08e59c700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd088101e10 0x7fd08819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:44.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.937+0000 7fd08d59a700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd088101480 0x7fd088198730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd07800b620 tx=0x7fd078005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:44.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.938+0000 7fd07effd700 1 -- 172.21.15.8:0/2034920996 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07801d070 con 0x7fd088101480 2023-11-27T15:23:44.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.938+0000 7fd07effd700 1 -- 172.21.15.8:0/2034920996 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd078022470 con 0x7fd088101480 2023-11-27T15:23:44.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.938+0000 7fd08ffff700 1 -- 172.21.15.8:0/2034920996 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08810bc50 con 0x7fd088101480 2023-11-27T15:23:44.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.938+0000 7fd07effd700 1 -- 172.21.15.8:0/2034920996 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078017650 con 0x7fd088101480 2023-11-27T15:23:44.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.938+0000 7fd08ffff700 1 -- 172.21.15.8:0/2034920996 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08810bac0 con 0x7fd088101480 2023-11-27T15:23:44.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.939+0000 7fd08ffff700 1 -- 172.21.15.8:0/2034920996 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0881928e0 con 0x7fd088101480 2023-11-27T15:23:44.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.941+0000 7fd07effd700 1 -- 172.21.15.8:0/2034920996 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd0780225e0 con 0x7fd088101480 2023-11-27T15:23:44.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.942+0000 7fd07effd700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd0740a11d0 0x7fd0740a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:44.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.942+0000 7fd07effd700 1 -- 172.21.15.8:0/2034920996 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd0780ce720 con 0x7fd088101480 2023-11-27T15:23:44.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.942+0000 7fd08dd9b700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd0740a11d0 0x7fd0740a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:44.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.943+0000 7fd08dd9b700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd0740a11d0 0x7fd0740a3680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd0881012e0 tx=0x7fd084009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:44.948 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:44.944+0000 7fd07effd700 1 -- 172.21.15.8:0/2034920996 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd078097cb0 con 0x7fd088101480 2023-11-27T15:23:45.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.155+0000 7fd08ffff700 1 -- 172.21.15.8:0/2034920996 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd088061700 con 0x7fd088101480 2023-11-27T15:23:45.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.156+0000 7fd07effd700 1 -- 172.21.15.8:0/2034920996 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd07809b2d0 con 0x7fd088101480 2023-11-27T15:23:45.160 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:23:45.162 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.158+0000 7fd08ffff700 1 -- 172.21.15.8:0/2034920996 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd0740a11d0 msgr2=0x7fd0740a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.158+0000 7fd08ffff700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd0740a11d0 0x7fd0740a3680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd0881012e0 tx=0x7fd084009260 comp rx=0 tx=0).stop 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.158+0000 7fd08ffff700 1 -- 172.21.15.8:0/2034920996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd088101480 msgr2=0x7fd088198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.158+0000 7fd08ffff700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd088101480 0x7fd088198730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd07800b620 tx=0x7fd078005740 comp rx=0 tx=0).stop 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.158+0000 7fd08ffff700 1 -- 172.21.15.8:0/2034920996 shutdown_connections 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.158+0000 7fd08ffff700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd0740a11d0 0x7fd0740a3680 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.158+0000 7fd08ffff700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd088100290 0x7fd0881981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.158+0000 7fd08ffff700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd088101e10 0x7fd08819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.158+0000 7fd08ffff700 1 --2- 172.21.15.8:0/2034920996 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd088101480 0x7fd088198730 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.158+0000 7fd08ffff700 1 -- 172.21.15.8:0/2034920996 >> 172.21.15.8:0/2034920996 conn(0x7fd0880fb870 msgr2=0x7fd088106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.159+0000 7fd08ffff700 1 -- 172.21.15.8:0/2034920996 shutdown_connections 2023-11-27T15:23:45.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.159+0000 7fd08ffff700 1 -- 172.21.15.8:0/2034920996 wait complete. 2023-11-27T15:23:45.181 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-27T15:23:45.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.382+0000 7f8f481ff700 1 -- 172.21.15.8:0/3419512484 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f40101420 msgr2=0x7f8f40101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.382+0000 7f8f481ff700 1 --2- 172.21.15.8:0/3419512484 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f40101420 0x7f8f40101870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009990 tx=0x7f8f30009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.382+0000 7f8f481ff700 1 -- 172.21.15.8:0/3419512484 shutdown_connections 2023-11-27T15:23:45.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.382+0000 7f8f481ff700 1 --2- 172.21.15.8:0/3419512484 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f40100230 0x7f8f40100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.382+0000 7f8f481ff700 1 --2- 172.21.15.8:0/3419512484 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f40101db0 0x7f8f40108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.382+0000 7f8f481ff700 1 --2- 172.21.15.8:0/3419512484 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f40101420 0x7f8f40101870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.382+0000 7f8f481ff700 1 -- 172.21.15.8:0/3419512484 >> 172.21.15.8:0/3419512484 conn(0x7f8f400fb810 msgr2=0x7f8f400fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.382+0000 7f8f481ff700 1 -- 172.21.15.8:0/3419512484 shutdown_connections 2023-11-27T15:23:45.387 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.382+0000 7f8f481ff700 1 -- 172.21.15.8:0/3419512484 wait complete. 2023-11-27T15:23:45.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.383+0000 7f8f481ff700 1 Processor -- start 2023-11-27T15:23:45.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.383+0000 7f8f481ff700 1 -- start start 2023-11-27T15:23:45.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.383+0000 7f8f481ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f40100230 0x7f8f401981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.383+0000 7f8f481ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f40101420 0x7f8f40198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.383+0000 7f8f481ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f40101db0 0x7f8f4019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.383+0000 7f8f481ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019fd50 con 0x7f8f40101420 2023-11-27T15:23:45.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.383+0000 7f8f481ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4019fe90 con 0x7f8f40101db0 2023-11-27T15:23:45.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.383+0000 7f8f481ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f401a0020 con 0x7f8f40100230 2023-11-27T15:23:45.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f4579a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f40101420 0x7f8f40198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f4579a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f40101420 0x7f8f40198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50068/0 (socket says 172.21.15.8:50068) 2023-11-27T15:23:45.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f4579a700 1 -- 172.21.15.8:0/430220686 learned_addr learned my addr 172.21.15.8:0/430220686 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:45.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f45f9b700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f40100230 0x7f8f401981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f4679c700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f40101db0 0x7f8f4019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f4579a700 1 -- 172.21.15.8:0/430220686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f40100230 msgr2=0x7f8f401981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f4579a700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f40100230 0x7f8f401981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f4579a700 1 -- 172.21.15.8:0/430220686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f40101db0 msgr2=0x7f8f4019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f4579a700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f40101db0 0x7f8f4019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f4579a700 1 -- 172.21.15.8:0/430220686 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300095f0 con 0x7f8f40101420 2023-11-27T15:23:45.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.384+0000 7f8f4579a700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f40101420 0x7f8f40198710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000b620 tx=0x7f8f30005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.385+0000 7f8f36ffd700 1 -- 172.21.15.8:0/430220686 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3001d070 con 0x7f8f40101420 2023-11-27T15:23:45.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.385+0000 7f8f36ffd700 1 -- 172.21.15.8:0/430220686 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f30022470 con 0x7f8f40101420 2023-11-27T15:23:45.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.385+0000 7f8f481ff700 1 -- 172.21.15.8:0/430220686 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4010bbf0 con 0x7f8f40101420 2023-11-27T15:23:45.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.385+0000 7f8f36ffd700 1 -- 172.21.15.8:0/430220686 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30017650 con 0x7f8f40101420 2023-11-27T15:23:45.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.385+0000 7f8f481ff700 1 -- 172.21.15.8:0/430220686 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4010ba60 con 0x7f8f40101420 2023-11-27T15:23:45.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.386+0000 7f8f481ff700 1 -- 172.21.15.8:0/430220686 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f401a0620 con 0x7f8f40101420 2023-11-27T15:23:45.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.388+0000 7f8f36ffd700 1 -- 172.21.15.8:0/430220686 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8f300225e0 con 0x7f8f40101420 2023-11-27T15:23:45.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.389+0000 7f8f36ffd700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8f2c0a11d0 0x7f8f2c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.389+0000 7f8f36ffd700 1 -- 172.21.15.8:0/430220686 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8f300ce060 con 0x7f8f40101420 2023-11-27T15:23:45.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.389+0000 7f8f45f9b700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8f2c0a11d0 0x7f8f2c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.390+0000 7f8f45f9b700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8f2c0a11d0 0x7f8f2c0a3680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8f40101280 tx=0x7f8f3c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.393+0000 7f8f36ffd700 1 -- 172.21.15.8:0/430220686 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8f300975f0 con 0x7f8f40101420 2023-11-27T15:23:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:45 smithi008 ceph-mon[128788]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:23:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:45 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3654372126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:23:45.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:45 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2034920996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:23:45.623 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.620+0000 7f8f481ff700 1 -- 172.21.15.8:0/430220686 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8f401a0cc0 con 0x7f8f40101420 2023-11-27T15:23:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.620+0000 7f8f36ffd700 1 -- 172.21.15.8:0/430220686 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+579 (secure 0 0 0) 0x7f8f30027260 con 0x7f8f40101420 2023-11-27T15:23:45.624 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/2324276308 2023-11-28T15:16:38.242903+0000 2023-11-27T15:23:45.624 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2225314492 2023-11-28T15:16:38.242903+0000 2023-11-27T15:23:45.624 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3792241060 2023-11-28T15:16:38.242903+0000 2023-11-27T15:23:45.624 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3188200818 2023-11-28T15:16:11.844441+0000 2023-11-27T15:23:45.624 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/128817483 2023-11-28T15:16:11.844441+0000 2023-11-27T15:23:45.624 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2304079767 2023-11-28T15:16:11.844441+0000 2023-11-27T15:23:45.624 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/560372725 2023-11-28T15:16:11.844441+0000 2023-11-27T15:23:45.625 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/2324276308 2023-11-28T15:16:38.242903+0000 2023-11-27T15:23:45.625 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/560372725 2023-11-28T15:16:11.844441+0000 2023-11-27T15:23:45.625 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3849980484 2023-11-28T15:16:38.242903+0000 2023-11-27T15:23:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 -- 172.21.15.8:0/430220686 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8f2c0a11d0 msgr2=0x7f8f2c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8f2c0a11d0 0x7f8f2c0a3680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8f40101280 tx=0x7f8f3c00b410 comp rx=0 tx=0).stop 2023-11-27T15:23:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 -- 172.21.15.8:0/430220686 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f40101420 msgr2=0x7f8f40198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f40101420 0x7f8f40198710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000b620 tx=0x7f8f30005740 comp rx=0 tx=0).stop 2023-11-27T15:23:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 -- 172.21.15.8:0/430220686 shutdown_connections 2023-11-27T15:23:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8f2c0a11d0 0x7f8f2c0a3680 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8f40100230 0x7f8f401981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8f40101db0 0x7f8f4019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 --2- 172.21.15.8:0/430220686 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f40101420 0x7f8f40198710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 -- 172.21.15.8:0/430220686 >> 172.21.15.8:0/430220686 conn(0x7f8f400fb810 msgr2=0x7f8f400fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 -- 172.21.15.8:0/430220686 shutdown_connections 2023-11-27T15:23:45.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.623+0000 7f8f481ff700 1 -- 172.21.15.8:0/430220686 wait complete. 2023-11-27T15:23:45.628 INFO:teuthology.orchestra.run.smithi008.stderr:listed 10 entries 2023-11-27T15:23:45.645 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:23:45.645 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:23:45.689 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:23:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:45 smithi060 ceph-mon[131518]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:23:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:45 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3654372126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:23:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:45 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2034920996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:23:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:45 smithi029 ceph-mon[131166]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:23:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:45 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3654372126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:23:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:45 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2034920996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:23:45.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.929+0000 7fdae8dff700 1 -- 172.21.15.8:0/1069193947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdae41002d0 msgr2=0x7fdae41006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.929+0000 7fdae8dff700 1 --2- 172.21.15.8:0/1069193947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdae41002d0 0x7fdae41006e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009990 tx=0x7fdad4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.929+0000 7fdae8dff700 1 -- 172.21.15.8:0/1069193947 shutdown_connections 2023-11-27T15:23:45.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.929+0000 7fdae8dff700 1 --2- 172.21.15.8:0/1069193947 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdae41002d0 0x7fdae41006e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.929+0000 7fdae8dff700 1 --2- 172.21.15.8:0/1069193947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae4101e50 0x7fdae4108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.929+0000 7fdae8dff700 1 --2- 172.21.15.8:0/1069193947 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdae41014c0 0x7fdae4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.929+0000 7fdae8dff700 1 -- 172.21.15.8:0/1069193947 >> 172.21.15.8:0/1069193947 conn(0x7fdae40fb8b0 msgr2=0x7fdae40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.929+0000 7fdae8dff700 1 -- 172.21.15.8:0/1069193947 shutdown_connections 2023-11-27T15:23:45.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.929+0000 7fdae8dff700 1 -- 172.21.15.8:0/1069193947 wait complete. 2023-11-27T15:23:45.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.930+0000 7fdae8dff700 1 Processor -- start 2023-11-27T15:23:45.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae8dff700 1 -- start start 2023-11-27T15:23:45.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae8dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdae41002d0 0x7fdae40748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae8dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdae41014c0 0x7fdae4072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae8dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae4101e50 0x7fdae4073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae8dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae4073bc0 con 0x7fdae4101e50 2023-11-27T15:23:45.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae8dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae4073d30 con 0x7fdae41002d0 2023-11-27T15:23:45.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae8dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae41a0080 con 0x7fdae41014c0 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae4101e50 0x7fdae4073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae1d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdae41014c0 0x7fdae4072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdae41002d0 0x7fdae40748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae1d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdae41014c0 0x7fdae4072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:45720/0 (socket says 172.21.15.8:45720) 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.931+0000 7fdae1d9b700 1 -- 172.21.15.8:0/2745090446 learned_addr learned my addr 172.21.15.8:0/2745090446 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdae1d9b700 1 -- 172.21.15.8:0/2745090446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdae41002d0 msgr2=0x7fdae40748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdae1d9b700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdae41002d0 0x7fdae40748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdae1d9b700 1 -- 172.21.15.8:0/2745090446 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae4101e50 msgr2=0x7fdae4073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdae1d9b700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae4101e50 0x7fdae4073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdae1d9b700 1 -- 172.21.15.8:0/2745090446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40095f0 con 0x7fdae41014c0 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdae2d9d700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae4101e50 0x7fdae4073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdae1d9b700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdae41014c0 0x7fdae4072ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdad800e9a0 tx=0x7fdad800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdad37fe700 1 -- 172.21.15.8:0/2745090446 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8009790 con 0x7fdae41014c0 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdae8dff700 1 -- 172.21.15.8:0/2745090446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae410bc20 con 0x7fdae41014c0 2023-11-27T15:23:45.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdad37fe700 1 -- 172.21.15.8:0/2745090446 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdad8010430 con 0x7fdae41014c0 2023-11-27T15:23:45.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdad37fe700 1 -- 172.21.15.8:0/2745090446 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8018680 con 0x7fdae41014c0 2023-11-27T15:23:45.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.932+0000 7fdae8dff700 1 -- 172.21.15.8:0/2745090446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae410ba90 con 0x7fdae41014c0 2023-11-27T15:23:45.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.933+0000 7fdae8dff700 1 -- 172.21.15.8:0/2745090446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae4067af0 con 0x7fdae41014c0 2023-11-27T15:23:45.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.935+0000 7fdad37fe700 1 -- 172.21.15.8:0/2745090446 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdad80105a0 con 0x7fdae41014c0 2023-11-27T15:23:45.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.937+0000 7fdad37fe700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdacc0a1150 0x7fdacc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.937+0000 7fdad37fe700 1 -- 172.21.15.8:0/2745090446 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdad8014070 con 0x7fdae41014c0 2023-11-27T15:23:45.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.937+0000 7fdae259c700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdacc0a1150 0x7fdacc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.938+0000 7fdae259c700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdacc0a1150 0x7fdacc0a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdad400b2e0 tx=0x7fdad4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:45.939+0000 7fdad37fe700 1 -- 172.21.15.8:0/2745090446 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdad8096420 con 0x7fdae41014c0 2023-11-27T15:23:46.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.151+0000 7fdae8dff700 1 -- 172.21.15.8:0/2745090446 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdae4107200 con 0x7fdae41014c0 2023-11-27T15:23:46.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.153+0000 7fdad37fe700 1 -- 172.21.15.8:0/2745090446 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23817 (secure 0 0 0) 0x7fdad8026020 con 0x7fdae41014c0 2023-11-27T15:23:46.157 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:46.157 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":88,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:23:32.146115+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 -- 172.21.15.8:0/2745090446 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdacc0a1150 msgr2=0x7fdacc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdacc0a1150 0x7fdacc0a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdad400b2e0 tx=0x7fdad4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 -- 172.21.15.8:0/2745090446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdae41014c0 msgr2=0x7fdae4072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdae41014c0 0x7fdae4072ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdad800e9a0 tx=0x7fdad800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 -- 172.21.15.8:0/2745090446 shutdown_connections 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdae4101e50 0x7fdae4073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fdae41002d0 0x7fdae40748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fdacc0a1150 0x7fdacc0a3600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 --2- 172.21.15.8:0/2745090446 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fdae41014c0 0x7fdae4072ee0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 -- 172.21.15.8:0/2745090446 >> 172.21.15.8:0/2745090446 conn(0x7fdae40fb8b0 msgr2=0x7fdae40fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.160 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 -- 172.21.15.8:0/2745090446 shutdown_connections 2023-11-27T15:23:46.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.156+0000 7fdae8dff700 1 -- 172.21.15.8:0/2745090446 wait complete. 2023-11-27T15:23:46.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:23:25.166148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:23:26.584539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:23:30.130965+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:23:46.179 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:23:46.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.381+0000 7f4e32bff700 1 -- 172.21.15.8:0/2742845881 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e2c073550 msgr2=0x7f4e2c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.381+0000 7f4e32bff700 1 --2- 172.21.15.8:0/2742845881 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e2c073550 0x7f4e2c0739c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009990 tx=0x7f4e18009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.381+0000 7f4e32bff700 1 -- 172.21.15.8:0/2742845881 shutdown_connections 2023-11-27T15:23:46.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.381+0000 7f4e32bff700 1 --2- 172.21.15.8:0/2742845881 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e2c074b30 0x7f4e2c072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.381+0000 7f4e32bff700 1 --2- 172.21.15.8:0/2742845881 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e2c10c860 0x7f4e2c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.381+0000 7f4e32bff700 1 --2- 172.21.15.8:0/2742845881 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e2c073550 0x7f4e2c0739c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.381+0000 7f4e32bff700 1 -- 172.21.15.8:0/2742845881 >> 172.21.15.8:0/2742845881 conn(0x7f4e2c0fb870 msgr2=0x7f4e2c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.381+0000 7f4e32bff700 1 -- 172.21.15.8:0/2742845881 shutdown_connections 2023-11-27T15:23:46.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.381+0000 7f4e32bff700 1 -- 172.21.15.8:0/2742845881 wait complete. 2023-11-27T15:23:46.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.382+0000 7f4e32bff700 1 Processor -- start 2023-11-27T15:23:46.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.382+0000 7f4e32bff700 1 -- start start 2023-11-27T15:23:46.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e32bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e2c073550 0x7f4e2c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e32bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e2c074b30 0x7f4e2c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e32bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e2c10c860 0x7f4e2c1a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e32bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c1a41e0 con 0x7f4e2c10c860 2023-11-27T15:23:46.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e32bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c1a4320 con 0x7f4e2c073550 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e32bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c1a44b0 con 0x7f4e2c074b30 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e3119c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e2c10c860 0x7f4e2c1a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e2bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e2c074b30 0x7f4e2c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e3119c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e2c10c860 0x7f4e2c1a3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50110/0 (socket says 172.21.15.8:50110) 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e2bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e2c074b30 0x7f4e2c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:45738/0 (socket says 172.21.15.8:45738) 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e3119c700 1 -- 172.21.15.8:0/762544632 learned_addr learned my addr 172.21.15.8:0/762544632 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.383+0000 7f4e3099b700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e2c073550 0x7f4e2c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e2bfff700 1 -- 172.21.15.8:0/762544632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e2c073550 msgr2=0x7f4e2c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e2bfff700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e2c073550 0x7f4e2c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e2bfff700 1 -- 172.21.15.8:0/762544632 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e2c10c860 msgr2=0x7f4e2c1a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e2bfff700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e2c10c860 0x7f4e2c1a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e2bfff700 1 -- 172.21.15.8:0/762544632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e180095f0 con 0x7f4e2c074b30 2023-11-27T15:23:46.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e2bfff700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e2c074b30 0x7f4e2c19cba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800b680 tx=0x7f4e18005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e29ffb700 1 -- 172.21.15.8:0/762544632 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1801d070 con 0x7f4e2c074b30 2023-11-27T15:23:46.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e32bff700 1 -- 172.21.15.8:0/762544632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2c112270 con 0x7f4e2c074b30 2023-11-27T15:23:46.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e29ffb700 1 -- 172.21.15.8:0/762544632 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e18021410 con 0x7f4e2c074b30 2023-11-27T15:23:46.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e29ffb700 1 -- 172.21.15.8:0/762544632 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18021ea0 con 0x7f4e2c074b30 2023-11-27T15:23:46.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.384+0000 7f4e32bff700 1 -- 172.21.15.8:0/762544632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c1120e0 con 0x7f4e2c074b30 2023-11-27T15:23:46.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.385+0000 7f4e32bff700 1 -- 172.21.15.8:0/762544632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e2c1023b0 con 0x7f4e2c074b30 2023-11-27T15:23:46.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.387+0000 7f4e29ffb700 1 -- 172.21.15.8:0/762544632 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4e180217f0 con 0x7f4e2c074b30 2023-11-27T15:23:46.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.389+0000 7f4e29ffb700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4e1c0a1150 0x7f4e1c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.389+0000 7f4e29ffb700 1 -- 172.21.15.8:0/762544632 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4e180ce140 con 0x7f4e2c074b30 2023-11-27T15:23:46.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.389+0000 7f4e3099b700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4e1c0a1150 0x7f4e1c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.390+0000 7f4e3099b700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4e1c0a1150 0x7f4e1c0a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c074f00 tx=0x7f4e20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.391+0000 7f4e29ffb700 1 -- 172.21.15.8:0/762544632 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4e180976d0 con 0x7f4e2c074b30 2023-11-27T15:23:46.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.600+0000 7f4e32bff700 1 -- 172.21.15.8:0/762544632 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e2c061700 con 0x7f4e2c074b30 2023-11-27T15:23:46.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.601+0000 7f4e29ffb700 1 -- 172.21.15.8:0/762544632 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e1809acf0 con 0x7f4e2c074b30 2023-11-27T15:23:46.604 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 -- 172.21.15.8:0/762544632 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4e1c0a1150 msgr2=0x7f4e1c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4e1c0a1150 0x7f4e1c0a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c074f00 tx=0x7f4e20009260 comp rx=0 tx=0).stop 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 -- 172.21.15.8:0/762544632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e2c074b30 msgr2=0x7f4e2c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e2c074b30 0x7f4e2c19cba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800b680 tx=0x7f4e18005740 comp rx=0 tx=0).stop 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 -- 172.21.15.8:0/762544632 shutdown_connections 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e2c10c860 0x7f4e2c1a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4e2c073550 0x7f4e2c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4e1c0a1150 0x7f4e1c0a3600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 --2- 172.21.15.8:0/762544632 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4e2c074b30 0x7f4e2c19cba0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 -- 172.21.15.8:0/762544632 >> 172.21.15.8:0/762544632 conn(0x7f4e2c0fb870 msgr2=0x7f4e2c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.607 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 -- 172.21.15.8:0/762544632 shutdown_connections 2023-11-27T15:23:46.608 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.603+0000 7f4e32bff700 1 -- 172.21.15.8:0/762544632 wait complete. 2023-11-27T15:23:46.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:46 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/430220686' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:23:46.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:46 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2745090446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:46.625 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:23:46.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:46 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/430220686' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:23:46.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:46 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2745090446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:46.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.828+0000 7f8643fff700 1 -- 172.21.15.8:0/1882738147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 msgr2=0x7f863c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.828+0000 7f8643fff700 1 --2- 172.21.15.8:0/1882738147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c1018d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f862c009990 tx=0x7f862c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.828+0000 7f8643fff700 1 -- 172.21.15.8:0/1882738147 shutdown_connections 2023-11-27T15:23:46.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.828+0000 7f8643fff700 1 --2- 172.21.15.8:0/1882738147 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c101e10 0x7f863c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.828+0000 7f8643fff700 1 --2- 172.21.15.8:0/1882738147 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c100290 0x7f863c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.828+0000 7f8643fff700 1 --2- 172.21.15.8:0/1882738147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c1018d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.828+0000 7f8643fff700 1 -- 172.21.15.8:0/1882738147 >> 172.21.15.8:0/1882738147 conn(0x7f863c0fb870 msgr2=0x7f863c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:46.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.828+0000 7f8643fff700 1 -- 172.21.15.8:0/1882738147 shutdown_connections 2023-11-27T15:23:46.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.828+0000 7f8643fff700 1 -- 172.21.15.8:0/1882738147 wait complete. 2023-11-27T15:23:46.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.829+0000 7f8643fff700 1 Processor -- start 2023-11-27T15:23:46.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.829+0000 7f8643fff700 1 -- start start 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f8643fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c100290 0x7f863c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f8643fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f8643fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c101e10 0x7f863c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f8643fff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c19fd80 con 0x7f863c101480 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f8643fff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c19fec0 con 0x7f863c101e10 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f8643fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c1a0050 con 0x7f863c100290 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f864159a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f864159a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50136/0 (socket says 172.21.15.8:50136) 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f864159a700 1 -- 172.21.15.8:0/1945617759 learned_addr learned my addr 172.21.15.8:0/1945617759 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f8641d9b700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c100290 0x7f863c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f864259c700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c101e10 0x7f863c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f864159a700 1 -- 172.21.15.8:0/1945617759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c100290 msgr2=0x7f863c198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f864159a700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c100290 0x7f863c198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f864159a700 1 -- 172.21.15.8:0/1945617759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c101e10 msgr2=0x7f863c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f864159a700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c101e10 0x7f863c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f864159a700 1 -- 172.21.15.8:0/1945617759 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0095f0 con 0x7f863c101480 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.830+0000 7f8641d9b700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c100290 0x7f863c198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.831+0000 7f864159a700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c198740 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f862c004760 tx=0x7f862c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.831+0000 7f8632ffd700 1 -- 172.21.15.8:0/1945617759 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c01d070 con 0x7f863c101480 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.831+0000 7f8632ffd700 1 -- 172.21.15.8:0/1945617759 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f862c004830 con 0x7f863c101480 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.831+0000 7f8643fff700 1 -- 172.21.15.8:0/1945617759 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c10bc50 con 0x7f863c101480 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.831+0000 7f8632ffd700 1 -- 172.21.15.8:0/1945617759 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c005350 con 0x7f863c101480 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.831+0000 7f8643fff700 1 -- 172.21.15.8:0/1945617759 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c10bac0 con 0x7f863c101480 2023-11-27T15:23:46.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.832+0000 7f8643fff700 1 -- 172.21.15.8:0/1945617759 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863c1928e0 con 0x7f863c101480 2023-11-27T15:23:46.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.834+0000 7f8632ffd700 1 -- 172.21.15.8:0/1945617759 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f862c0054f0 con 0x7f863c101480 2023-11-27T15:23:46.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.835+0000 7f8632ffd700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86280a11d0 0x7f86280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:46.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.835+0000 7f8632ffd700 1 -- 172.21.15.8:0/1945617759 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f862c0ce140 con 0x7f863c101480 2023-11-27T15:23:46.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.836+0000 7f8641d9b700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86280a11d0 0x7f86280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:46.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.836+0000 7f8641d9b700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86280a11d0 0x7f86280a3680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f863c1012e0 tx=0x7f8638009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:46.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:46.837+0000 7f8632ffd700 1 -- 172.21.15.8:0/1945617759 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f862c0976d0 con 0x7f863c101480 2023-11-27T15:23:46.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:46 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/430220686' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:23:46.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:46 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2745090446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:47.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.050+0000 7f8643fff700 1 -- 172.21.15.8:0/1945617759 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f863c067a50 con 0x7f863c101480 2023-11-27T15:23:47.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.050+0000 7f8632ffd700 1 -- 172.21.15.8:0/1945617759 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f862c09acf0 con 0x7f863c101480 2023-11-27T15:23:47.054 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 -- 172.21.15.8:0/1945617759 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86280a11d0 msgr2=0x7f86280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86280a11d0 0x7f86280a3680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f863c1012e0 tx=0x7f8638009260 comp rx=0 tx=0).stop 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 -- 172.21.15.8:0/1945617759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 msgr2=0x7f863c198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c198740 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f862c004760 tx=0x7f862c004790 comp rx=0 tx=0).stop 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 -- 172.21.15.8:0/1945617759 shutdown_connections 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f86280a11d0 0x7f86280a3680 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f863c100290 0x7f863c198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f863c101e10 0x7f863c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 --2- 172.21.15.8:0/1945617759 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f863c101480 0x7f863c198740 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 -- 172.21.15.8:0/1945617759 >> 172.21.15.8:0/1945617759 conn(0x7f863c0fb870 msgr2=0x7f863c106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 -- 172.21.15.8:0/1945617759 shutdown_connections 2023-11-27T15:23:47.058 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.053+0000 7f8643fff700 1 -- 172.21.15.8:0/1945617759 wait complete. 2023-11-27T15:23:47.075 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:23:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.277+0000 7f7f58dff700 1 -- 172.21.15.8:0/3588259103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f54101e10 msgr2=0x7f7f541086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.277+0000 7f7f58dff700 1 --2- 172.21.15.8:0/3588259103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f54101e10 0x7f7f541086d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009f90 tx=0x7f7f4c00b330 comp rx=0 tx=0).stop 2023-11-27T15:23:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.277+0000 7f7f58dff700 1 -- 172.21.15.8:0/3588259103 shutdown_connections 2023-11-27T15:23:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.277+0000 7f7f58dff700 1 --2- 172.21.15.8:0/3588259103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f54101e10 0x7f7f541086d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.277+0000 7f7f58dff700 1 --2- 172.21.15.8:0/3588259103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f54100290 0x7f7f541006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.277+0000 7f7f58dff700 1 --2- 172.21.15.8:0/3588259103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f54101480 0x7f7f541018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.277+0000 7f7f58dff700 1 -- 172.21.15.8:0/3588259103 >> 172.21.15.8:0/3588259103 conn(0x7f7f540fb870 msgr2=0x7f7f540fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.277+0000 7f7f58dff700 1 -- 172.21.15.8:0/3588259103 shutdown_connections 2023-11-27T15:23:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.277+0000 7f7f58dff700 1 -- 172.21.15.8:0/3588259103 wait complete. 2023-11-27T15:23:47.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.278+0000 7f7f58dff700 1 Processor -- start 2023-11-27T15:23:47.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.278+0000 7f7f58dff700 1 -- start start 2023-11-27T15:23:47.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.278+0000 7f7f58dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f54100290 0x7f7f540748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.278+0000 7f7f58dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f54101480 0x7f7f54072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f58dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f54101e10 0x7f7f54073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f58dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54073bc0 con 0x7f7f54101e10 2023-11-27T15:23:47.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f58dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54073d30 con 0x7f7f54100290 2023-11-27T15:23:47.283 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f58dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5419fef0 con 0x7f7f54101480 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f52d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f54101e10 0x7f7f54073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f52d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f54101e10 0x7f7f54073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50166/0 (socket says 172.21.15.8:50166) 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f52d9d700 1 -- 172.21.15.8:0/1329624445 learned_addr learned my addr 172.21.15.8:0/1329624445 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f4bfff700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f54101480 0x7f7f54072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f5259c700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f54100290 0x7f7f540748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f52d9d700 1 -- 172.21.15.8:0/1329624445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f54101480 msgr2=0x7f7f54072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f52d9d700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f54101480 0x7f7f54072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f52d9d700 1 -- 172.21.15.8:0/1329624445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f54100290 msgr2=0x7f7f540748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f52d9d700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f54100290 0x7f7f540748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f52d9d700 1 -- 172.21.15.8:0/1329624445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c0095f0 con 0x7f7f54101e10 2023-11-27T15:23:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.279+0000 7f7f52d9d700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f54101e10 0x7f7f54073420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009790 tx=0x7f7f4c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.280+0000 7f7f4b7fe700 1 -- 172.21.15.8:0/1329624445 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00e040 con 0x7f7f54101e10 2023-11-27T15:23:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.280+0000 7f7f4b7fe700 1 -- 172.21.15.8:0/1329624445 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f4c008f50 con 0x7f7f54101e10 2023-11-27T15:23:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.280+0000 7f7f58dff700 1 -- 172.21.15.8:0/1329624445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c009c40 con 0x7f7f54101e10 2023-11-27T15:23:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.280+0000 7f7f4b7fe700 1 -- 172.21.15.8:0/1329624445 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0052a0 con 0x7f7f54101e10 2023-11-27T15:23:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.280+0000 7f7f58dff700 1 -- 172.21.15.8:0/1329624445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5410bb90 con 0x7f7f54101e10 2023-11-27T15:23:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.281+0000 7f7f58dff700 1 -- 172.21.15.8:0/1329624445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5410ba00 con 0x7f7f54101e10 2023-11-27T15:23:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.283+0000 7f7f4b7fe700 1 -- 172.21.15.8:0/1329624445 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7f4c01d470 con 0x7f7f54101e10 2023-11-27T15:23:47.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.284+0000 7f7f4b7fe700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7f400a1150 0x7f7f400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.284+0000 7f7f4b7fe700 1 -- 172.21.15.8:0/1329624445 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7f4c0ce220 con 0x7f7f54101e10 2023-11-27T15:23:47.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.284+0000 7f7f5259c700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7f400a1150 0x7f7f400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.288 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.285+0000 7f7f5259c700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7f400a1150 0x7f7f400a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7f541012e0 tx=0x7f7f44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.288+0000 7f7f4b7fe700 1 -- 172.21.15.8:0/1329624445 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7f4c0977b0 con 0x7f7f54101e10 2023-11-27T15:23:47.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.500+0000 7f7f58dff700 1 -- 172.21.15.8:0/1329624445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f5410bd20 con 0x7f7f54101e10 2023-11-27T15:23:47.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.500+0000 7f7f4b7fe700 1 -- 172.21.15.8:0/1329624445 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f4c09add0 con 0x7f7f54101e10 2023-11-27T15:23:47.504 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-11-27T15:23:47.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 -- 172.21.15.8:0/1329624445 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7f400a1150 msgr2=0x7f7f400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7f400a1150 0x7f7f400a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7f541012e0 tx=0x7f7f44009260 comp rx=0 tx=0).stop 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 -- 172.21.15.8:0/1329624445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f54101e10 msgr2=0x7f7f54073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f54101e10 0x7f7f54073420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009790 tx=0x7f7f4c004c80 comp rx=0 tx=0).stop 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 -- 172.21.15.8:0/1329624445 shutdown_connections 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f54101e10 0x7f7f54073420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7f54100290 0x7f7f540748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7f400a1150 0x7f7f400a3600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 --2- 172.21.15.8:0/1329624445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7f54101480 0x7f7f54072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 -- 172.21.15.8:0/1329624445 >> 172.21.15.8:0/1329624445 conn(0x7f7f540fb870 msgr2=0x7f7f540fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 -- 172.21.15.8:0/1329624445 shutdown_connections 2023-11-27T15:23:47.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.503+0000 7f7f58dff700 1 -- 172.21.15.8:0/1329624445 wait complete. 2023-11-27T15:23:47.525 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:23:47.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.731+0000 7efec2bff700 1 -- 172.21.15.8:0/4031152188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efebc101480 msgr2=0x7efebc1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.731+0000 7efec2bff700 1 --2- 172.21.15.8:0/4031152188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efebc101480 0x7efebc1018d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efea80098f0 tx=0x7efea8009c00 comp rx=0 tx=0).stop 2023-11-27T15:23:47.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.731+0000 7efec2bff700 1 -- 172.21.15.8:0/4031152188 shutdown_connections 2023-11-27T15:23:47.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.731+0000 7efec2bff700 1 --2- 172.21.15.8:0/4031152188 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efebc100290 0x7efebc1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.731+0000 7efec2bff700 1 --2- 172.21.15.8:0/4031152188 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efebc101e10 0x7efebc1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.731+0000 7efec2bff700 1 --2- 172.21.15.8:0/4031152188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efebc101480 0x7efebc1018d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.731+0000 7efec2bff700 1 -- 172.21.15.8:0/4031152188 >> 172.21.15.8:0/4031152188 conn(0x7efebc0fb870 msgr2=0x7efebc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.731+0000 7efec2bff700 1 -- 172.21.15.8:0/4031152188 shutdown_connections 2023-11-27T15:23:47.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.731+0000 7efec2bff700 1 -- 172.21.15.8:0/4031152188 wait complete. 2023-11-27T15:23:47.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.732+0000 7efec2bff700 1 Processor -- start 2023-11-27T15:23:47.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec2bff700 1 -- start start 2023-11-27T15:23:47.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec2bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efebc100290 0x7efebc198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec2bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efebc101480 0x7efebc198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec2bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efebc101e10 0x7efebc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec2bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc19fda0 con 0x7efebc100290 2023-11-27T15:23:47.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec2bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc19fee0 con 0x7efebc101e10 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec2bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc1a0070 con 0x7efebc101480 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec099b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efebc100290 0x7efebc198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efebbfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efebc101480 0x7efebc198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec099b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efebc100290 0x7efebc198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50184/0 (socket says 172.21.15.8:50184) 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec099b700 1 -- 172.21.15.8:0/1412426757 learned_addr learned my addr 172.21.15.8:0/1412426757 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.733+0000 7efec119c700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efebc101e10 0x7efebc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efebbfff700 1 -- 172.21.15.8:0/1412426757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efebc101e10 msgr2=0x7efebc19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efebbfff700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efebc101e10 0x7efebc19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efebbfff700 1 -- 172.21.15.8:0/1412426757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efebc100290 msgr2=0x7efebc198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efebbfff700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efebc100290 0x7efebc198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efebbfff700 1 -- 172.21.15.8:0/1412426757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb4009d00 con 0x7efebc101480 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efec099b700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efebc100290 0x7efebc198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efebbfff700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efebc101480 0x7efebc198760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efea8009230 tx=0x7efea800b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efeb9ffb700 1 -- 172.21.15.8:0/1412426757 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea801c070 con 0x7efebc101480 2023-11-27T15:23:47.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efec2bff700 1 -- 172.21.15.8:0/1412426757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea8009520 con 0x7efebc101480 2023-11-27T15:23:47.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efeb9ffb700 1 -- 172.21.15.8:0/1412426757 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efea800f7f0 con 0x7efebc101480 2023-11-27T15:23:47.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efeb9ffb700 1 -- 172.21.15.8:0/1412426757 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8003990 con 0x7efebc101480 2023-11-27T15:23:47.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.734+0000 7efec2bff700 1 -- 172.21.15.8:0/1412426757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efebc10bc50 con 0x7efebc101480 2023-11-27T15:23:47.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.735+0000 7efec2bff700 1 -- 172.21.15.8:0/1412426757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efebc10bde0 con 0x7efebc101480 2023-11-27T15:23:47.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.737+0000 7efeb9ffb700 1 -- 172.21.15.8:0/1412426757 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7efea8020030 con 0x7efebc101480 2023-11-27T15:23:47.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.739+0000 7efeb9ffb700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efeac0a1150 0x7efeac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:47.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.739+0000 7efeb9ffb700 1 -- 172.21.15.8:0/1412426757 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efea80ce480 con 0x7efebc101480 2023-11-27T15:23:47.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.739+0000 7efec099b700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efeac0a1150 0x7efeac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:47.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.739+0000 7efec099b700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efeac0a1150 0x7efeac0a3600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efeb0005950 tx=0x7efeb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:47.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.741+0000 7efeb9ffb700 1 -- 172.21.15.8:0/1412426757 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efea8097a10 con 0x7efebc101480 2023-11-27T15:23:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:47 smithi060 ceph-mon[131518]: pgmap v292: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:23:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:47 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/762544632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:23:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:47 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1945617759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:23:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:47 smithi008 ceph-mon[128788]: pgmap v292: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:23:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:47 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/762544632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:23:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:47 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1945617759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:23:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:47 smithi029 ceph-mon[131166]: pgmap v292: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:23:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:47 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/762544632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:23:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:47 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1945617759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:23:47.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.950+0000 7efec2bff700 1 -- 172.21.15.8:0/1412426757 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7efebc067a50 con 0x7efebc101480 2023-11-27T15:23:47.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.951+0000 7efeb9ffb700 1 -- 172.21.15.8:0/1412426757 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7efea809b030 con 0x7efebc101480 2023-11-27T15:23:47.954 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:23:47.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 -- 172.21.15.8:0/1412426757 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efeac0a1150 msgr2=0x7efeac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efeac0a1150 0x7efeac0a3600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efeb0005950 tx=0x7efeb0009260 comp rx=0 tx=0).stop 2023-11-27T15:23:47.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 -- 172.21.15.8:0/1412426757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efebc101480 msgr2=0x7efebc198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:47.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efebc101480 0x7efebc198760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efea8009230 tx=0x7efea800b730 comp rx=0 tx=0).stop 2023-11-27T15:23:47.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 -- 172.21.15.8:0/1412426757 shutdown_connections 2023-11-27T15:23:47.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efebc101e10 0x7efebc19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efebc100290 0x7efebc198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efeac0a1150 0x7efeac0a3600 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 --2- 172.21.15.8:0/1412426757 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efebc101480 0x7efebc198760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:47.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 -- 172.21.15.8:0/1412426757 >> 172.21.15.8:0/1412426757 conn(0x7efebc0fb870 msgr2=0x7efebc106e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:47.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 -- 172.21.15.8:0/1412426757 shutdown_connections 2023-11-27T15:23:47.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:47.953+0000 7efec2bff700 1 -- 172.21.15.8:0/1412426757 wait complete. 2023-11-27T15:23:47.975 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-27T15:23:48.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.175+0000 7f2ba23ff700 1 -- 172.21.15.8:0/4117648723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c101420 msgr2=0x7f2b9c101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.175+0000 7f2ba23ff700 1 --2- 172.21.15.8:0/4117648723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c101420 0x7f2b9c101870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009990 tx=0x7f2b90009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.175+0000 7f2ba23ff700 1 -- 172.21.15.8:0/4117648723 shutdown_connections 2023-11-27T15:23:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.175+0000 7f2ba23ff700 1 --2- 172.21.15.8:0/4117648723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c101db0 0x7f2b9c108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.175+0000 7f2ba23ff700 1 --2- 172.21.15.8:0/4117648723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b9c100230 0x7f2b9c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.175+0000 7f2ba23ff700 1 --2- 172.21.15.8:0/4117648723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c101420 0x7f2b9c101870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.175+0000 7f2ba23ff700 1 -- 172.21.15.8:0/4117648723 >> 172.21.15.8:0/4117648723 conn(0x7f2b9c0fb810 msgr2=0x7f2b9c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.175+0000 7f2ba23ff700 1 -- 172.21.15.8:0/4117648723 shutdown_connections 2023-11-27T15:23:48.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.175+0000 7f2ba23ff700 1 -- 172.21.15.8:0/4117648723 wait complete. 2023-11-27T15:23:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.176+0000 7f2ba23ff700 1 Processor -- start 2023-11-27T15:23:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.176+0000 7f2ba23ff700 1 -- start start 2023-11-27T15:23:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.176+0000 7f2ba23ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c100230 0x7f2b9c195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:48.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.176+0000 7f2ba23ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b9c101420 0x7f2b9c196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.176+0000 7f2ba23ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c101db0 0x7f2b9c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.176+0000 7f2ba23ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c19db70 con 0x7f2b9c101db0 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.176+0000 7f2ba23ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c19dcb0 con 0x7f2b9c101420 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.176+0000 7f2ba23ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c19de40 con 0x7f2b9c100230 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2ba099c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c101db0 0x7f2b9c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2b9b7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b9c101420 0x7f2b9c196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2ba099c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c101db0 0x7f2b9c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50198/0 (socket says 172.21.15.8:50198) 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2ba099c700 1 -- 172.21.15.8:0/3686635831 learned_addr learned my addr 172.21.15.8:0/3686635831 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2b9bfff700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c100230 0x7f2b9c195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2ba099c700 1 -- 172.21.15.8:0/3686635831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c100230 msgr2=0x7f2b9c195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:48.182 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2ba099c700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c100230 0x7f2b9c195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2ba099c700 1 -- 172.21.15.8:0/3686635831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b9c101420 msgr2=0x7f2b9c196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:48.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2ba099c700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b9c101420 0x7f2b9c196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2ba099c700 1 -- 172.21.15.8:0/3686635831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b900095f0 con 0x7f2b9c101db0 2023-11-27T15:23:48.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2ba099c700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c101db0 0x7f2b9c19d5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400ef30 tx=0x7f2b9400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:48.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2b997fa700 1 -- 172.21.15.8:0/3686635831 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94018070 con 0x7f2b9c101db0 2023-11-27T15:23:48.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2b997fa700 1 -- 172.21.15.8:0/3686635831 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b9400f040 con 0x7f2b9c101db0 2023-11-27T15:23:48.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2ba23ff700 1 -- 172.21.15.8:0/3686635831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c10bbf0 con 0x7f2b9c101db0 2023-11-27T15:23:48.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.177+0000 7f2b997fa700 1 -- 172.21.15.8:0/3686635831 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400cdb0 con 0x7f2b9c101db0 2023-11-27T15:23:48.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.178+0000 7f2ba23ff700 1 -- 172.21.15.8:0/3686635831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c10ba60 con 0x7f2b9c101db0 2023-11-27T15:23:48.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.179+0000 7f2ba23ff700 1 -- 172.21.15.8:0/3686635831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c19e4d0 con 0x7f2b9c101db0 2023-11-27T15:23:48.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.181+0000 7f2b997fa700 1 -- 172.21.15.8:0/3686635831 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2b94003a00 con 0x7f2b9c101db0 2023-11-27T15:23:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.182+0000 7f2b997fa700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2b840a11d0 0x7f2b840a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.182+0000 7f2b997fa700 1 -- 172.21.15.8:0/3686635831 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2b940cd040 con 0x7f2b9c101db0 2023-11-27T15:23:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.182+0000 7f2b9bfff700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2b840a11d0 0x7f2b840a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.183+0000 7f2b9bfff700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2b840a11d0 0x7f2b840a3680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c101280 tx=0x7f2b8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:48.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.184+0000 7f2b997fa700 1 -- 172.21.15.8:0/3686635831 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2b94096630 con 0x7f2b9c101db0 2023-11-27T15:23:48.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.411+0000 7f2ba23ff700 1 -- 172.21.15.8:0/3686635831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b9c107160 con 0x7f2b9c101db0 2023-11-27T15:23:48.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.412+0000 7f2b997fa700 1 -- 172.21.15.8:0/3686635831 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+579 (secure 0 0 0) 0x7f2b94099c50 con 0x7f2b9c101db0 2023-11-27T15:23:48.415 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/2324276308 2023-11-28T15:16:38.242903+0000 2023-11-27T15:23:48.415 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2225314492 2023-11-28T15:16:38.242903+0000 2023-11-27T15:23:48.416 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3792241060 2023-11-28T15:16:38.242903+0000 2023-11-27T15:23:48.416 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3188200818 2023-11-28T15:16:11.844441+0000 2023-11-27T15:23:48.416 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/128817483 2023-11-28T15:16:11.844441+0000 2023-11-27T15:23:48.416 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2304079767 2023-11-28T15:16:11.844441+0000 2023-11-27T15:23:48.416 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/560372725 2023-11-28T15:16:11.844441+0000 2023-11-27T15:23:48.416 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/2324276308 2023-11-28T15:16:38.242903+0000 2023-11-27T15:23:48.416 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/560372725 2023-11-28T15:16:11.844441+0000 2023-11-27T15:23:48.416 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3849980484 2023-11-28T15:16:38.242903+0000 2023-11-27T15:23:48.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 -- 172.21.15.8:0/3686635831 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2b840a11d0 msgr2=0x7f2b840a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:48.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2b840a11d0 0x7f2b840a3680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c101280 tx=0x7f2b8c009260 comp rx=0 tx=0).stop 2023-11-27T15:23:48.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 -- 172.21.15.8:0/3686635831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c101db0 msgr2=0x7f2b9c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:48.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c101db0 0x7f2b9c19d5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400ef30 tx=0x7f2b9400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 -- 172.21.15.8:0/3686635831 shutdown_connections 2023-11-27T15:23:48.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2b840a11d0 0x7f2b840a3680 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2b9c100230 0x7f2b9c195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.418 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2b9c101db0 0x7f2b9c19d5a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 --2- 172.21.15.8:0/3686635831 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2b9c101420 0x7f2b9c196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:48.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 -- 172.21.15.8:0/3686635831 >> 172.21.15.8:0/3686635831 conn(0x7f2b9c0fb810 msgr2=0x7f2b9c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:48.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 -- 172.21.15.8:0/3686635831 shutdown_connections 2023-11-27T15:23:48.419 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:48.414+0000 7f2ba23ff700 1 -- 172.21.15.8:0/3686635831 wait complete. 2023-11-27T15:23:48.419 INFO:teuthology.orchestra.run.smithi008.stderr:listed 10 entries 2023-11-27T15:23:48.436 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi008.front.sepia.ceph.com... 2023-11-27T15:23:48.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:48.436 DEBUG:teuthology.orchestra.run.smithi008:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:23:48.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi008.stderr:ceph-fuse[165782]: fuse finished with error 0 and tester_r 0 2023-11-27T15:23:48.508 INFO:teuthology.orchestra.run:waiting for 300 2023-11-27T15:23:48.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:48 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1329624445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:23:48.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:48 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1412426757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:23:48.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:48 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1329624445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:23:48.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:48 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1412426757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:23:48.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:48 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1329624445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:23:48.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:48 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1412426757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:23:49.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:49 smithi060 ceph-mon[131518]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:23:49.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:49 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3686635831' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:23:49.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:49 smithi008 ceph-mon[128788]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:23:49.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:49 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3686635831' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:23:49.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:49 smithi029 ceph-mon[131166]: pgmap v293: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:23:49.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:49 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3686635831' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:23:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:51 smithi060 ceph-mon[131518]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:51.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:51 smithi008 ceph-mon[128788]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:51.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:51 smithi029 ceph-mon[131166]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:53 smithi060 ceph-mon[131518]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:53.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:53 smithi008 ceph-mon[128788]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:53.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:53 smithi029 ceph-mon[131166]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:54.610 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi008.front.sepia.ceph.com 2023-11-27T15:23:54.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:23:54.610 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-27T15:23:54.671 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:23:54.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.877+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/3944270527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9a4101e50 msgr2=0x7fa9a4108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:54.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.877+0000 7fa9aa7ff700 1 --2- 172.21.15.8:0/3944270527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9a4101e50 0x7fa9a4108710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00b0a0 tx=0x7fa99c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:23:54.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.877+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/3944270527 shutdown_connections 2023-11-27T15:23:54.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.877+0000 7fa9aa7ff700 1 --2- 172.21.15.8:0/3944270527 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a41002d0 0x7fa9a41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:54.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.877+0000 7fa9aa7ff700 1 --2- 172.21.15.8:0/3944270527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9a4101e50 0x7fa9a4108710 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:54.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.877+0000 7fa9aa7ff700 1 --2- 172.21.15.8:0/3944270527 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9a41014c0 0x7fa9a4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:54.881 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.877+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/3944270527 >> 172.21.15.8:0/3944270527 conn(0x7fa9a40fb8b0 msgr2=0x7fa9a40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:54.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.877+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/3944270527 shutdown_connections 2023-11-27T15:23:54.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.877+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/3944270527 wait complete. 2023-11-27T15:23:54.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.878+0000 7fa9aa7ff700 1 Processor -- start 2023-11-27T15:23:54.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.878+0000 7fa9aa7ff700 1 -- start start 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.878+0000 7fa9aa7ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9a41002d0 0x7fa9a40748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.878+0000 7fa9aa7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a41014c0 0x7fa9a4072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.878+0000 7fa9aa7ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9a4101e50 0x7fa9a4073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.878+0000 7fa9aa7ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a4073bc0 con 0x7fa9a4101e50 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9aa7ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a4073d30 con 0x7fa9a41002d0 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9aa7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41a01b0 con 0x7fa9a41014c0 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a8d9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9a4101e50 0x7fa9a4073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a8d9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9a4101e50 0x7fa9a4073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59350/0 (socket says 172.21.15.8:59350) 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a37fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a41014c0 0x7fa9a4072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a8d9c700 1 -- 172.21.15.8:0/311303233 learned_addr learned my addr 172.21.15.8:0/311303233 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a3fff700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9a41002d0 0x7fa9a40748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a8d9c700 1 -- 172.21.15.8:0/311303233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a41014c0 msgr2=0x7fa9a4072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:54.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a8d9c700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a41014c0 0x7fa9a4072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:54.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a8d9c700 1 -- 172.21.15.8:0/311303233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9a41002d0 msgr2=0x7fa9a40748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:54.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a8d9c700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9a41002d0 0x7fa9a40748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:54.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a8d9c700 1 -- 172.21.15.8:0/311303233 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c009d00 con 0x7fa9a4101e50 2023-11-27T15:23:54.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.879+0000 7fa9a8d9c700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9a4101e50 0x7fa9a4073420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00d010 tx=0x7fa99c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:54.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.880+0000 7fa9a17fa700 1 -- 172.21.15.8:0/311303233 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c00e040 con 0x7fa9a4101e50 2023-11-27T15:23:54.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.880+0000 7fa9a17fa700 1 -- 172.21.15.8:0/311303233 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa99c003e10 con 0x7fa9a4101e50 2023-11-27T15:23:54.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.880+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/311303233 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a410bca0 con 0x7fa9a4101e50 2023-11-27T15:23:54.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.880+0000 7fa9a17fa700 1 -- 172.21.15.8:0/311303233 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c0052a0 con 0x7fa9a4101e50 2023-11-27T15:23:54.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.880+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/311303233 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a410bb10 con 0x7fa9a4101e50 2023-11-27T15:23:54.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.881+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/311303233 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a41077d0 con 0x7fa9a4101e50 2023-11-27T15:23:54.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.883+0000 7fa9a17fa700 1 -- 172.21.15.8:0/311303233 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa99c01d480 con 0x7fa9a4101e50 2023-11-27T15:23:54.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.884+0000 7fa9a17fa700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa98c0a1150 0x7fa98c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:54.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.885+0000 7fa9a17fa700 1 -- 172.21.15.8:0/311303233 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa99c0cdeb0 con 0x7fa9a4101e50 2023-11-27T15:23:54.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.885+0000 7fa9a3fff700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa98c0a1150 0x7fa98c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:54.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.885+0000 7fa9a3fff700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa98c0a1150 0x7fa98c0a3600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4101320 tx=0x7fa994009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:54.892 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:54.888+0000 7fa9a17fa700 1 -- 172.21.15.8:0/311303233 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa99c097440 con 0x7fa9a4101e50 2023-11-27T15:23:55.104 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.100+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/311303233 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9a402cfe0 con 0x7fa9a4101e50 2023-11-27T15:23:55.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.102+0000 7fa9a17fa700 1 -- 172.21.15.8:0/311303233 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23817 (secure 0 0 0) 0x7fa99c09aa60 con 0x7fa9a4101e50 2023-11-27T15:23:55.106 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:55.106 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":88,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:23:32.146115+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:23:55.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.104+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/311303233 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa98c0a1150 msgr2=0x7fa98c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:55.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.104+0000 7fa9aa7ff700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa98c0a1150 0x7fa98c0a3600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4101320 tx=0x7fa994009260 comp rx=0 tx=0).stop 2023-11-27T15:23:55.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.104+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/311303233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9a4101e50 msgr2=0x7fa9a4073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:55.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.104+0000 7fa9aa7ff700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9a4101e50 0x7fa9a4073420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00d010 tx=0x7fa99c00bc80 comp rx=0 tx=0).stop 2023-11-27T15:23:55.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.104+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/311303233 shutdown_connections 2023-11-27T15:23:55.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.104+0000 7fa9aa7ff700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9a4101e50 0x7fa9a4073420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.104+0000 7fa9aa7ff700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa9a41002d0 0x7fa9a40748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.104+0000 7fa9aa7ff700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa98c0a1150 0x7fa98c0a3600 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.105+0000 7fa9aa7ff700 1 --2- 172.21.15.8:0/311303233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa9a41014c0 0x7fa9a4072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.105+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/311303233 >> 172.21.15.8:0/311303233 conn(0x7fa9a40fb8b0 msgr2=0x7fa9a4104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:55.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.105+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/311303233 shutdown_connections 2023-11-27T15:23:55.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.105+0000 7fa9aa7ff700 1 -- 172.21.15.8:0/311303233 wait complete. 2023-11-27T15:23:55.127 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:23:55.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.328+0000 7fa0c9bff700 1 -- 172.21.15.8:0/3970991521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0c41002d0 msgr2=0x7fa0c41006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:55.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.328+0000 7fa0c9bff700 1 --2- 172.21.15.8:0/3970991521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0c41002d0 0x7fa0c41006e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009990 tx=0x7fa0b4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.328+0000 7fa0c9bff700 1 -- 172.21.15.8:0/3970991521 shutdown_connections 2023-11-27T15:23:55.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.328+0000 7fa0c9bff700 1 --2- 172.21.15.8:0/3970991521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c4101e50 0x7fa0c4108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.328+0000 7fa0c9bff700 1 --2- 172.21.15.8:0/3970991521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0c41002d0 0x7fa0c41006e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.328+0000 7fa0c9bff700 1 --2- 172.21.15.8:0/3970991521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c41014c0 0x7fa0c4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.328+0000 7fa0c9bff700 1 -- 172.21.15.8:0/3970991521 >> 172.21.15.8:0/3970991521 conn(0x7fa0c40fb8b0 msgr2=0x7fa0c40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:55.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.328+0000 7fa0c9bff700 1 -- 172.21.15.8:0/3970991521 shutdown_connections 2023-11-27T15:23:55.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.328+0000 7fa0c9bff700 1 -- 172.21.15.8:0/3970991521 wait complete. 2023-11-27T15:23:55.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.329+0000 7fa0c9bff700 1 Processor -- start 2023-11-27T15:23:55.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.329+0000 7fa0c9bff700 1 -- start start 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.329+0000 7fa0c9bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0c41002d0 0x7fa0c41960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.329+0000 7fa0c9bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c41014c0 0x7fa0c41965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.329+0000 7fa0c9bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c4101e50 0x7fa0c419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c9bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c419dc30 con 0x7fa0c41002d0 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c9bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c419dd70 con 0x7fa0c4101e50 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c9bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c419df00 con 0x7fa0c41014c0 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0c41002d0 0x7fa0c41960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0c41002d0 0x7fa0c41960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59362/0 (socket says 172.21.15.8:59362) 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c37fe700 1 -- 172.21.15.8:0/1366935126 learned_addr learned my addr 172.21.15.8:0/1366935126 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c2ffd700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c41014c0 0x7fa0c41965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c3fff700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c4101e50 0x7fa0c419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:55.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c37fe700 1 -- 172.21.15.8:0/1366935126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c41014c0 msgr2=0x7fa0c41965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c37fe700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c41014c0 0x7fa0c41965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c37fe700 1 -- 172.21.15.8:0/1366935126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c4101e50 msgr2=0x7fa0c419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c37fe700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c4101e50 0x7fa0c419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c37fe700 1 -- 172.21.15.8:0/1366935126 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40095f0 con 0x7fa0c41002d0 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.330+0000 7fa0c2ffd700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c41014c0 0x7fa0c41965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.331+0000 7fa0c37fe700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0c41002d0 0x7fa0c41960b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400b620 tx=0x7fa0b4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.331+0000 7fa0c0ff9700 1 -- 172.21.15.8:0/1366935126 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b401d070 con 0x7fa0c41002d0 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.331+0000 7fa0c0ff9700 1 -- 172.21.15.8:0/1366935126 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0b4021410 con 0x7fa0c41002d0 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.331+0000 7fa0c0ff9700 1 -- 172.21.15.8:0/1366935126 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4017650 con 0x7fa0c41002d0 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.331+0000 7fa0c9bff700 1 -- 172.21.15.8:0/1366935126 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c410bca0 con 0x7fa0c41002d0 2023-11-27T15:23:55.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.331+0000 7fa0c9bff700 1 -- 172.21.15.8:0/1366935126 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c410bb10 con 0x7fa0c41002d0 2023-11-27T15:23:55.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.332+0000 7fa0c9bff700 1 -- 172.21.15.8:0/1366935126 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c4067af0 con 0x7fa0c41002d0 2023-11-27T15:23:55.337 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.334+0000 7fa0c0ff9700 1 -- 172.21.15.8:0/1366935126 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa0b4021580 con 0x7fa0c41002d0 2023-11-27T15:23:55.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.336+0000 7fa0c0ff9700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa0ac0a1150 0x7fa0ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:55.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.336+0000 7fa0c0ff9700 1 -- 172.21.15.8:0/1366935126 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa0b40cdff0 con 0x7fa0c41002d0 2023-11-27T15:23:55.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.336+0000 7fa0c2ffd700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa0ac0a1150 0x7fa0ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:55.340 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.337+0000 7fa0c2ffd700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa0ac0a1150 0x7fa0ac0a3600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8011490 tx=0x7fa0b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:55.343 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.340+0000 7fa0c0ff9700 1 -- 172.21.15.8:0/1366935126 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa0b40975c0 con 0x7fa0c41002d0 2023-11-27T15:23:55.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.551+0000 7fa0c9bff700 1 -- 172.21.15.8:0/1366935126 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0c41071b0 con 0x7fa0c41002d0 2023-11-27T15:23:55.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.553+0000 7fa0c0ff9700 1 -- 172.21.15.8:0/1366935126 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23817 (secure 0 0 0) 0x7fa0b409abe0 con 0x7fa0c41002d0 2023-11-27T15:23:55.557 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:23:55.557 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":88,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:23:32.146115+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:23:55.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.555+0000 7fa0c9bff700 1 -- 172.21.15.8:0/1366935126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa0ac0a1150 msgr2=0x7fa0ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:55.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa0ac0a1150 0x7fa0ac0a3600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8011490 tx=0x7fa0b8009260 comp rx=0 tx=0).stop 2023-11-27T15:23:55.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 -- 172.21.15.8:0/1366935126 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0c41002d0 msgr2=0x7fa0c41960b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:55.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0c41002d0 0x7fa0c41960b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400b620 tx=0x7fa0b4005740 comp rx=0 tx=0).stop 2023-11-27T15:23:55.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 -- 172.21.15.8:0/1366935126 shutdown_connections 2023-11-27T15:23:55.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa0c4101e50 0x7fa0c419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa0c41002d0 0x7fa0c41960b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa0ac0a1150 0x7fa0ac0a3600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 --2- 172.21.15.8:0/1366935126 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa0c41014c0 0x7fa0c41965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 -- 172.21.15.8:0/1366935126 >> 172.21.15.8:0/1366935126 conn(0x7fa0c40fb8b0 msgr2=0x7fa0c40fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:55.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 -- 172.21.15.8:0/1366935126 shutdown_connections 2023-11-27T15:23:55.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.556+0000 7fa0c9bff700 1 -- 172.21.15.8:0/1366935126 wait complete. 2023-11-27T15:23:55.578 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-27T15:23:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.784+0000 7f18513ff700 1 -- 172.21.15.8:0/2454736497 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f184c1036b0 msgr2=0x7f184c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.784+0000 7f18513ff700 1 --2- 172.21.15.8:0/2454736497 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f184c1036b0 0x7f184c105a90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1834009990 tx=0x7f1834009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.784+0000 7f18513ff700 1 -- 172.21.15.8:0/2454736497 shutdown_connections 2023-11-27T15:23:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.784+0000 7f18513ff700 1 --2- 172.21.15.8:0/2454736497 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f184c100d90 0x7f184c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.784+0000 7f18513ff700 1 --2- 172.21.15.8:0/2454736497 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f184c105fd0 0x7f184c1083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.784+0000 7f18513ff700 1 --2- 172.21.15.8:0/2454736497 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f184c1036b0 0x7f184c105a90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.784+0000 7f18513ff700 1 -- 172.21.15.8:0/2454736497 >> 172.21.15.8:0/2454736497 conn(0x7f184c0fa750 msgr2=0x7f184c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:55.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.784+0000 7f18513ff700 1 -- 172.21.15.8:0/2454736497 shutdown_connections 2023-11-27T15:23:55.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.784+0000 7f18513ff700 1 -- 172.21.15.8:0/2454736497 wait complete. 2023-11-27T15:23:55.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.786+0000 7f18513ff700 1 Processor -- start 2023-11-27T15:23:55.789 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.786+0000 7f18513ff700 1 -- start start 2023-11-27T15:23:55.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.786+0000 7f18513ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f184c100d90 0x7f184c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:55.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.786+0000 7f18513ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f184c1036b0 0x7f184c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.786+0000 7f18513ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f184c105fd0 0x7f184c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.786+0000 7f18513ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c19fda0 con 0x7f184c105fd0 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.786+0000 7f18513ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c19fee0 con 0x7f184c100d90 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.786+0000 7f18513ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c1a0070 con 0x7f184c1036b0 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f184b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f184c105fd0 0x7f184c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f184b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f184c105fd0 0x7f184c19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59372/0 (socket says 172.21.15.8:59372) 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f184b7fe700 1 -- 172.21.15.8:0/972329603 learned_addr learned my addr 172.21.15.8:0/972329603 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f184affd700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f184c100d90 0x7f184c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f1843fff700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f184c1036b0 0x7f184c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f184b7fe700 1 -- 172.21.15.8:0/972329603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f184c1036b0 msgr2=0x7f184c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:55.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f184b7fe700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f184c1036b0 0x7f184c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f184b7fe700 1 -- 172.21.15.8:0/972329603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f184c100d90 msgr2=0x7f184c198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:55.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f184b7fe700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f184c100d90 0x7f184c198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:55.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f184b7fe700 1 -- 172.21.15.8:0/972329603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18340095f0 con 0x7f184c105fd0 2023-11-27T15:23:55.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.787+0000 7f184b7fe700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f184c105fd0 0x7f184c19f7d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f184400efc0 tx=0x7f184400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:55.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.788+0000 7f1848ff9700 1 -- 172.21.15.8:0/972329603 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844018070 con 0x7f184c105fd0 2023-11-27T15:23:55.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.788+0000 7f18513ff700 1 -- 172.21.15.8:0/972329603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184c10bc60 con 0x7f184c105fd0 2023-11-27T15:23:55.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.788+0000 7f1848ff9700 1 -- 172.21.15.8:0/972329603 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f184400f040 con 0x7f184c105fd0 2023-11-27T15:23:55.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.788+0000 7f1848ff9700 1 -- 172.21.15.8:0/972329603 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400cca0 con 0x7f184c105fd0 2023-11-27T15:23:55.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.788+0000 7f18513ff700 1 -- 172.21.15.8:0/972329603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c10bad0 con 0x7f184c105fd0 2023-11-27T15:23:55.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.789+0000 7f18513ff700 1 -- 172.21.15.8:0/972329603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184c107b40 con 0x7f184c105fd0 2023-11-27T15:23:55.794 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.791+0000 7f1848ff9700 1 -- 172.21.15.8:0/972329603 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1844003a00 con 0x7f184c105fd0 2023-11-27T15:23:55.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.792+0000 7f1848ff9700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18380a1150 0x7f18380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:55.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.792+0000 7f1848ff9700 1 -- 172.21.15.8:0/972329603 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f18440cc520 con 0x7f184c105fd0 2023-11-27T15:23:55.796 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.793+0000 7f184affd700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18380a1150 0x7f18380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:55.797 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.793+0000 7f184affd700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18380a1150 0x7f18380a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f184c068880 tx=0x7f183c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:55.799 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:55.796+0000 7f1848ff9700 1 -- 172.21.15.8:0/972329603 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1844095ab0 con 0x7f184c105fd0 2023-11-27T15:23:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:55 smithi060 ceph-mon[131518]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:55 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/311303233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:55 smithi008 ceph-mon[128788]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:55 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/311303233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:55.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:55 smithi029 ceph-mon[131166]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:55.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:55 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/311303233' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:56.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.052+0000 7f18513ff700 1 -- 172.21.15.8:0/972329603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f184c02cfe0 con 0x7f184c105fd0 2023-11-27T15:23:56.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.423+0000 7f1848ff9700 1 -- 172.21.15.8:0/972329603 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f18440990d0 con 0x7f184c105fd0 2023-11-27T15:23:56.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 -- 172.21.15.8:0/972329603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18380a1150 msgr2=0x7f18380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:56.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18380a1150 0x7f18380a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f184c068880 tx=0x7f183c009260 comp rx=0 tx=0).stop 2023-11-27T15:23:56.430 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 -- 172.21.15.8:0/972329603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f184c105fd0 msgr2=0x7f184c19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:56.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f184c105fd0 0x7f184c19f7d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f184400efc0 tx=0x7f184400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:23:56.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 -- 172.21.15.8:0/972329603 shutdown_connections 2023-11-27T15:23:56.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f184c105fd0 0x7f184c19f7d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:56.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f184c100d90 0x7f184c198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:56.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18380a1150 0x7f18380a3600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:56.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 --2- 172.21.15.8:0/972329603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f184c1036b0 0x7f184c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:56.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 -- 172.21.15.8:0/972329603 >> 172.21.15.8:0/972329603 conn(0x7f184c0fa750 msgr2=0x7f184c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:56.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 -- 172.21.15.8:0/972329603 shutdown_connections 2023-11-27T15:23:56.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.426+0000 7f18513ff700 1 -- 172.21.15.8:0/972329603 wait complete. 2023-11-27T15:23:56.450 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-11-27T15:23:56.450 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-27T15:23:56.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.655+0000 7f1e749ff700 1 -- 172.21.15.8:0/1503747028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e70101480 msgr2=0x7f1e701018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:56.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.655+0000 7f1e749ff700 1 --2- 172.21.15.8:0/1503747028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e70101480 0x7f1e701018d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e640098f0 tx=0x7f1e64009c00 comp rx=0 tx=0).stop 2023-11-27T15:23:56.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.655+0000 7f1e749ff700 1 -- 172.21.15.8:0/1503747028 shutdown_connections 2023-11-27T15:23:56.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.655+0000 7f1e749ff700 1 --2- 172.21.15.8:0/1503747028 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e70101e10 0x7f1e701086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:56.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.655+0000 7f1e749ff700 1 --2- 172.21.15.8:0/1503747028 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e70100290 0x7f1e701006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:56.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.655+0000 7f1e749ff700 1 --2- 172.21.15.8:0/1503747028 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e70101480 0x7f1e701018d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:56.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.655+0000 7f1e749ff700 1 -- 172.21.15.8:0/1503747028 >> 172.21.15.8:0/1503747028 conn(0x7f1e700fb870 msgr2=0x7f1e700fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:56.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.655+0000 7f1e749ff700 1 -- 172.21.15.8:0/1503747028 shutdown_connections 2023-11-27T15:23:56.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.655+0000 7f1e749ff700 1 -- 172.21.15.8:0/1503747028 wait complete. 2023-11-27T15:23:56.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.656+0000 7f1e749ff700 1 Processor -- start 2023-11-27T15:23:56.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e749ff700 1 -- start start 2023-11-27T15:23:56.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e749ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e70100290 0x7f1e700748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:56.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e749ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e70101480 0x7f1e70072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:56.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e749ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e70101e10 0x7f1e70073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:56.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e749ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70073bc0 con 0x7f1e70101e10 2023-11-27T15:23:56.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e749ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70073d30 con 0x7f1e70100290 2023-11-27T15:23:56.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e749ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e701a00d0 con 0x7f1e70101480 2023-11-27T15:23:56.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e6ed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e70101e10 0x7f1e70073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:56.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e6e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e70100290 0x7f1e700748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:56.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e6e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e70100290 0x7f1e700748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:53456/0 (socket says 172.21.15.8:53456) 2023-11-27T15:23:56.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e6e59c700 1 -- 172.21.15.8:0/1173916574 learned_addr learned my addr 172.21.15.8:0/1173916574 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:56.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e6ed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e70101e10 0x7f1e70073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59392/0 (socket says 172.21.15.8:59392) 2023-11-27T15:23:56.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.657+0000 7f1e6dd9b700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e70101480 0x7f1e70072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:56.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e6ed9d700 1 -- 172.21.15.8:0/1173916574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e70101480 msgr2=0x7f1e70072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:56.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e6ed9d700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e70101480 0x7f1e70072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:56.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e6ed9d700 1 -- 172.21.15.8:0/1173916574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e70100290 msgr2=0x7f1e700748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:56.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e6ed9d700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e70100290 0x7f1e700748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:56.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e6ed9d700 1 -- 172.21.15.8:0/1173916574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68009d00 con 0x7f1e70101e10 2023-11-27T15:23:56.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e6e59c700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e70100290 0x7f1e700748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:56.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e6ed9d700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e70101e10 0x7f1e70073420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009cd0 tx=0x7f1e6800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:56.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e5f7fe700 1 -- 172.21.15.8:0/1173916574 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68017070 con 0x7f1e70101e10 2023-11-27T15:23:56.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e749ff700 1 -- 172.21.15.8:0/1173916574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e64009520 con 0x7f1e70101e10 2023-11-27T15:23:56.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e5f7fe700 1 -- 172.21.15.8:0/1173916574 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e6800f040 con 0x7f1e70101e10 2023-11-27T15:23:56.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e5f7fe700 1 -- 172.21.15.8:0/1173916574 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800e050 con 0x7f1e70101e10 2023-11-27T15:23:56.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.658+0000 7f1e749ff700 1 -- 172.21.15.8:0/1173916574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7010bbf0 con 0x7f1e70101e10 2023-11-27T15:23:56.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.659+0000 7f1e749ff700 1 -- 172.21.15.8:0/1173916574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7010bd80 con 0x7f1e70101e10 2023-11-27T15:23:56.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.661+0000 7f1e5f7fe700 1 -- 172.21.15.8:0/1173916574 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1e6800cd10 con 0x7f1e70101e10 2023-11-27T15:23:56.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.663+0000 7f1e5f7fe700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1e580a1150 0x7f1e580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:56.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.663+0000 7f1e5f7fe700 1 -- 172.21.15.8:0/1173916574 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1e680cc3f0 con 0x7f1e70101e10 2023-11-27T15:23:56.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.663+0000 7f1e6e59c700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1e580a1150 0x7f1e580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:56.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.664+0000 7f1e6e59c700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1e580a1150 0x7f1e580a3600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1e60005950 tx=0x7f1e60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:56.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.667+0000 7f1e5f7fe700 1 -- 172.21.15.8:0/1173916574 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1e680958f0 con 0x7f1e70101e10 2023-11-27T15:23:56.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:56 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1366935126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:56.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:56 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/972329603' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-27T15:23:56.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:56 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1366935126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:56.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:56 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/972329603' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-27T15:23:56.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:56 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1366935126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:23:56.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:56 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/972329603' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-27T15:23:56.926 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:56.923+0000 7f1e749ff700 1 -- 172.21.15.8:0/1173916574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f1e700617a0 con 0x7f1e70101e10 2023-11-27T15:23:57.433 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.429+0000 7f1e5f7fe700 1 -- 172.21.15.8:0/1173916574 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f1e68098f10 con 0x7f1e70101e10 2023-11-27T15:23:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 -- 172.21.15.8:0/1173916574 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1e580a1150 msgr2=0x7f1e580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1e580a1150 0x7f1e580a3600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1e60005950 tx=0x7f1e60009260 comp rx=0 tx=0).stop 2023-11-27T15:23:57.436 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 -- 172.21.15.8:0/1173916574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e70101e10 msgr2=0x7f1e70073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e70101e10 0x7f1e70073420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009cd0 tx=0x7f1e6800c620 comp rx=0 tx=0).stop 2023-11-27T15:23:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 -- 172.21.15.8:0/1173916574 shutdown_connections 2023-11-27T15:23:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e70101e10 0x7f1e70073420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1e70100290 0x7f1e700748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1e580a1150 0x7f1e580a3600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 --2- 172.21.15.8:0/1173916574 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1e70101480 0x7f1e70072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 -- 172.21.15.8:0/1173916574 >> 172.21.15.8:0/1173916574 conn(0x7f1e700fb870 msgr2=0x7f1e70104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:57.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 -- 172.21.15.8:0/1173916574 shutdown_connections 2023-11-27T15:23:57.438 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.432+0000 7f1e749ff700 1 -- 172.21.15.8:0/1173916574 wait complete. 2023-11-27T15:23:57.460 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-11-27T15:23:57.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.664+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/3184141654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6a4074b30 msgr2=0x7fc6a4072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.664+0000 7fc6ac5ff700 1 --2- 172.21.15.8:0/3184141654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6a4074b30 0x7fc6a4072f80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009990 tx=0x7fc6a0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.664+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/3184141654 shutdown_connections 2023-11-27T15:23:57.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.664+0000 7fc6ac5ff700 1 --2- 172.21.15.8:0/3184141654 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a410c860 0x7fc6a410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.664+0000 7fc6ac5ff700 1 --2- 172.21.15.8:0/3184141654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6a4074b30 0x7fc6a4072f80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.664+0000 7fc6ac5ff700 1 --2- 172.21.15.8:0/3184141654 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6a4073550 0x7fc6a40739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.664+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/3184141654 >> 172.21.15.8:0/3184141654 conn(0x7fc6a40fb870 msgr2=0x7fc6a40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:57.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.664+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/3184141654 shutdown_connections 2023-11-27T15:23:57.670 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.664+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/3184141654 wait complete. 2023-11-27T15:23:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.665+0000 7fc6ac5ff700 1 Processor -- start 2023-11-27T15:23:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6ac5ff700 1 -- start start 2023-11-27T15:23:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6ac5ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6a4073550 0x7fc6a419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6ac5ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6a4074b30 0x7fc6a419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6ac5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a410c860 0x7fc6a41a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.673 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6ac5ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a41a4170 con 0x7fc6a4074b30 2023-11-27T15:23:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6ac5ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a41a42b0 con 0x7fc6a4073550 2023-11-27T15:23:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6ac5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a41a4440 con 0x7fc6a410c860 2023-11-27T15:23:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6a9b9a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6a4074b30 0x7fc6a419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6a9b9a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6a4074b30 0x7fc6a419cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59402/0 (socket says 172.21.15.8:59402) 2023-11-27T15:23:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6a9b9a700 1 -- 172.21.15.8:0/254433968 learned_addr learned my addr 172.21.15.8:0/254433968 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6aa39b700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6a4073550 0x7fc6a419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.666+0000 7fc6aab9c700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a410c860 0x7fc6a41a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc6aa39b700 1 -- 172.21.15.8:0/254433968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a410c860 msgr2=0x7fc6a41a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc6aa39b700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a410c860 0x7fc6a41a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc6aa39b700 1 -- 172.21.15.8:0/254433968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6a4074b30 msgr2=0x7fc6a419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc6aa39b700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6a4074b30 0x7fc6a419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc6aa39b700 1 -- 172.21.15.8:0/254433968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a00095f0 con 0x7fc6a4073550 2023-11-27T15:23:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc6a9b9a700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6a4074b30 0x7fc6a419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc6aa39b700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6a4073550 0x7fc6a419c5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00098e0 tx=0x7fc6a0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc69b7fe700 1 -- 172.21.15.8:0/254433968 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a001d070 con 0x7fc6a4073550 2023-11-27T15:23:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/254433968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a4112250 con 0x7fc6a4073550 2023-11-27T15:23:57.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc69b7fe700 1 -- 172.21.15.8:0/254433968 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6a0021410 con 0x7fc6a4073550 2023-11-27T15:23:57.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc69b7fe700 1 -- 172.21.15.8:0/254433968 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0017650 con 0x7fc6a4073550 2023-11-27T15:23:57.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.667+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/254433968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a41120c0 con 0x7fc6a4073550 2023-11-27T15:23:57.676 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.668+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/254433968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a41023b0 con 0x7fc6a4073550 2023-11-27T15:23:57.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.670+0000 7fc69b7fe700 1 -- 172.21.15.8:0/254433968 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc6a0021580 con 0x7fc6a4073550 2023-11-27T15:23:57.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.672+0000 7fc69b7fe700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc6900a1150 0x7fc6900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.672+0000 7fc69b7fe700 1 -- 172.21.15.8:0/254433968 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc6a00ce500 con 0x7fc6a4073550 2023-11-27T15:23:57.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.672+0000 7fc6a9b9a700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc6900a1150 0x7fc6900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.673+0000 7fc6a9b9a700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc6900a1150 0x7fc6900a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc694005950 tx=0x7fc694009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:57.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.674+0000 7fc69b7fe700 1 -- 172.21.15.8:0/254433968 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc6a0097a90 con 0x7fc6a4073550 2023-11-27T15:23:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:57 smithi060 ceph-mon[131518]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:57 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/972329603' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-27T15:23:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:57 smithi060 ceph-mon[131518]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:57.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:57 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1173916574' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:23:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:57 smithi008 ceph-mon[128788]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:57 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/972329603' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-27T15:23:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:57 smithi008 ceph-mon[128788]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:57 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1173916574' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:23:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:57 smithi029 ceph-mon[131166]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:23:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:57 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/972329603' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-27T15:23:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:57 smithi029 ceph-mon[131166]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:57.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:57 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1173916574' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:23:57.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:57.929+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/254433968 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fc6a4067a50 con 0x7fc6a4073550 2023-11-27T15:23:58.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.235+0000 7fc69b7fe700 1 -- 172.21.15.8:0/254433968 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fc6a009b0b0 con 0x7fc6a4073550 2023-11-27T15:23:58.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/254433968 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc6900a1150 msgr2=0x7fc6900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:58.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc6900a1150 0x7fc6900a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc694005950 tx=0x7fc694009260 comp rx=0 tx=0).stop 2023-11-27T15:23:58.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/254433968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6a4073550 msgr2=0x7fc6a419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:58.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6a4073550 0x7fc6a419c5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00098e0 tx=0x7fc6a0009580 comp rx=0 tx=0).stop 2023-11-27T15:23:58.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/254433968 shutdown_connections 2023-11-27T15:23:58.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc6900a1150 0x7fc6900a3600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:58.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc6a410c860 0x7fc6a41a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:58.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc6a4073550 0x7fc6a419c5f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:58.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 --2- 172.21.15.8:0/254433968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6a4074b30 0x7fc6a419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:58.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/254433968 >> 172.21.15.8:0/254433968 conn(0x7fc6a40fb870 msgr2=0x7fc6a40fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:58.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/254433968 shutdown_connections 2023-11-27T15:23:58.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.238+0000 7fc6ac5ff700 1 -- 172.21.15.8:0/254433968 wait complete. 2023-11-27T15:23:58.267 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-11-27T15:23:58.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.475+0000 7f591ebff700 1 -- 172.21.15.8:0/1554257495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5918101e10 msgr2=0x7f59181086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:58.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.475+0000 7f591ebff700 1 --2- 172.21.15.8:0/1554257495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5918101e10 0x7f59181086d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f591000b0a0 tx=0x7f591000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:23:58.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.475+0000 7f591ebff700 1 -- 172.21.15.8:0/1554257495 shutdown_connections 2023-11-27T15:23:58.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.475+0000 7f591ebff700 1 --2- 172.21.15.8:0/1554257495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5918101e10 0x7f59181086d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:58.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.475+0000 7f591ebff700 1 --2- 172.21.15.8:0/1554257495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5918100290 0x7f59181006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:58.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.475+0000 7f591ebff700 1 --2- 172.21.15.8:0/1554257495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5918101480 0x7f59181018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:58.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.475+0000 7f591ebff700 1 -- 172.21.15.8:0/1554257495 >> 172.21.15.8:0/1554257495 conn(0x7f59180fb870 msgr2=0x7f59180fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:58.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.475+0000 7f591ebff700 1 -- 172.21.15.8:0/1554257495 shutdown_connections 2023-11-27T15:23:58.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.475+0000 7f591ebff700 1 -- 172.21.15.8:0/1554257495 wait complete. 2023-11-27T15:23:58.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.476+0000 7f591ebff700 1 Processor -- start 2023-11-27T15:23:58.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591ebff700 1 -- start start 2023-11-27T15:23:58.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591ebff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5918100290 0x7f59180748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:58.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591ebff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5918101480 0x7f5918072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:58.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591ebff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5918101e10 0x7f5918073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:58.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591ebff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5918073bc0 con 0x7f5918100290 2023-11-27T15:23:58.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591ebff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5918073d30 con 0x7f5918101480 2023-11-27T15:23:58.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591ebff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59181a0160 con 0x7f5918101e10 2023-11-27T15:23:58.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591c99b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5918100290 0x7f59180748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:58.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591c99b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5918100290 0x7f59180748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59420/0 (socket says 172.21.15.8:59420) 2023-11-27T15:23:58.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591c99b700 1 -- 172.21.15.8:0/1610372897 learned_addr learned my addr 172.21.15.8:0/1610372897 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:58.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f5917fff700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5918101480 0x7f5918072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:58.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.477+0000 7f591d19c700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5918101e10 0x7f5918073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:58.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f591c99b700 1 -- 172.21.15.8:0/1610372897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5918101e10 msgr2=0x7f5918073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:58.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f591c99b700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5918101e10 0x7f5918073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:58.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f591c99b700 1 -- 172.21.15.8:0/1610372897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5918101480 msgr2=0x7f5918072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:58.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f591c99b700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5918101480 0x7f5918072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:58.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f591c99b700 1 -- 172.21.15.8:0/1610372897 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910009d00 con 0x7f5918100290 2023-11-27T15:23:58.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f591c99b700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5918100290 0x7f59180748a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f590c00d980 tx=0x7f590c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:58.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f5915ffb700 1 -- 172.21.15.8:0/1610372897 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c00f890 con 0x7f5918100290 2023-11-27T15:23:58.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f5915ffb700 1 -- 172.21.15.8:0/1610372897 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f590c010460 con 0x7f5918100290 2023-11-27T15:23:58.485 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f591ebff700 1 -- 172.21.15.8:0/1610372897 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591810bc70 con 0x7f5918100290 2023-11-27T15:23:58.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f5915ffb700 1 -- 172.21.15.8:0/1610372897 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c0195c0 con 0x7f5918100290 2023-11-27T15:23:58.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.478+0000 7f591ebff700 1 -- 172.21.15.8:0/1610372897 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591810bae0 con 0x7f5918100290 2023-11-27T15:23:58.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.479+0000 7f591ebff700 1 -- 172.21.15.8:0/1610372897 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5918107840 con 0x7f5918100290 2023-11-27T15:23:58.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.481+0000 7f5915ffb700 1 -- 172.21.15.8:0/1610372897 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f590c0105d0 con 0x7f5918100290 2023-11-27T15:23:58.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.483+0000 7f5915ffb700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59080a1150 0x7f59080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:58.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.483+0000 7f5915ffb700 1 -- 172.21.15.8:0/1610372897 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f590c0cc8c0 con 0x7f5918100290 2023-11-27T15:23:58.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.483+0000 7f5917fff700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59080a1150 0x7f59080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:58.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.484+0000 7f5917fff700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59080a1150 0x7f59080a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5904005950 tx=0x7f590400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:58.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.487+0000 7f5915ffb700 1 -- 172.21.15.8:0/1610372897 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f590c095dc0 con 0x7f5918100290 2023-11-27T15:23:58.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-27T15:23:58.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1173916574' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-27T15:23:58.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-27T15:23:58.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:58.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2023-11-27T15:23:58.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-11-27T15:23:58.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/254433968' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:23:58.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:23:58.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] up:active 2023-11-27T15:23:58.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-27T15:23:58.614 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-27T15:23:58.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-27T15:23:58.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: Cluster is now healthy 2023-11-27T15:23:58.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 2 up:standby-replay 8 up:standby 2023-11-27T15:23:58.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2023-11-27T15:23:58.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-mon[128788]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-11-27T15:23:58.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:58.747+0000 7f591ebff700 1 -- 172.21.15.8:0/1610372897 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f591802cfe0 con 0x7f5918100290 2023-11-27T15:23:58.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-27T15:23:58.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1173916574' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-27T15:23:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-27T15:23:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2023-11-27T15:23:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-11-27T15:23:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/254433968' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:23:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:23:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] up:active 2023-11-27T15:23:58.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-27T15:23:58.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-27T15:23:58.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-27T15:23:58.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: Cluster is now healthy 2023-11-27T15:23:58.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 2 up:standby-replay 8 up:standby 2023-11-27T15:23:58.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2023-11-27T15:23:58.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:58 smithi060 ceph-mon[131518]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-11-27T15:23:58.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-27T15:23:58.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1173916574' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-27T15:23:58.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-27T15:23:58.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: fsmap cephfs:1 {0=a=up:active} 11 up:standby 2023-11-27T15:23:58.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: fsmap cephfs:2 {0=a=up:active,1=b=up:creating} 10 up:standby 2023-11-27T15:23:58.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-11-27T15:23:58.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/254433968' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:23:58.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:23:58.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] up:active 2023-11-27T15:23:58.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-27T15:23:58.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-27T15:23:58.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-27T15:23:58.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: Cluster is now healthy 2023-11-27T15:23:58.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: fsmap cephfs:2 {0=a=up:active,1=b=up:active} 2 up:standby-replay 8 up:standby 2023-11-27T15:23:58.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2023-11-27T15:23:58.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:58 smithi029 ceph-mon[131166]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-11-27T15:23:59.113 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:23:58 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-a[163286]: 2023-11-27T15:23:58.767+0000 7f4f5d3c2700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-27T15:23:59.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.240+0000 7f5915ffb700 1 -- 172.21.15.8:0/1610372897 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f590c0200b0 con 0x7f5918100290 2023-11-27T15:23:59.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 -- 172.21.15.8:0/1610372897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59080a1150 msgr2=0x7f59080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:59.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59080a1150 0x7f59080a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5904005950 tx=0x7f590400b410 comp rx=0 tx=0).stop 2023-11-27T15:23:59.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 -- 172.21.15.8:0/1610372897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5918100290 msgr2=0x7f59180748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:59.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5918100290 0x7f59180748a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f590c00d980 tx=0x7f590c00dc90 comp rx=0 tx=0).stop 2023-11-27T15:23:59.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 -- 172.21.15.8:0/1610372897 shutdown_connections 2023-11-27T15:23:59.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f59080a1150 0x7f59080a3600 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5918101e10 0x7f5918073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5918100290 0x7f59180748a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 --2- 172.21.15.8:0/1610372897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5918101480 0x7f5918072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 -- 172.21.15.8:0/1610372897 >> 172.21.15.8:0/1610372897 conn(0x7f59180fb870 msgr2=0x7f5918104980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:59.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 -- 172.21.15.8:0/1610372897 shutdown_connections 2023-11-27T15:23:59.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.243+0000 7f591ebff700 1 -- 172.21.15.8:0/1610372897 wait complete. 2023-11-27T15:23:59.267 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-11-27T15:23:59.267 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-11-27T15:23:59.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.472+0000 7fe9939ff700 1 -- 172.21.15.8:0/3271999675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe98c068b00 msgr2=0x7fe98c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:59.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.472+0000 7fe9939ff700 1 --2- 172.21.15.8:0/3271999675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe98c068b00 0x7fe98c102e00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe988009990 tx=0x7fe988009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.472+0000 7fe9939ff700 1 -- 172.21.15.8:0/3271999675 shutdown_connections 2023-11-27T15:23:59.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.472+0000 7fe9939ff700 1 --2- 172.21.15.8:0/3271999675 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe98c105e50 0x7fe98c108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.472+0000 7fe9939ff700 1 --2- 172.21.15.8:0/3271999675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe98c068b00 0x7fe98c102e00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.472+0000 7fe9939ff700 1 --2- 172.21.15.8:0/3271999675 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c103340 0x7fe98c1057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.472+0000 7fe9939ff700 1 -- 172.21.15.8:0/3271999675 >> 172.21.15.8:0/3271999675 conn(0x7fe98c0fa750 msgr2=0x7fe98c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:59.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.472+0000 7fe9939ff700 1 -- 172.21.15.8:0/3271999675 shutdown_connections 2023-11-27T15:23:59.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.472+0000 7fe9939ff700 1 -- 172.21.15.8:0/3271999675 wait complete. 2023-11-27T15:23:59.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.473+0000 7fe9939ff700 1 Processor -- start 2023-11-27T15:23:59.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.473+0000 7fe9939ff700 1 -- start start 2023-11-27T15:23:59.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe9939ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe98c068b00 0x7fe98c100760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:59.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe9939ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe98c103340 0x7fe98c0feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe9939ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c105e50 0x7fe98c0ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe9939ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0ffa80 con 0x7fe98c068b00 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe9939ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0ffbf0 con 0x7fe98c105e50 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe9939ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c1a0050 con 0x7fe98c103340 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe99179b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe98c068b00 0x7fe98c100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe991f9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c105e50 0x7fe98c0ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe99179b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe98c068b00 0x7fe98c100760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59442/0 (socket says 172.21.15.8:59442) 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe991f9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c105e50 0x7fe98c0ff2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:53528/0 (socket says 172.21.15.8:53528) 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe99179b700 1 -- 172.21.15.8:0/202648991 learned_addr learned my addr 172.21.15.8:0/202648991 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.474+0000 7fe990f9a700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe98c103340 0x7fe98c0feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:59.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe991f9c700 1 -- 172.21.15.8:0/202648991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe98c103340 msgr2=0x7fe98c0feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe991f9c700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe98c103340 0x7fe98c0feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe991f9c700 1 -- 172.21.15.8:0/202648991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe98c068b00 msgr2=0x7fe98c100760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe991f9c700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe98c068b00 0x7fe98c100760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe991f9c700 1 -- 172.21.15.8:0/202648991 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9880095f0 con 0x7fe98c105e50 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe99179b700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe98c068b00 0x7fe98c100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe990f9a700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe98c103340 0x7fe98c0feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe991f9c700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c105e50 0x7fe98c0ff2e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe98400efc0 tx=0x7fe98400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe9827fc700 1 -- 172.21.15.8:0/202648991 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984018070 con 0x7fe98c105e50 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe9939ff700 1 -- 172.21.15.8:0/202648991 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c10bc30 con 0x7fe98c105e50 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe9827fc700 1 -- 172.21.15.8:0/202648991 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe98400f040 con 0x7fe98c105e50 2023-11-27T15:23:59.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe9827fc700 1 -- 172.21.15.8:0/202648991 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98400cd40 con 0x7fe98c105e50 2023-11-27T15:23:59.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.475+0000 7fe9939ff700 1 -- 172.21.15.8:0/202648991 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c10baa0 con 0x7fe98c105e50 2023-11-27T15:23:59.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.476+0000 7fe9939ff700 1 -- 172.21.15.8:0/202648991 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c0fc2c0 con 0x7fe98c105e50 2023-11-27T15:23:59.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.478+0000 7fe9827fc700 1 -- 172.21.15.8:0/202648991 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe984003a00 con 0x7fe98c105e50 2023-11-27T15:23:59.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.480+0000 7fe9827fc700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9780a11d0 0x7fe9780a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:59.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.480+0000 7fe9827fc700 1 -- 172.21.15.8:0/202648991 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe9840cc630 con 0x7fe98c105e50 2023-11-27T15:23:59.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.480+0000 7fe99179b700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9780a11d0 0x7fe9780a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:59.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.481+0000 7fe99179b700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9780a11d0 0x7fe9780a3680 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b2e0 tx=0x7fe988005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:59.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.484+0000 7fe9827fc700 1 -- 172.21.15.8:0/202648991 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe984095bc0 con 0x7fe98c105e50 2023-11-27T15:23:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-mon[128788]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-11-27T15:23:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1610372897' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-27T15:23:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] up:active 2023-11-27T15:23:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1610372897' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-27T15:23:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:23:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:23:59.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.727+0000 7fe9939ff700 1 -- 172.21.15.8:0/202648991 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fe98c061c40 con 0x7fe9780a11d0 2023-11-27T15:23:59.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.774+0000 7fe9827fc700 1 -- 172.21.15.8:0/202648991 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe98c061c40 con 0x7fe9780a11d0 2023-11-27T15:23:59.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 -- 172.21.15.8:0/202648991 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9780a11d0 msgr2=0x7fe9780a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:59.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9780a11d0 0x7fe9780a3680 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b2e0 tx=0x7fe988005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 -- 172.21.15.8:0/202648991 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c105e50 msgr2=0x7fe98c0ff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:59.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c105e50 0x7fe98c0ff2e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe98400efc0 tx=0x7fe98400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 -- 172.21.15.8:0/202648991 shutdown_connections 2023-11-27T15:23:59.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe98c105e50 0x7fe98c0ff2e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe98c068b00 0x7fe98c100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9780a11d0 0x7fe9780a3680 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 --2- 172.21.15.8:0/202648991 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe98c103340 0x7fe98c0feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:59.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 -- 172.21.15.8:0/202648991 >> 172.21.15.8:0/202648991 conn(0x7fe98c0fa750 msgr2=0x7fe98c101680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:59.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 -- 172.21.15.8:0/202648991 shutdown_connections 2023-11-27T15:23:59.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:23:59.777+0000 7fe9939ff700 1 -- 172.21.15.8:0/202648991 wait complete. 2023-11-27T15:23:59.800 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-11-27T15:23:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:59 smithi060 ceph-mon[131518]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-11-27T15:23:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:59 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1610372897' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-27T15:23:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:59 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] up:active 2023-11-27T15:23:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:59 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1610372897' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-27T15:23:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:59 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:23:59.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:23:59 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:23:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:59 smithi029 ceph-mon[131166]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-11-27T15:23:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:59 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1610372897' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-27T15:23:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:59 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] up:active 2023-11-27T15:23:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:59 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1610372897' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-27T15:23:59.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:59 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:23:59.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:23:59 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:24:00.007 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.004+0000 7f248b3ff700 1 -- 172.21.15.8:0/318274286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2484100290 msgr2=0x7f24841006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:00.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.004+0000 7f248b3ff700 1 --2- 172.21.15.8:0/318274286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2484100290 0x7f24841006a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2480009990 tx=0x7f2480009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.004+0000 7f248b3ff700 1 -- 172.21.15.8:0/318274286 shutdown_connections 2023-11-27T15:24:00.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.004+0000 7f248b3ff700 1 --2- 172.21.15.8:0/318274286 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2484101e10 0x7f24841086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.004+0000 7f248b3ff700 1 --2- 172.21.15.8:0/318274286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2484100290 0x7f24841006a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.004+0000 7f248b3ff700 1 --2- 172.21.15.8:0/318274286 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2484101480 0x7f24841018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.004+0000 7f248b3ff700 1 -- 172.21.15.8:0/318274286 >> 172.21.15.8:0/318274286 conn(0x7f24840fb870 msgr2=0x7f24840fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:00.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.004+0000 7f248b3ff700 1 -- 172.21.15.8:0/318274286 shutdown_connections 2023-11-27T15:24:00.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.004+0000 7f248b3ff700 1 -- 172.21.15.8:0/318274286 wait complete. 2023-11-27T15:24:00.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.005+0000 7f248b3ff700 1 Processor -- start 2023-11-27T15:24:00.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.005+0000 7f248b3ff700 1 -- start start 2023-11-27T15:24:00.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.005+0000 7f248b3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2484100290 0x7f2484198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:00.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.005+0000 7f248b3ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2484101480 0x7f24841986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:00.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.005+0000 7f248b3ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2484101e10 0x7f248419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:00.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.005+0000 7f248b3ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419fd10 con 0x7f2484101480 2023-11-27T15:24:00.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.005+0000 7f248b3ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419fe50 con 0x7f2484101e10 2023-11-27T15:24:00.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.005+0000 7f248b3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419ffe0 con 0x7f2484100290 2023-11-27T15:24:00.011 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2484100290 0x7f2484198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248899a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2484101480 0x7f24841986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248919b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2484100290 0x7f2484198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:52010/0 (socket says 172.21.15.8:52010) 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248919b700 1 -- 172.21.15.8:0/1902224318 learned_addr learned my addr 172.21.15.8:0/1902224318 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248999c700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2484101e10 0x7f248419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248919b700 1 -- 172.21.15.8:0/1902224318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2484101e10 msgr2=0x7f248419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248919b700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2484101e10 0x7f248419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248919b700 1 -- 172.21.15.8:0/1902224318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2484101480 msgr2=0x7f24841986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248919b700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2484101480 0x7f24841986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248919b700 1 -- 172.21.15.8:0/1902224318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24800095f0 con 0x7f2484100290 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248899a700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2484101480 0x7f24841986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:00.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.006+0000 7f248919b700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2484100290 0x7f2484198190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2480005740 tx=0x7f2480005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:00.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.007+0000 7f247a7fc700 1 -- 172.21.15.8:0/1902224318 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248001d070 con 0x7f2484100290 2023-11-27T15:24:00.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.007+0000 7f248b3ff700 1 -- 172.21.15.8:0/1902224318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248410bc40 con 0x7f2484100290 2023-11-27T15:24:00.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.007+0000 7f247a7fc700 1 -- 172.21.15.8:0/1902224318 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2480021410 con 0x7f2484100290 2023-11-27T15:24:00.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.007+0000 7f247a7fc700 1 -- 172.21.15.8:0/1902224318 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480017650 con 0x7f2484100290 2023-11-27T15:24:00.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.007+0000 7f248b3ff700 1 -- 172.21.15.8:0/1902224318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248410bab0 con 0x7f2484100290 2023-11-27T15:24:00.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.008+0000 7f248b3ff700 1 -- 172.21.15.8:0/1902224318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484067a50 con 0x7f2484100290 2023-11-27T15:24:00.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.010+0000 7f247a7fc700 1 -- 172.21.15.8:0/1902224318 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2480021580 con 0x7f2484100290 2023-11-27T15:24:00.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.011+0000 7f247a7fc700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24700a1150 0x7f24700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:00.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.011+0000 7f247a7fc700 1 -- 172.21.15.8:0/1902224318 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f24800ce0f0 con 0x7f2484100290 2023-11-27T15:24:00.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.011+0000 7f248899a700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24700a1150 0x7f24700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:00.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.012+0000 7f248899a700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24700a1150 0x7f24700a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2474011490 tx=0x7f2474009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:00.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.014+0000 7f247a7fc700 1 -- 172.21.15.8:0/1902224318 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2480097680 con 0x7f2484100290 2023-11-27T15:24:00.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:23:59.733+0000 7f94d0778700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:24:00.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:23:59.733+0000 7f94d0778700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:24:00.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:23:59.733+0000 7f94d0778700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:24:00.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:23:59.733+0000 7f94d0778700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:24:00.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:23:59 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:23:59.733+0000 7f94d0778700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:24:00.260 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.257+0000 7f248b3ff700 1 -- 172.21.15.8:0/1902224318 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f2484107e20 con 0x7f24700a1150 2023-11-27T15:24:00.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.261+0000 7f247a7fc700 1 -- 172.21.15.8:0/1902224318 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f2484107e20 con 0x7f24700a1150 2023-11-27T15:24:00.265 INFO:teuthology.orchestra.run.smithi008.stdout:/volumes/_nogroup/sv_0/bf0bef1f-d74b-4aa8-862a-1367d7cae8e3 2023-11-27T15:24:00.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 -- 172.21.15.8:0/1902224318 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24700a1150 msgr2=0x7f24700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:00.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24700a1150 0x7f24700a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2474011490 tx=0x7f2474009260 comp rx=0 tx=0).stop 2023-11-27T15:24:00.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 -- 172.21.15.8:0/1902224318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2484100290 msgr2=0x7f2484198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:00.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2484100290 0x7f2484198190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2480005740 tx=0x7f2480005770 comp rx=0 tx=0).stop 2023-11-27T15:24:00.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 -- 172.21.15.8:0/1902224318 shutdown_connections 2023-11-27T15:24:00.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f24700a1150 0x7f24700a3600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2484100290 0x7f2484198190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2484101e10 0x7f248419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 --2- 172.21.15.8:0/1902224318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2484101480 0x7f24841986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 -- 172.21.15.8:0/1902224318 >> 172.21.15.8:0/1902224318 conn(0x7f24840fb870 msgr2=0x7f24840fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:00.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 -- 172.21.15.8:0/1902224318 shutdown_connections 2023-11-27T15:24:00.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.264+0000 7f248b3ff700 1 -- 172.21.15.8:0/1902224318 wait complete. 2023-11-27T15:24:00.286 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-11-27T15:24:00.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.496+0000 7f3560dff700 1 -- 172.21.15.8:0/1228313448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f355c100290 msgr2=0x7f355c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:00.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.496+0000 7f3560dff700 1 --2- 172.21.15.8:0/1228313448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f355c100290 0x7f355c1006a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f354c009990 tx=0x7f354c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.496+0000 7f3560dff700 1 -- 172.21.15.8:0/1228313448 shutdown_connections 2023-11-27T15:24:00.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.496+0000 7f3560dff700 1 --2- 172.21.15.8:0/1228313448 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f355c101e10 0x7f355c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.496+0000 7f3560dff700 1 --2- 172.21.15.8:0/1228313448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f355c100290 0x7f355c1006a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.496+0000 7f3560dff700 1 --2- 172.21.15.8:0/1228313448 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f355c101480 0x7f355c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.496+0000 7f3560dff700 1 -- 172.21.15.8:0/1228313448 >> 172.21.15.8:0/1228313448 conn(0x7f355c0fb870 msgr2=0x7f355c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:00.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.496+0000 7f3560dff700 1 -- 172.21.15.8:0/1228313448 shutdown_connections 2023-11-27T15:24:00.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.496+0000 7f3560dff700 1 -- 172.21.15.8:0/1228313448 wait complete. 2023-11-27T15:24:00.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.497+0000 7f3560dff700 1 Processor -- start 2023-11-27T15:24:00.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.497+0000 7f3560dff700 1 -- start start 2023-11-27T15:24:00.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f3560dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f355c100290 0x7f355c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:00.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f3560dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f355c101480 0x7f355c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:00.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f3560dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f355c101e10 0x7f355c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:00.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f3560dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c19fd80 con 0x7f355c100290 2023-11-27T15:24:00.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f3560dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c19fec0 con 0x7f355c101480 2023-11-27T15:24:00.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f3560dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c1a0050 con 0x7f355c101e10 2023-11-27T15:24:00.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f355a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f355c100290 0x7f355c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:00.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f355a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f355c100290 0x7f355c198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53532/0 (socket says 172.21.15.8:53532) 2023-11-27T15:24:00.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f355a59c700 1 -- 172.21.15.8:0/2408019692 learned_addr learned my addr 172.21.15.8:0/2408019692 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:00.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f3559d9b700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f355c101480 0x7f355c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:00.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f355ad9d700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f355c101e10 0x7f355c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:00.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f355a59c700 1 -- 172.21.15.8:0/2408019692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f355c101e10 msgr2=0x7f355c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:00.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f355a59c700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f355c101e10 0x7f355c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f355a59c700 1 -- 172.21.15.8:0/2408019692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f355c101480 msgr2=0x7f355c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:00.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f355a59c700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f355c101480 0x7f355c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.498+0000 7f355a59c700 1 -- 172.21.15.8:0/2408019692 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c0095f0 con 0x7f355c100290 2023-11-27T15:24:00.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.499+0000 7f355a59c700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f355c100290 0x7f355c198200 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f354c004730 tx=0x7f354c004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:00.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.499+0000 7f354b7fe700 1 -- 172.21.15.8:0/2408019692 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c01d070 con 0x7f355c100290 2023-11-27T15:24:00.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.499+0000 7f354b7fe700 1 -- 172.21.15.8:0/2408019692 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f354c021410 con 0x7f355c100290 2023-11-27T15:24:00.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.499+0000 7f3560dff700 1 -- 172.21.15.8:0/2408019692 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c10bc40 con 0x7f355c100290 2023-11-27T15:24:00.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.499+0000 7f354b7fe700 1 -- 172.21.15.8:0/2408019692 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c017650 con 0x7f355c100290 2023-11-27T15:24:00.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.500+0000 7f3560dff700 1 -- 172.21.15.8:0/2408019692 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c10bab0 con 0x7f355c100290 2023-11-27T15:24:00.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.501+0000 7f3560dff700 1 -- 172.21.15.8:0/2408019692 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c1a05c0 con 0x7f355c100290 2023-11-27T15:24:00.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.503+0000 7f354b7fe700 1 -- 172.21.15.8:0/2408019692 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f354c021860 con 0x7f355c100290 2023-11-27T15:24:00.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.504+0000 7f354b7fe700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f35440a11e0 0x7f35440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:00.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.504+0000 7f354b7fe700 1 -- 172.21.15.8:0/2408019692 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f354c0cdf30 con 0x7f355c100290 2023-11-27T15:24:00.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.504+0000 7f3559d9b700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f35440a11e0 0x7f35440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:00.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.505+0000 7f3559d9b700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f35440a11e0 0x7f35440a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3550009c60 tx=0x7f3550009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:00.512 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.508+0000 7f354b7fe700 1 -- 172.21.15.8:0/2408019692 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f354c097360 con 0x7f355c100290 2023-11-27T15:24:00.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.753+0000 7f3560dff700 1 -- 172.21.15.8:0/2408019692 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f355c107e20 con 0x7f35440a11e0 2023-11-27T15:24:00.780 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.776+0000 7f354b7fe700 1 -- 172.21.15.8:0/2408019692 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f355c107e20 con 0x7f35440a11e0 2023-11-27T15:24:00.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 -- 172.21.15.8:0/2408019692 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f35440a11e0 msgr2=0x7f35440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:00.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f35440a11e0 0x7f35440a3690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3550009c60 tx=0x7f3550009260 comp rx=0 tx=0).stop 2023-11-27T15:24:00.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 -- 172.21.15.8:0/2408019692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f355c100290 msgr2=0x7f355c198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:00.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f355c100290 0x7f355c198200 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f354c004730 tx=0x7f354c004760 comp rx=0 tx=0).stop 2023-11-27T15:24:00.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 -- 172.21.15.8:0/2408019692 shutdown_connections 2023-11-27T15:24:00.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f35440a11e0 0x7f35440a3690 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f355c101e10 0x7f355c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f355c100290 0x7f355c198200 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 --2- 172.21.15.8:0/2408019692 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f355c101480 0x7f355c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:00.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 -- 172.21.15.8:0/2408019692 >> 172.21.15.8:0/2408019692 conn(0x7f355c0fb870 msgr2=0x7f355c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:00.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 -- 172.21.15.8:0/2408019692 shutdown_connections 2023-11-27T15:24:00.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:00.779+0000 7f3560dff700 1 -- 172.21.15.8:0/2408019692 wait complete. 2023-11-27T15:24:00.802 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-11-27T15:24:01.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.006+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/557508209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8101e10 msgr2=0x7fa5c81086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.006+0000 7fa5cd5ff700 1 --2- 172.21.15.8:0/557508209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8101e10 0x7fa5c81086d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000b0a0 tx=0x7fa5c000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.006+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/557508209 shutdown_connections 2023-11-27T15:24:01.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.006+0000 7fa5cd5ff700 1 --2- 172.21.15.8:0/557508209 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c8100290 0x7fa5c81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.006+0000 7fa5cd5ff700 1 --2- 172.21.15.8:0/557508209 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8101e10 0x7fa5c81086d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.006+0000 7fa5cd5ff700 1 --2- 172.21.15.8:0/557508209 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c8101480 0x7fa5c81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.006+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/557508209 >> 172.21.15.8:0/557508209 conn(0x7fa5c80fb870 msgr2=0x7fa5c80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:01.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.006+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/557508209 shutdown_connections 2023-11-27T15:24:01.010 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.006+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/557508209 wait complete. 2023-11-27T15:24:01.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.007+0000 7fa5cd5ff700 1 Processor -- start 2023-11-27T15:24:01.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.007+0000 7fa5cd5ff700 1 -- start start 2023-11-27T15:24:01.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.007+0000 7fa5cd5ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8100290 0x7fa5c8196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.007+0000 7fa5cd5ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c8101480 0x7fa5c8196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.007+0000 7fa5cd5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c8101e10 0x7fa5c819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.007+0000 7fa5cd5ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c819dc80 con 0x7fa5c8100290 2023-11-27T15:24:01.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.007+0000 7fa5cd5ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c819ddc0 con 0x7fa5c8101480 2023-11-27T15:24:01.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.007+0000 7fa5cd5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c819df50 con 0x7fa5c8101e10 2023-11-27T15:24:01.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5c6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8100290 0x7fa5c8196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5c6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8100290 0x7fa5c8196100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53546/0 (socket says 172.21.15.8:53546) 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5c6ffd700 1 -- 172.21.15.8:0/3336633590 learned_addr learned my addr 172.21.15.8:0/3336633590 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5bffff700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c8101480 0x7fa5c8196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5c77fe700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c8101e10 0x7fa5c819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5c6ffd700 1 -- 172.21.15.8:0/3336633590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c8101e10 msgr2=0x7fa5c819d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5c6ffd700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c8101e10 0x7fa5c819d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5c6ffd700 1 -- 172.21.15.8:0/3336633590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c8101480 msgr2=0x7fa5c8196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5c6ffd700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c8101480 0x7fa5c8196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5c6ffd700 1 -- 172.21.15.8:0/3336633590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c0009d00 con 0x7fa5c8100290 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.008+0000 7fa5c6ffd700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8100290 0x7fa5c8196100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800e970 tx=0x7fa5b800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.009+0000 7fa5c4ff9700 1 -- 172.21.15.8:0/3336633590 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8009790 con 0x7fa5c8100290 2023-11-27T15:24:01.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.009+0000 7fa5c4ff9700 1 -- 172.21.15.8:0/3336633590 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5b800c7e0 con 0x7fa5c8100290 2023-11-27T15:24:01.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.009+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/3336633590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c810bc70 con 0x7fa5c8100290 2023-11-27T15:24:01.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.009+0000 7fa5c4ff9700 1 -- 172.21.15.8:0/3336633590 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8003c70 con 0x7fa5c8100290 2023-11-27T15:24:01.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.009+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/3336633590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c810bae0 con 0x7fa5c8100290 2023-11-27T15:24:01.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.010+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/3336633590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c8107840 con 0x7fa5c8100290 2023-11-27T15:24:01.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.012+0000 7fa5c4ff9700 1 -- 172.21.15.8:0/3336633590 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fa5b800cc50 con 0x7fa5c8100290 2023-11-27T15:24:01.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.013+0000 7fa5c4ff9700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5b40a1260 0x7fa5b40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.013+0000 7fa5c4ff9700 1 -- 172.21.15.8:0/3336633590 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa5b8014070 con 0x7fa5c8100290 2023-11-27T15:24:01.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.014+0000 7fa5bffff700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5b40a1260 0x7fa5b40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.014+0000 7fa5bffff700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5b40a1260 0x7fa5b40a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009c60 tx=0x7fa5b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:01.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.017+0000 7fa5c4ff9700 1 -- 172.21.15.8:0/3336633590 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa5b80961d0 con 0x7fa5c8100290 2023-11-27T15:24:01.267 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.264+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/3336633590 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5c8061be0 con 0x7fa5b40a1260 2023-11-27T15:24:01.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.268+0000 7fa5c4ff9700 1 -- 172.21.15.8:0/3336633590 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fa5c8061be0 con 0x7fa5b40a1260 2023-11-27T15:24:01.271 INFO:teuthology.orchestra.run.smithi008.stdout:/volumes/_nogroup/sv_1/49db8ba1-39b2-4d18-8e42-2d890bb92a8e 2023-11-27T15:24:01.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/3336633590 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5b40a1260 msgr2=0x7fa5b40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5b40a1260 0x7fa5b40a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009c60 tx=0x7fa5b0009260 comp rx=0 tx=0).stop 2023-11-27T15:24:01.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/3336633590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8100290 msgr2=0x7fa5c8196100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8100290 0x7fa5c8196100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800e970 tx=0x7fa5b800ec80 comp rx=0 tx=0).stop 2023-11-27T15:24:01.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/3336633590 shutdown_connections 2023-11-27T15:24:01.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5b40a1260 0x7fa5b40a3710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa5c8101e10 0x7fa5c819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8100290 0x7fa5c8196100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 --2- 172.21.15.8:0/3336633590 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa5c8101480 0x7fa5c8196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/3336633590 >> 172.21.15.8:0/3336633590 conn(0x7fa5c80fb870 msgr2=0x7fa5c8104980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:01.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/3336633590 shutdown_connections 2023-11-27T15:24:01.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.270+0000 7fa5cd5ff700 1 -- 172.21.15.8:0/3336633590 wait complete. 2023-11-27T15:24:01.292 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:24:01.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.493+0000 7f6beb3ff700 1 -- 172.21.15.8:0/3247139252 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6be4100d40 msgr2=0x7f6be4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.493+0000 7f6beb3ff700 1 --2- 172.21.15.8:0/3247139252 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6be4100d40 0x7f6be4103120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6be0009990 tx=0x7f6be0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.493+0000 7f6beb3ff700 1 -- 172.21.15.8:0/3247139252 shutdown_connections 2023-11-27T15:24:01.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.493+0000 7f6beb3ff700 1 --2- 172.21.15.8:0/3247139252 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be4105f80 0x7f6be41083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.493+0000 7f6beb3ff700 1 --2- 172.21.15.8:0/3247139252 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6be4100d40 0x7f6be4103120 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.493+0000 7f6beb3ff700 1 --2- 172.21.15.8:0/3247139252 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6be4103660 0x7f6be4105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.493+0000 7f6beb3ff700 1 -- 172.21.15.8:0/3247139252 >> 172.21.15.8:0/3247139252 conn(0x7f6be40fa7c0 msgr2=0x7f6be40fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:01.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.493+0000 7f6beb3ff700 1 -- 172.21.15.8:0/3247139252 shutdown_connections 2023-11-27T15:24:01.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.493+0000 7f6beb3ff700 1 -- 172.21.15.8:0/3247139252 wait complete. 2023-11-27T15:24:01.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.494+0000 7f6beb3ff700 1 Processor -- start 2023-11-27T15:24:01.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.495+0000 7f6beb3ff700 1 -- start start 2023-11-27T15:24:01.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.495+0000 7f6beb3ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6be4100d40 0x7f6be4196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.495+0000 7f6beb3ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6be4103660 0x7f6be4196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.495+0000 7f6beb3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be4105f80 0x7f6be419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.495+0000 7f6beb3ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be419dc40 con 0x7f6be4100d40 2023-11-27T15:24:01.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.495+0000 7f6beb3ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be419dd80 con 0x7f6be4103660 2023-11-27T15:24:01.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.495+0000 7f6beb3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be419df10 con 0x7f6be4105f80 2023-11-27T15:24:01.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.495+0000 7f6be919b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6be4100d40 0x7f6be4196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.495+0000 7f6be919b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6be4100d40 0x7f6be4196110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53560/0 (socket says 172.21.15.8:53560) 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.495+0000 7f6be919b700 1 -- 172.21.15.8:0/876045884 learned_addr learned my addr 172.21.15.8:0/876045884 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.496+0000 7f6be899a700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6be4103660 0x7f6be4196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.496+0000 7f6be999c700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be4105f80 0x7f6be419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.496+0000 7f6be919b700 1 -- 172.21.15.8:0/876045884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be4105f80 msgr2=0x7f6be419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.496+0000 7f6be919b700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be4105f80 0x7f6be419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.496+0000 7f6be919b700 1 -- 172.21.15.8:0/876045884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6be4103660 msgr2=0x7f6be4196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.496+0000 7f6be919b700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6be4103660 0x7f6be4196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.496+0000 7f6be919b700 1 -- 172.21.15.8:0/876045884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd4009520 con 0x7f6be4100d40 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.496+0000 7f6be899a700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6be4103660 0x7f6be4196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.496+0000 7f6be919b700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6be4100d40 0x7f6be4196110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6be000b680 tx=0x7f6be0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:01.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.496+0000 7f6bda7fc700 1 -- 172.21.15.8:0/876045884 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be001d070 con 0x7f6be4100d40 2023-11-27T15:24:01.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.497+0000 7f6bda7fc700 1 -- 172.21.15.8:0/876045884 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6be0021410 con 0x7f6be4100d40 2023-11-27T15:24:01.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.497+0000 7f6bda7fc700 1 -- 172.21.15.8:0/876045884 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be0017650 con 0x7f6be4100d40 2023-11-27T15:24:01.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.497+0000 7f6beb3ff700 1 -- 172.21.15.8:0/876045884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be00095f0 con 0x7f6be4100d40 2023-11-27T15:24:01.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.497+0000 7f6beb3ff700 1 -- 172.21.15.8:0/876045884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be410bca0 con 0x7f6be4100d40 2023-11-27T15:24:01.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.498+0000 7f6beb3ff700 1 -- 172.21.15.8:0/876045884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be40fc330 con 0x7f6be4100d40 2023-11-27T15:24:01.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.499+0000 7f6bda7fc700 1 -- 172.21.15.8:0/876045884 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f6be000fb10 con 0x7f6be4100d40 2023-11-27T15:24:01.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.501+0000 7f6bda7fc700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6bd00a1260 0x7f6bd00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.504 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.501+0000 7f6bda7fc700 1 -- 172.21.15.8:0/876045884 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6be00ce190 con 0x7f6be4100d40 2023-11-27T15:24:01.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.501+0000 7f6be899a700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6bd00a1260 0x7f6bd00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.502+0000 7f6be899a700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6bd00a1260 0x7f6bd00a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009e40 tx=0x7f6bd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:01.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.505+0000 7f6bda7fc700 1 -- 172.21.15.8:0/876045884 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6be00975c0 con 0x7f6be4100d40 2023-11-27T15:24:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:01 smithi008 ceph-mon[128788]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:01 smithi008 ceph-mon[128788]: from='client.24856 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:01 smithi008 ceph-mon[128788]: pgmap v299: 129 pgs: 129 active+clean; 598 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2023-11-27T15:24:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:01 smithi008 ceph-mon[128788]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-27T15:24:01.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.761+0000 7f6beb3ff700 1 -- 172.21.15.8:0/876045884 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6be402cfe0 con 0x7f6be4100d40 2023-11-27T15:24:01.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.762+0000 7f6bda7fc700 1 -- 172.21.15.8:0/876045884 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10768 (secure 0 0 0) 0x7f6be009abe0 con 0x7f6be4100d40 2023-11-27T15:24:01.766 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:01.766 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:23:27.167939+0000","modified":"2023-11-27T15:23:59.241472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24595,"mds_1":24593,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24593":{"gid":24593,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24595":{"gid":24595,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:24:01.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 -- 172.21.15.8:0/876045884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6bd00a1260 msgr2=0x7f6bd00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6bd00a1260 0x7f6bd00a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009e40 tx=0x7f6bd4009260 comp rx=0 tx=0).stop 2023-11-27T15:24:01.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 -- 172.21.15.8:0/876045884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6be4100d40 msgr2=0x7f6be4196110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6be4100d40 0x7f6be4196110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6be000b680 tx=0x7f6be0005740 comp rx=0 tx=0).stop 2023-11-27T15:24:01.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 -- 172.21.15.8:0/876045884 shutdown_connections 2023-11-27T15:24:01.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6bd00a1260 0x7f6bd00a3710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6be4105f80 0x7f6be419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6be4100d40 0x7f6be4196110 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 --2- 172.21.15.8:0/876045884 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6be4103660 0x7f6be4196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 -- 172.21.15.8:0/876045884 >> 172.21.15.8:0/876045884 conn(0x7f6be40fa7c0 msgr2=0x7f6be41019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:01.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 -- 172.21.15.8:0/876045884 shutdown_connections 2023-11-27T15:24:01.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.764+0000 7f6beb3ff700 1 -- 172.21.15.8:0/876045884 wait complete. 2023-11-27T15:24:01.771 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2023-11-27T15:24:01.787 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:24:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:01 smithi060 ceph-mon[131518]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:01 smithi060 ceph-mon[131518]: from='client.24856 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:01.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:01 smithi060 ceph-mon[131518]: pgmap v299: 129 pgs: 129 active+clean; 598 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2023-11-27T15:24:01.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:01 smithi060 ceph-mon[131518]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-27T15:24:01.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:01 smithi029 ceph-mon[131166]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:01.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:01 smithi029 ceph-mon[131166]: from='client.24856 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:01.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:01 smithi029 ceph-mon[131166]: pgmap v299: 129 pgs: 129 active+clean; 598 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2023-11-27T15:24:01.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:01 smithi029 ceph-mon[131166]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-27T15:24:01.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.986+0000 7fe2f19ff700 1 -- 172.21.15.8:0/256266040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec1002d0 msgr2=0x7fe2ec1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.986+0000 7fe2f19ff700 1 --2- 172.21.15.8:0/256266040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec1002d0 0x7fe2ec1006e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0098f0 tx=0x7fe2dc009c00 comp rx=0 tx=0).stop 2023-11-27T15:24:01.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.986+0000 7fe2f19ff700 1 -- 172.21.15.8:0/256266040 shutdown_connections 2023-11-27T15:24:01.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.986+0000 7fe2f19ff700 1 --2- 172.21.15.8:0/256266040 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec1002d0 0x7fe2ec1006e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.986+0000 7fe2f19ff700 1 --2- 172.21.15.8:0/256266040 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2ec101e50 0x7fe2ec108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.986+0000 7fe2f19ff700 1 --2- 172.21.15.8:0/256266040 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2ec1014c0 0x7fe2ec101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.986+0000 7fe2f19ff700 1 -- 172.21.15.8:0/256266040 >> 172.21.15.8:0/256266040 conn(0x7fe2ec0fb8b0 msgr2=0x7fe2ec0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:01.991 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.986+0000 7fe2f19ff700 1 -- 172.21.15.8:0/256266040 shutdown_connections 2023-11-27T15:24:01.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.986+0000 7fe2f19ff700 1 -- 172.21.15.8:0/256266040 wait complete. 2023-11-27T15:24:01.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.987+0000 7fe2f19ff700 1 Processor -- start 2023-11-27T15:24:01.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2f19ff700 1 -- start start 2023-11-27T15:24:01.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2f19ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2ec1002d0 0x7fe2ec0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2f19ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec1014c0 0x7fe2ec072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2f19ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2ec101e50 0x7fe2ec073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:01.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2f19ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec073bc0 con 0x7fe2ec1002d0 2023-11-27T15:24:01.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2f19ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec073d30 con 0x7fe2ec101e50 2023-11-27T15:24:01.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2f19ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec1a0100 con 0x7fe2ec1014c0 2023-11-27T15:24:01.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2eaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2ec1002d0 0x7fe2ec0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2eaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2ec1002d0 0x7fe2ec0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53574/0 (socket says 172.21.15.8:53574) 2023-11-27T15:24:01.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2eaffd700 1 -- 172.21.15.8:0/2678733243 learned_addr learned my addr 172.21.15.8:0/2678733243 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:01.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2eb7fe700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2ec101e50 0x7fe2ec073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.988+0000 7fe2e3fff700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec1014c0 0x7fe2ec072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:01.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2eaffd700 1 -- 172.21.15.8:0/2678733243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec1014c0 msgr2=0x7fe2ec072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2eaffd700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec1014c0 0x7fe2ec072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2eaffd700 1 -- 172.21.15.8:0/2678733243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2ec101e50 msgr2=0x7fe2ec073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:01.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2eaffd700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2ec101e50 0x7fe2ec073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:01.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2eaffd700 1 -- 172.21.15.8:0/2678733243 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc009520 con 0x7fe2ec1002d0 2023-11-27T15:24:01.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2e3fff700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec1014c0 0x7fe2ec072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:01.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2eaffd700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2ec1002d0 0x7fe2ec0748a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009fd0 tx=0x7fe2dc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:01.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2f09fd700 1 -- 172.21.15.8:0/2678733243 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc01c070 con 0x7fe2ec1002d0 2023-11-27T15:24:01.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2f09fd700 1 -- 172.21.15.8:0/2678733243 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2dc00f7d0 con 0x7fe2ec1002d0 2023-11-27T15:24:01.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2f09fd700 1 -- 172.21.15.8:0/2678733243 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc003b00 con 0x7fe2ec1002d0 2023-11-27T15:24:01.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2f19ff700 1 -- 172.21.15.8:0/2678733243 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec10bc20 con 0x7fe2ec1002d0 2023-11-27T15:24:01.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.989+0000 7fe2f19ff700 1 -- 172.21.15.8:0/2678733243 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec10ba90 con 0x7fe2ec1002d0 2023-11-27T15:24:01.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.990+0000 7fe2f19ff700 1 -- 172.21.15.8:0/2678733243 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ec067af0 con 0x7fe2ec1002d0 2023-11-27T15:24:01.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.992+0000 7fe2f09fd700 1 -- 172.21.15.8:0/2678733243 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe2dc020030 con 0x7fe2ec1002d0 2023-11-27T15:24:02.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.996+0000 7fe2f09fd700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2d80a11e0 0x7fe2d80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.996+0000 7fe2f09fd700 1 -- 172.21.15.8:0/2678733243 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe2dc0cdd10 con 0x7fe2ec1002d0 2023-11-27T15:24:02.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.996+0000 7fe2e3fff700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2d80a11e0 0x7fe2d80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.997+0000 7fe2e3fff700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2d80a11e0 0x7fe2d80a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005fd0 tx=0x7fe2d4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.003 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:01.999+0000 7fe2f09fd700 1 -- 172.21.15.8:0/2678733243 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe2dc0d3020 con 0x7fe2ec1002d0 2023-11-27T15:24:02.213 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.209+0000 7fe2f19ff700 1 -- 172.21.15.8:0/2678733243 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2ec107220 con 0x7fe2ec1002d0 2023-11-27T15:24:02.214 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.211+0000 7fe2f09fd700 1 -- 172.21.15.8:0/2678733243 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23817 (secure 0 0 0) 0x7fe2dc097140 con 0x7fe2ec1002d0 2023-11-27T15:24:02.215 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:02.215 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":88,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:23:32.146115+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:24:02.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 -- 172.21.15.8:0/2678733243 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2d80a11e0 msgr2=0x7fe2d80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2d80a11e0 0x7fe2d80a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005fd0 tx=0x7fe2d4009310 comp rx=0 tx=0).stop 2023-11-27T15:24:02.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 -- 172.21.15.8:0/2678733243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2ec1002d0 msgr2=0x7fe2ec0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2ec1002d0 0x7fe2ec0748a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009fd0 tx=0x7fe2dc00b790 comp rx=0 tx=0).stop 2023-11-27T15:24:02.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 -- 172.21.15.8:0/2678733243 shutdown_connections 2023-11-27T15:24:02.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2ec101e50 0x7fe2ec073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2ec1002d0 0x7fe2ec0748a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2d80a11e0 0x7fe2d80a3690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 --2- 172.21.15.8:0/2678733243 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2ec1014c0 0x7fe2ec072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 -- 172.21.15.8:0/2678733243 >> 172.21.15.8:0/2678733243 conn(0x7fe2ec0fb8b0 msgr2=0x7fe2ec0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.213+0000 7fe2f19ff700 1 -- 172.21.15.8:0/2678733243 shutdown_connections 2023-11-27T15:24:02.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.214+0000 7fe2f19ff700 1 -- 172.21.15.8:0/2678733243 wait complete. 2023-11-27T15:24:02.236 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-27T15:24:02.236 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:24:02.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.438+0000 7fe8d83ff700 1 -- 172.21.15.8:0/851537576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8d0101e50 msgr2=0x7fe8d0108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.438+0000 7fe8d83ff700 1 --2- 172.21.15.8:0/851537576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8d0101e50 0x7fe8d0108710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009f90 tx=0x7fe8c800b330 comp rx=0 tx=0).stop 2023-11-27T15:24:02.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.438+0000 7fe8d83ff700 1 -- 172.21.15.8:0/851537576 shutdown_connections 2023-11-27T15:24:02.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.438+0000 7fe8d83ff700 1 --2- 172.21.15.8:0/851537576 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d01002d0 0x7fe8d01006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.438+0000 7fe8d83ff700 1 --2- 172.21.15.8:0/851537576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8d0101e50 0x7fe8d0108710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.443 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.438+0000 7fe8d83ff700 1 --2- 172.21.15.8:0/851537576 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8d01014c0 0x7fe8d0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.438+0000 7fe8d83ff700 1 -- 172.21.15.8:0/851537576 >> 172.21.15.8:0/851537576 conn(0x7fe8d00fb8b0 msgr2=0x7fe8d00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.439+0000 7fe8d83ff700 1 -- 172.21.15.8:0/851537576 shutdown_connections 2023-11-27T15:24:02.444 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.439+0000 7fe8d83ff700 1 -- 172.21.15.8:0/851537576 wait complete. 2023-11-27T15:24:02.446 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.440+0000 7fe8d83ff700 1 Processor -- start 2023-11-27T15:24:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.440+0000 7fe8d83ff700 1 -- start start 2023-11-27T15:24:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.440+0000 7fe8d83ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8d01002d0 0x7fe8d00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.440+0000 7fe8d83ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8d01014c0 0x7fe8d0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.440+0000 7fe8d83ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0101e50 0x7fe8d0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.440+0000 7fe8d83ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d0073bc0 con 0x7fe8d01014c0 2023-11-27T15:24:02.447 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.440+0000 7fe8d83ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d0073d30 con 0x7fe8d01002d0 2023-11-27T15:24:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.440+0000 7fe8d83ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d01a01b0 con 0x7fe8d0101e50 2023-11-27T15:24:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d599a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8d01014c0 0x7fe8d0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d619b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8d01002d0 0x7fe8d00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d599a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8d01014c0 0x7fe8d0072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53592/0 (socket says 172.21.15.8:53592) 2023-11-27T15:24:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d599a700 1 -- 172.21.15.8:0/3697047008 learned_addr learned my addr 172.21.15.8:0/3697047008 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d699c700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0101e50 0x7fe8d0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.448 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d619b700 1 -- 172.21.15.8:0/3697047008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0101e50 msgr2=0x7fe8d0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d619b700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0101e50 0x7fe8d0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d619b700 1 -- 172.21.15.8:0/3697047008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8d01014c0 msgr2=0x7fe8d0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d619b700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8d01014c0 0x7fe8d0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d619b700 1 -- 172.21.15.8:0/3697047008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c00095f0 con 0x7fe8d01002d0 2023-11-27T15:24:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d699c700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0101e50 0x7fe8d0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d599a700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8d01014c0 0x7fe8d0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8d619b700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8d01002d0 0x7fe8d00748a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00e9c0 tx=0x7fe8cc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.441+0000 7fe8c77fe700 1 -- 172.21.15.8:0/3697047008 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc004220 con 0x7fe8d01002d0 2023-11-27T15:24:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.442+0000 7fe8c77fe700 1 -- 172.21.15.8:0/3697047008 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8cc004380 con 0x7fe8d01002d0 2023-11-27T15:24:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.442+0000 7fe8d83ff700 1 -- 172.21.15.8:0/3697047008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c8009c40 con 0x7fe8d01002d0 2023-11-27T15:24:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.442+0000 7fe8c77fe700 1 -- 172.21.15.8:0/3697047008 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc003be0 con 0x7fe8d01002d0 2023-11-27T15:24:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.442+0000 7fe8d83ff700 1 -- 172.21.15.8:0/3697047008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d010bca0 con 0x7fe8d01002d0 2023-11-27T15:24:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.443+0000 7fe8d83ff700 1 -- 172.21.15.8:0/3697047008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d01077d0 con 0x7fe8d01002d0 2023-11-27T15:24:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.444+0000 7fe8c77fe700 1 -- 172.21.15.8:0/3697047008 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe8cc003d40 con 0x7fe8d01002d0 2023-11-27T15:24:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.446+0000 7fe8c77fe700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8bc0a1260 0x7fe8bc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.446+0000 7fe8c77fe700 1 -- 172.21.15.8:0/3697047008 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe8cc014070 con 0x7fe8d01002d0 2023-11-27T15:24:02.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.446+0000 7fe8d599a700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8bc0a1260 0x7fe8bc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.452 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.447+0000 7fe8d599a700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8bc0a1260 0x7fe8bc0a3710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0005d10 tx=0x7fe8c0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.450+0000 7fe8c77fe700 1 -- 172.21.15.8:0/3697047008 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe8cc0962a0 con 0x7fe8d01002d0 2023-11-27T15:24:02.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:02 smithi008 ceph-mon[128788]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:02.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:02 smithi008 ceph-mon[128788]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:02.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:02 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/876045884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:02.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:02 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2678733243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:02.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.656+0000 7fe8d83ff700 1 -- 172.21.15.8:0/3697047008 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8d0060f30 con 0x7fe8bc0a1260 2023-11-27T15:24:02.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.672+0000 7fe8c77fe700 1 -- 172.21.15.8:0/3697047008 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422357 (secure 0 0 0) 0x7fe8d0060f30 con 0x7fe8bc0a1260 2023-11-27T15:24:02.684 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 -- 172.21.15.8:0/3697047008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8bc0a1260 msgr2=0x7fe8bc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8bc0a1260 0x7fe8bc0a3710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0005d10 tx=0x7fe8c0009310 comp rx=0 tx=0).stop 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 -- 172.21.15.8:0/3697047008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8d01002d0 msgr2=0x7fe8d00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8d01002d0 0x7fe8d00748a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00e9c0 tx=0x7fe8cc00ecd0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 -- 172.21.15.8:0/3697047008 shutdown_connections 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8bc0a1260 0x7fe8bc0a3710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8d0101e50 0x7fe8d0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8d01002d0 0x7fe8d00748a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 --2- 172.21.15.8:0/3697047008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8d01014c0 0x7fe8d0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 -- 172.21.15.8:0/3697047008 >> 172.21.15.8:0/3697047008 conn(0x7fe8d00fb8b0 msgr2=0x7fe8d0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 -- 172.21.15.8:0/3697047008 shutdown_connections 2023-11-27T15:24:02.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.678+0000 7fe8d83ff700 1 -- 172.21.15.8:0/3697047008 wait complete. 2023-11-27T15:24:02.691 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-11-27T15:24:02.701 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-11-27T15:24:02.334942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616083,"num_objects":63,"num_object_clones":0,"num_object_copies":191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":158,"num_read_kb":137,"num_write":320,"num_write_kb":3005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":259,"ondisk_log_size":259,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":81880,"kb_used_data":13304,"kb_used_omap":0,"kb_used_meta":68480,"kb_avail":1124859944,"statfs":{"total":1151940427776,"available":1151856582656,"internally_reserved":0,"allocated":13623296,"data_stored":7467553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70123520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21123,"num_objects":39,"num_object_clones":0,"num_object_copies":119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":90,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003826"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.152009+0000","last_change":"2023-11-27T15:23:27.211699+0000","last_active":"2023-11-27T15:23:32.152009+0000","last_peered":"2023-11-27T15:23:32.152009+0000","last_clean":"2023-11-27T15:23:32.152009+0000","last_became_active":"2023-11-27T15:23:27.211372+0000","last_became_peered":"2023-11-27T15:23:27.211372+0000","last_unstale":"2023-11-27T15:23:32.152009+0000","last_undegraded":"2023-11-27T15:23:32.152009+0000","last_fullsized":"2023-11-27T15:23:32.152009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:36:00.909796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807370+0000","last_change":"2023-11-27T15:23:27.210567+0000","last_active":"2023-11-27T15:23:32.807370+0000","last_peered":"2023-11-27T15:23:32.807370+0000","last_clean":"2023-11-27T15:23:32.807370+0000","last_became_active":"2023-11-27T15:23:27.208340+0000","last_became_peered":"2023-11-27T15:23:27.208340+0000","last_unstale":"2023-11-27T15:23:32.807370+0000","last_undegraded":"2023-11-27T15:23:32.807370+0000","last_fullsized":"2023-11-27T15:23:32.807370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:10:44.168450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806815+0000","last_change":"2023-11-27T15:23:27.196040+0000","last_active":"2023-11-27T15:23:32.806815+0000","last_peered":"2023-11-27T15:23:32.806815+0000","last_clean":"2023-11-27T15:23:32.806815+0000","last_became_active":"2023-11-27T15:23:27.195195+0000","last_became_peered":"2023-11-27T15:23:27.195195+0000","last_unstale":"2023-11-27T15:23:32.806815+0000","last_undegraded":"2023-11-27T15:23:32.806815+0000","last_fullsized":"2023-11-27T15:23:32.806815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:57:28.442265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240980+0000","last_change":"2023-11-27T15:23:27.190096+0000","last_active":"2023-11-27T15:23:32.240980+0000","last_peered":"2023-11-27T15:23:32.240980+0000","last_clean":"2023-11-27T15:23:32.240980+0000","last_became_active":"2023-11-27T15:23:27.189768+0000","last_became_peered":"2023-11-27T15:23:27.189768+0000","last_unstale":"2023-11-27T15:23:32.240980+0000","last_undegraded":"2023-11-27T15:23:32.240980+0000","last_fullsized":"2023-11-27T15:23:32.240980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:16:12.429083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805525+0000","last_change":"2023-11-27T15:23:27.200974+0000","last_active":"2023-11-27T15:23:32.805525+0000","last_peered":"2023-11-27T15:23:32.805525+0000","last_clean":"2023-11-27T15:23:32.805525+0000","last_became_active":"2023-11-27T15:23:27.200647+0000","last_became_peered":"2023-11-27T15:23:27.200647+0000","last_unstale":"2023-11-27T15:23:32.805525+0000","last_undegraded":"2023-11-27T15:23:32.805525+0000","last_fullsized":"2023-11-27T15:23:32.805525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:53:56.304270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151519+0000","last_change":"2023-11-27T15:23:27.188417+0000","last_active":"2023-11-27T15:23:32.151519+0000","last_peered":"2023-11-27T15:23:32.151519+0000","last_clean":"2023-11-27T15:23:32.151519+0000","last_became_active":"2023-11-27T15:23:27.188079+0000","last_became_peered":"2023-11-27T15:23:27.188079+0000","last_unstale":"2023-11-27T15:23:32.151519+0000","last_undegraded":"2023-11-27T15:23:32.151519+0000","last_fullsized":"2023-11-27T15:23:32.151519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:53:23.107972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387140+0000","last_change":"2023-11-27T15:23:27.192410+0000","last_active":"2023-11-27T15:23:33.387140+0000","last_peered":"2023-11-27T15:23:33.387140+0000","last_clean":"2023-11-27T15:23:33.387140+0000","last_became_active":"2023-11-27T15:23:27.192026+0000","last_became_peered":"2023-11-27T15:23:27.192026+0000","last_unstale":"2023-11-27T15:23:33.387140+0000","last_undegraded":"2023-11-27T15:23:33.387140+0000","last_fullsized":"2023-11-27T15:23:33.387140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:35:50.652363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445258+0000","last_change":"2023-11-27T15:23:27.205778+0000","last_active":"2023-11-27T15:23:32.445258+0000","last_peered":"2023-11-27T15:23:32.445258+0000","last_clean":"2023-11-27T15:23:32.445258+0000","last_became_active":"2023-11-27T15:23:27.205467+0000","last_became_peered":"2023-11-27T15:23:27.205467+0000","last_unstale":"2023-11-27T15:23:32.445258+0000","last_undegraded":"2023-11-27T15:23:32.445258+0000","last_fullsized":"2023-11-27T15:23:32.445258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:35:33.177488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805937+0000","last_change":"2023-11-27T15:23:27.187920+0000","last_active":"2023-11-27T15:23:32.805937+0000","last_peered":"2023-11-27T15:23:32.805937+0000","last_clean":"2023-11-27T15:23:32.805937+0000","last_became_active":"2023-11-27T15:23:27.186989+0000","last_became_peered":"2023-11-27T15:23:27.186989+0000","last_unstale":"2023-11-27T15:23:32.805937+0000","last_undegraded":"2023-11-27T15:23:32.805937+0000","last_fullsized":"2023-11-27T15:23:32.805937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:17:57.515405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.152137+0000","last_change":"2023-11-27T15:23:27.181330+0000","last_active":"2023-11-27T15:23:32.152137+0000","last_peered":"2023-11-27T15:23:32.152137+0000","last_clean":"2023-11-27T15:23:32.152137+0000","last_became_active":"2023-11-27T15:23:27.180986+0000","last_became_peered":"2023-11-27T15:23:27.180986+0000","last_unstale":"2023-11-27T15:23:32.152137+0000","last_undegraded":"2023-11-27T15:23:32.152137+0000","last_fullsized":"2023-11-27T15:23:32.152137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:32:05.556276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387781+0000","last_change":"2023-11-27T15:23:27.190673+0000","last_active":"2023-11-27T15:23:33.387781+0000","last_peered":"2023-11-27T15:23:33.387781+0000","last_clean":"2023-11-27T15:23:33.387781+0000","last_became_active":"2023-11-27T15:23:27.186256+0000","last_became_peered":"2023-11-27T15:23:27.186256+0000","last_unstale":"2023-11-27T15:23:33.387781+0000","last_undegraded":"2023-11-27T15:23:33.387781+0000","last_fullsized":"2023-11-27T15:23:33.387781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:59:37.848691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.241078+0000","last_change":"2023-11-27T15:23:27.185077+0000","last_active":"2023-11-27T15:23:32.241078+0000","last_peered":"2023-11-27T15:23:32.241078+0000","last_clean":"2023-11-27T15:23:32.241078+0000","last_became_active":"2023-11-27T15:23:27.184227+0000","last_became_peered":"2023-11-27T15:23:27.184227+0000","last_unstale":"2023-11-27T15:23:32.241078+0000","last_undegraded":"2023-11-27T15:23:32.241078+0000","last_fullsized":"2023-11-27T15:23:32.241078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:51.267197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806664+0000","last_change":"2023-11-27T15:23:27.190146+0000","last_active":"2023-11-27T15:23:32.806664+0000","last_peered":"2023-11-27T15:23:32.806664+0000","last_clean":"2023-11-27T15:23:32.806664+0000","last_became_active":"2023-11-27T15:23:27.189133+0000","last_became_peered":"2023-11-27T15:23:27.189133+0000","last_unstale":"2023-11-27T15:23:32.806664+0000","last_undegraded":"2023-11-27T15:23:32.806664+0000","last_fullsized":"2023-11-27T15:23:32.806664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:08:16.740298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806059+0000","last_change":"2023-11-27T15:23:27.207461+0000","last_active":"2023-11-27T15:23:32.806059+0000","last_peered":"2023-11-27T15:23:32.806059+0000","last_clean":"2023-11-27T15:23:32.806059+0000","last_became_active":"2023-11-27T15:23:27.206837+0000","last_became_peered":"2023-11-27T15:23:27.206837+0000","last_unstale":"2023-11-27T15:23:32.806059+0000","last_undegraded":"2023-11-27T15:23:32.806059+0000","last_fullsized":"2023-11-27T15:23:32.806059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:58:03.194278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805991+0000","last_change":"2023-11-27T15:23:27.211206+0000","last_active":"2023-11-27T15:23:32.805991+0000","last_peered":"2023-11-27T15:23:32.805991+0000","last_clean":"2023-11-27T15:23:32.805991+0000","last_became_active":"2023-11-27T15:23:27.209933+0000","last_became_peered":"2023-11-27T15:23:27.209933+0000","last_unstale":"2023-11-27T15:23:32.805991+0000","last_undegraded":"2023-11-27T15:23:32.805991+0000","last_fullsized":"2023-11-27T15:23:32.805991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:23:51.245058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805184+0000","last_change":"2023-11-27T15:23:27.188863+0000","last_active":"2023-11-27T15:23:32.805184+0000","last_peered":"2023-11-27T15:23:32.805184+0000","last_clean":"2023-11-27T15:23:32.805184+0000","last_became_active":"2023-11-27T15:23:27.188534+0000","last_became_peered":"2023-11-27T15:23:27.188534+0000","last_unstale":"2023-11-27T15:23:32.805184+0000","last_undegraded":"2023-11-27T15:23:32.805184+0000","last_fullsized":"2023-11-27T15:23:32.805184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:27:47.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151901+0000","last_change":"2023-11-27T15:23:27.211786+0000","last_active":"2023-11-27T15:23:32.151901+0000","last_peered":"2023-11-27T15:23:32.151901+0000","last_clean":"2023-11-27T15:23:32.151901+0000","last_became_active":"2023-11-27T15:23:27.211475+0000","last_became_peered":"2023-11-27T15:23:27.211475+0000","last_unstale":"2023-11-27T15:23:32.151901+0000","last_undegraded":"2023-11-27T15:23:32.151901+0000","last_fullsized":"2023-11-27T15:23:32.151901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:34:31.524704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150698+0000","last_change":"2023-11-27T15:23:27.183246+0000","last_active":"2023-11-27T15:23:32.150698+0000","last_peered":"2023-11-27T15:23:32.150698+0000","last_clean":"2023-11-27T15:23:32.150698+0000","last_became_active":"2023-11-27T15:23:27.182454+0000","last_became_peered":"2023-11-27T15:23:27.182454+0000","last_unstale":"2023-11-27T15:23:32.150698+0000","last_undegraded":"2023-11-27T15:23:32.150698+0000","last_fullsized":"2023-11-27T15:23:32.150698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:18:44.304123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806853+0000","last_change":"2023-11-27T15:23:27.196461+0000","last_active":"2023-11-27T15:23:32.806853+0000","last_peered":"2023-11-27T15:23:32.806853+0000","last_clean":"2023-11-27T15:23:32.806853+0000","last_became_active":"2023-11-27T15:23:27.196122+0000","last_became_peered":"2023-11-27T15:23:27.196122+0000","last_unstale":"2023-11-27T15:23:32.806853+0000","last_undegraded":"2023-11-27T15:23:32.806853+0000","last_fullsized":"2023-11-27T15:23:32.806853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:25:09.594663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806301+0000","last_change":"2023-11-27T15:23:31.649582+0000","last_active":"2023-11-27T15:23:32.806301+0000","last_peered":"2023-11-27T15:23:32.806301+0000","last_clean":"2023-11-27T15:23:32.806301+0000","last_became_active":"2023-11-27T15:23:31.649172+0000","last_became_peered":"2023-11-27T15:23:31.649172+0000","last_unstale":"2023-11-27T15:23:32.806301+0000","last_undegraded":"2023-11-27T15:23:32.806301+0000","last_fullsized":"2023-11-27T15:23:32.806301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:11:01.675107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.249201+0000","last_change":"2023-11-27T15:23:26.609457+0000","last_active":"2023-11-27T15:23:58.249201+0000","last_peered":"2023-11-27T15:23:58.249201+0000","last_clean":"2023-11-27T15:23:58.249201+0000","last_became_active":"2023-11-27T15:23:26.609144+0000","last_became_peered":"2023-11-27T15:23:26.609144+0000","last_unstale":"2023-11-27T15:23:58.249201+0000","last_undegraded":"2023-11-27T15:23:58.249201+0000","last_fullsized":"2023-11-27T15:23:58.249201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:24:37.816167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445664+0000","last_change":"2023-11-27T15:23:27.204423+0000","last_active":"2023-11-27T15:23:32.445664+0000","last_peered":"2023-11-27T15:23:32.445664+0000","last_clean":"2023-11-27T15:23:32.445664+0000","last_became_active":"2023-11-27T15:23:27.204104+0000","last_became_peered":"2023-11-27T15:23:27.204104+0000","last_unstale":"2023-11-27T15:23:32.445664+0000","last_undegraded":"2023-11-27T15:23:32.445664+0000","last_fullsized":"2023-11-27T15:23:32.445664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:23:10.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240285+0000","last_change":"2023-11-27T15:23:31.655641+0000","last_active":"2023-11-27T15:23:32.240285+0000","last_peered":"2023-11-27T15:23:32.240285+0000","last_clean":"2023-11-27T15:23:32.240285+0000","last_became_active":"2023-11-27T15:23:31.654991+0000","last_became_peered":"2023-11-27T15:23:31.654991+0000","last_unstale":"2023-11-27T15:23:32.240285+0000","last_undegraded":"2023-11-27T15:23:32.240285+0000","last_fullsized":"2023-11-27T15:23:32.240285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:11:50.509562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445960+0000","last_change":"2023-11-27T15:23:26.620780+0000","last_active":"2023-11-27T15:23:32.445960+0000","last_peered":"2023-11-27T15:23:32.445960+0000","last_clean":"2023-11-27T15:23:32.445960+0000","last_became_active":"2023-11-27T15:23:26.620438+0000","last_became_peered":"2023-11-27T15:23:26.620438+0000","last_unstale":"2023-11-27T15:23:32.445960+0000","last_undegraded":"2023-11-27T15:23:32.445960+0000","last_fullsized":"2023-11-27T15:23:32.445960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:12:21.282175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807271+0000","last_change":"2023-11-27T15:23:27.195581+0000","last_active":"2023-11-27T15:23:32.807271+0000","last_peered":"2023-11-27T15:23:32.807271+0000","last_clean":"2023-11-27T15:23:32.807271+0000","last_became_active":"2023-11-27T15:23:27.195260+0000","last_became_peered":"2023-11-27T15:23:27.195260+0000","last_unstale":"2023-11-27T15:23:32.807271+0000","last_undegraded":"2023-11-27T15:23:32.807271+0000","last_fullsized":"2023-11-27T15:23:32.807271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:05:13.244643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240518+0000","last_change":"2023-11-27T15:23:31.655464+0000","last_active":"2023-11-27T15:23:32.240518+0000","last_peered":"2023-11-27T15:23:32.240518+0000","last_clean":"2023-11-27T15:23:32.240518+0000","last_became_active":"2023-11-27T15:23:31.654952+0000","last_became_peered":"2023-11-27T15:23:31.654952+0000","last_unstale":"2023-11-27T15:23:32.240518+0000","last_undegraded":"2023-11-27T15:23:32.240518+0000","last_fullsized":"2023-11-27T15:23:32.240518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:47:45.892988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.247586+0000","last_change":"2023-11-27T15:23:26.606593+0000","last_active":"2023-11-27T15:23:58.247586+0000","last_peered":"2023-11-27T15:23:58.247586+0000","last_clean":"2023-11-27T15:23:58.247586+0000","last_became_active":"2023-11-27T15:23:26.605719+0000","last_became_peered":"2023-11-27T15:23:26.605719+0000","last_unstale":"2023-11-27T15:23:58.247586+0000","last_undegraded":"2023-11-27T15:23:58.247586+0000","last_fullsized":"2023-11-27T15:23:58.247586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:24.130402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805664+0000","last_change":"2023-11-27T15:23:27.192629+0000","last_active":"2023-11-27T15:23:32.805664+0000","last_peered":"2023-11-27T15:23:32.805664+0000","last_clean":"2023-11-27T15:23:32.805664+0000","last_became_active":"2023-11-27T15:23:27.192327+0000","last_became_peered":"2023-11-27T15:23:27.192327+0000","last_unstale":"2023-11-27T15:23:32.805664+0000","last_undegraded":"2023-11-27T15:23:32.805664+0000","last_fullsized":"2023-11-27T15:23:32.805664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:50:01.298879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240541+0000","last_change":"2023-11-27T15:23:31.648296+0000","last_active":"2023-11-27T15:23:32.240541+0000","last_peered":"2023-11-27T15:23:32.240541+0000","last_clean":"2023-11-27T15:23:32.240541+0000","last_became_active":"2023-11-27T15:23:31.647756+0000","last_became_peered":"2023-11-27T15:23:31.647756+0000","last_unstale":"2023-11-27T15:23:32.240541+0000","last_undegraded":"2023-11-27T15:23:32.240541+0000","last_fullsized":"2023-11-27T15:23:32.240541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:36:20.124660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806078+0000","last_change":"2023-11-27T15:23:26.604061+0000","last_active":"2023-11-27T15:23:32.806078+0000","last_peered":"2023-11-27T15:23:32.806078+0000","last_clean":"2023-11-27T15:23:32.806078+0000","last_became_active":"2023-11-27T15:23:26.603725+0000","last_became_peered":"2023-11-27T15:23:26.603725+0000","last_unstale":"2023-11-27T15:23:32.806078+0000","last_undegraded":"2023-11-27T15:23:32.806078+0000","last_fullsized":"2023-11-27T15:23:32.806078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:57:33.099922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806841+0000","last_change":"2023-11-27T15:23:27.189389+0000","last_active":"2023-11-27T15:23:32.806841+0000","last_peered":"2023-11-27T15:23:32.806841+0000","last_clean":"2023-11-27T15:23:32.806841+0000","last_became_active":"2023-11-27T15:23:27.189072+0000","last_became_peered":"2023-11-27T15:23:27.189072+0000","last_unstale":"2023-11-27T15:23:32.806841+0000","last_undegraded":"2023-11-27T15:23:32.806841+0000","last_fullsized":"2023-11-27T15:23:32.806841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:31:14.484755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806619+0000","last_change":"2023-11-27T15:23:31.639042+0000","last_active":"2023-11-27T15:23:32.806619+0000","last_peered":"2023-11-27T15:23:32.806619+0000","last_clean":"2023-11-27T15:23:32.806619+0000","last_became_active":"2023-11-27T15:23:31.638611+0000","last_became_peered":"2023-11-27T15:23:31.638611+0000","last_unstale":"2023-11-27T15:23:32.806619+0000","last_undegraded":"2023-11-27T15:23:32.806619+0000","last_fullsized":"2023-11-27T15:23:32.806619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:30:47.533959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:24:00.246985+0000","last_change":"2023-11-27T15:23:26.602979+0000","last_active":"2023-11-27T15:24:00.246985+0000","last_peered":"2023-11-27T15:24:00.246985+0000","last_clean":"2023-11-27T15:24:00.246985+0000","last_became_active":"2023-11-27T15:23:26.602681+0000","last_became_peered":"2023-11-27T15:23:26.602681+0000","last_unstale":"2023-11-27T15:24:00.246985+0000","last_undegraded":"2023-11-27T15:24:00.246985+0000","last_fullsized":"2023-11-27T15:24:00.246985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:21:25.923749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807277+0000","last_change":"2023-11-27T15:23:27.187294+0000","last_active":"2023-11-27T15:23:32.807277+0000","last_peered":"2023-11-27T15:23:32.807277+0000","last_clean":"2023-11-27T15:23:32.807277+0000","last_became_active":"2023-11-27T15:23:27.186445+0000","last_became_peered":"2023-11-27T15:23:27.186445+0000","last_unstale":"2023-11-27T15:23:32.807277+0000","last_undegraded":"2023-11-27T15:23:32.807277+0000","last_fullsized":"2023-11-27T15:23:32.807277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:37:31.842245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806339+0000","last_change":"2023-11-27T15:23:31.647523+0000","last_active":"2023-11-27T15:23:32.806339+0000","last_peered":"2023-11-27T15:23:32.806339+0000","last_clean":"2023-11-27T15:23:32.806339+0000","last_became_active":"2023-11-27T15:23:31.646945+0000","last_became_peered":"2023-11-27T15:23:31.646945+0000","last_unstale":"2023-11-27T15:23:32.806339+0000","last_undegraded":"2023-11-27T15:23:32.806339+0000","last_fullsized":"2023-11-27T15:23:32.806339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:40:32.187956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.248958+0000","last_change":"2023-11-27T15:23:26.602182+0000","last_active":"2023-11-27T15:23:58.248958+0000","last_peered":"2023-11-27T15:23:58.248958+0000","last_clean":"2023-11-27T15:23:58.248958+0000","last_became_active":"2023-11-27T15:23:26.601828+0000","last_became_peered":"2023-11-27T15:23:26.601828+0000","last_unstale":"2023-11-27T15:23:58.248958+0000","last_undegraded":"2023-11-27T15:23:58.248958+0000","last_fullsized":"2023-11-27T15:23:58.248958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:41:27.271885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807145+0000","last_change":"2023-11-27T15:23:27.204874+0000","last_active":"2023-11-27T15:23:32.807145+0000","last_peered":"2023-11-27T15:23:32.807145+0000","last_clean":"2023-11-27T15:23:32.807145+0000","last_became_active":"2023-11-27T15:23:27.204588+0000","last_became_peered":"2023-11-27T15:23:27.204588+0000","last_unstale":"2023-11-27T15:23:32.807145+0000","last_undegraded":"2023-11-27T15:23:32.807145+0000","last_fullsized":"2023-11-27T15:23:32.807145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:51:34.799883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151899+0000","last_change":"2023-11-27T15:23:31.655262+0000","last_active":"2023-11-27T15:23:32.151899+0000","last_peered":"2023-11-27T15:23:32.151899+0000","last_clean":"2023-11-27T15:23:32.151899+0000","last_became_active":"2023-11-27T15:23:31.654840+0000","last_became_peered":"2023-11-27T15:23:31.654840+0000","last_unstale":"2023-11-27T15:23:32.151899+0000","last_undegraded":"2023-11-27T15:23:32.151899+0000","last_fullsized":"2023-11-27T15:23:32.151899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:48:46.408391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.250179+0000","last_change":"2023-11-27T15:23:26.610208+0000","last_active":"2023-11-27T15:23:58.250179+0000","last_peered":"2023-11-27T15:23:58.250179+0000","last_clean":"2023-11-27T15:23:58.250179+0000","last_became_active":"2023-11-27T15:23:26.609922+0000","last_became_peered":"2023-11-27T15:23:26.609922+0000","last_unstale":"2023-11-27T15:23:58.250179+0000","last_undegraded":"2023-11-27T15:23:58.250179+0000","last_fullsized":"2023-11-27T15:23:58.250179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:42:52.139587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445392+0000","last_change":"2023-11-27T15:23:27.201533+0000","last_active":"2023-11-27T15:23:32.445392+0000","last_peered":"2023-11-27T15:23:32.445392+0000","last_clean":"2023-11-27T15:23:32.445392+0000","last_became_active":"2023-11-27T15:23:27.200516+0000","last_became_peered":"2023-11-27T15:23:27.200516+0000","last_unstale":"2023-11-27T15:23:32.445392+0000","last_undegraded":"2023-11-27T15:23:32.445392+0000","last_fullsized":"2023-11-27T15:23:32.445392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:11:52.020082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807126+0000","last_change":"2023-11-27T15:23:31.642858+0000","last_active":"2023-11-27T15:23:32.807126+0000","last_peered":"2023-11-27T15:23:32.807126+0000","last_clean":"2023-11-27T15:23:32.807126+0000","last_became_active":"2023-11-27T15:23:31.642367+0000","last_became_peered":"2023-11-27T15:23:31.642367+0000","last_unstale":"2023-11-27T15:23:32.807126+0000","last_undegraded":"2023-11-27T15:23:32.807126+0000","last_fullsized":"2023-11-27T15:23:32.807126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:48:26.840176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806517+0000","last_change":"2023-11-27T15:23:26.602865+0000","last_active":"2023-11-27T15:23:32.806517+0000","last_peered":"2023-11-27T15:23:32.806517+0000","last_clean":"2023-11-27T15:23:32.806517+0000","last_became_active":"2023-11-27T15:23:26.602591+0000","last_became_peered":"2023-11-27T15:23:26.602591+0000","last_unstale":"2023-11-27T15:23:32.806517+0000","last_undegraded":"2023-11-27T15:23:32.806517+0000","last_fullsized":"2023-11-27T15:23:32.806517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:16:04.355452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150163+0000","last_change":"2023-11-27T15:23:27.186551+0000","last_active":"2023-11-27T15:23:32.150163+0000","last_peered":"2023-11-27T15:23:32.150163+0000","last_clean":"2023-11-27T15:23:32.150163+0000","last_became_active":"2023-11-27T15:23:27.185713+0000","last_became_peered":"2023-11-27T15:23:27.185713+0000","last_unstale":"2023-11-27T15:23:32.150163+0000","last_undegraded":"2023-11-27T15:23:32.150163+0000","last_fullsized":"2023-11-27T15:23:32.150163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:01:49.805801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445260+0000","last_change":"2023-11-27T15:23:31.641039+0000","last_active":"2023-11-27T15:23:32.445260+0000","last_peered":"2023-11-27T15:23:32.445260+0000","last_clean":"2023-11-27T15:23:32.445260+0000","last_became_active":"2023-11-27T15:23:31.639234+0000","last_became_peered":"2023-11-27T15:23:31.639234+0000","last_unstale":"2023-11-27T15:23:32.445260+0000","last_undegraded":"2023-11-27T15:23:32.445260+0000","last_fullsized":"2023-11-27T15:23:32.445260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:19:05.470708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.240994+0000","last_change":"2023-11-27T15:23:26.606409+0000","last_active":"2023-11-27T15:23:58.240994+0000","last_peered":"2023-11-27T15:23:58.240994+0000","last_clean":"2023-11-27T15:23:58.240994+0000","last_became_active":"2023-11-27T15:23:26.606060+0000","last_became_peered":"2023-11-27T15:23:26.606060+0000","last_unstale":"2023-11-27T15:23:58.240994+0000","last_undegraded":"2023-11-27T15:23:58.240994+0000","last_fullsized":"2023-11-27T15:23:58.240994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:34:52.933461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805905+0000","last_change":"2023-11-27T15:23:27.180766+0000","last_active":"2023-11-27T15:23:32.805905+0000","last_peered":"2023-11-27T15:23:32.805905+0000","last_clean":"2023-11-27T15:23:32.805905+0000","last_became_active":"2023-11-27T15:23:27.180382+0000","last_became_peered":"2023-11-27T15:23:27.180382+0000","last_unstale":"2023-11-27T15:23:32.805905+0000","last_undegraded":"2023-11-27T15:23:32.805905+0000","last_fullsized":"2023-11-27T15:23:32.805905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:50:04.142745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.149973+0000","last_change":"2023-11-27T15:23:31.641697+0000","last_active":"2023-11-27T15:23:32.149973+0000","last_peered":"2023-11-27T15:23:32.149973+0000","last_clean":"2023-11-27T15:23:32.149973+0000","last_became_active":"2023-11-27T15:23:31.641246+0000","last_became_peered":"2023-11-27T15:23:31.641246+0000","last_unstale":"2023-11-27T15:23:32.149973+0000","last_undegraded":"2023-11-27T15:23:32.149973+0000","last_fullsized":"2023-11-27T15:23:32.149973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:51:50.437000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.444150+0000","last_change":"2023-11-27T15:23:26.605307+0000","last_active":"2023-11-27T15:23:57.444150+0000","last_peered":"2023-11-27T15:23:57.444150+0000","last_clean":"2023-11-27T15:23:57.444150+0000","last_became_active":"2023-11-27T15:23:26.605047+0000","last_became_peered":"2023-11-27T15:23:26.605047+0000","last_unstale":"2023-11-27T15:23:57.444150+0000","last_undegraded":"2023-11-27T15:23:57.444150+0000","last_fullsized":"2023-11-27T15:23:57.444150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:43:04.881701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807258+0000","last_change":"2023-11-27T15:23:27.194469+0000","last_active":"2023-11-27T15:23:32.807258+0000","last_peered":"2023-11-27T15:23:32.807258+0000","last_clean":"2023-11-27T15:23:32.807258+0000","last_became_active":"2023-11-27T15:23:27.193930+0000","last_became_peered":"2023-11-27T15:23:27.193930+0000","last_unstale":"2023-11-27T15:23:32.807258+0000","last_undegraded":"2023-11-27T15:23:32.807258+0000","last_fullsized":"2023-11-27T15:23:32.807258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:42:19.160539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240743+0000","last_change":"2023-11-27T15:23:31.655787+0000","last_active":"2023-11-27T15:23:32.240743+0000","last_peered":"2023-11-27T15:23:32.240743+0000","last_clean":"2023-11-27T15:23:32.240743+0000","last_became_active":"2023-11-27T15:23:31.654847+0000","last_became_peered":"2023-11-27T15:23:31.654847+0000","last_unstale":"2023-11-27T15:23:32.240743+0000","last_undegraded":"2023-11-27T15:23:32.240743+0000","last_fullsized":"2023-11-27T15:23:32.240743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:39:48.554692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.444225+0000","last_change":"2023-11-27T15:23:26.598669+0000","last_active":"2023-11-27T15:23:57.444225+0000","last_peered":"2023-11-27T15:23:57.444225+0000","last_clean":"2023-11-27T15:23:57.444225+0000","last_became_active":"2023-11-27T15:23:26.598224+0000","last_became_peered":"2023-11-27T15:23:26.598224+0000","last_unstale":"2023-11-27T15:23:57.444225+0000","last_undegraded":"2023-11-27T15:23:57.444225+0000","last_fullsized":"2023-11-27T15:23:57.444225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:54:30.744998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805760+0000","last_change":"2023-11-27T15:23:27.182800+0000","last_active":"2023-11-27T15:23:32.805760+0000","last_peered":"2023-11-27T15:23:32.805760+0000","last_clean":"2023-11-27T15:23:32.805760+0000","last_became_active":"2023-11-27T15:23:27.182572+0000","last_became_peered":"2023-11-27T15:23:27.182572+0000","last_unstale":"2023-11-27T15:23:32.805760+0000","last_undegraded":"2023-11-27T15:23:32.805760+0000","last_fullsized":"2023-11-27T15:23:32.805760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:32:57.327675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150116+0000","last_change":"2023-11-27T15:23:31.645586+0000","last_active":"2023-11-27T15:23:32.150116+0000","last_peered":"2023-11-27T15:23:32.150116+0000","last_clean":"2023-11-27T15:23:32.150116+0000","last_became_active":"2023-11-27T15:23:31.644897+0000","last_became_peered":"2023-11-27T15:23:31.644897+0000","last_unstale":"2023-11-27T15:23:32.150116+0000","last_undegraded":"2023-11-27T15:23:32.150116+0000","last_fullsized":"2023-11-27T15:23:32.150116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:44:50.702316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'19","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:59.767999+0000","last_change":"2023-11-27T15:23:26.601652+0000","last_active":"2023-11-27T15:23:59.767999+0000","last_peered":"2023-11-27T15:23:59.767999+0000","last_clean":"2023-11-27T15:23:59.767999+0000","last_became_active":"2023-11-27T15:23:26.601306+0000","last_became_peered":"2023-11-27T15:23:26.601306+0000","last_unstale":"2023-11-27T15:23:59.767999+0000","last_undegraded":"2023-11-27T15:23:59.767999+0000","last_fullsized":"2023-11-27T15:23:59.767999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:02:11.299102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21175,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":7,"num_write":20,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805854+0000","last_change":"2023-11-27T15:23:27.186709+0000","last_active":"2023-11-27T15:23:32.805854+0000","last_peered":"2023-11-27T15:23:32.805854+0000","last_clean":"2023-11-27T15:23:32.805854+0000","last_became_active":"2023-11-27T15:23:27.186390+0000","last_became_peered":"2023-11-27T15:23:27.186390+0000","last_unstale":"2023-11-27T15:23:32.805854+0000","last_undegraded":"2023-11-27T15:23:32.805854+0000","last_fullsized":"2023-11-27T15:23:32.805854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:42:34.421537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807393+0000","last_change":"2023-11-27T15:23:31.651851+0000","last_active":"2023-11-27T15:23:32.807393+0000","last_peered":"2023-11-27T15:23:32.807393+0000","last_clean":"2023-11-27T15:23:32.807393+0000","last_became_active":"2023-11-27T15:23:31.651409+0000","last_became_peered":"2023-11-27T15:23:31.651409+0000","last_unstale":"2023-11-27T15:23:32.807393+0000","last_undegraded":"2023-11-27T15:23:32.807393+0000","last_fullsized":"2023-11-27T15:23:32.807393+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:39:45.865051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.248472+0000","last_change":"2023-11-27T15:23:26.602719+0000","last_active":"2023-11-27T15:23:58.248472+0000","last_peered":"2023-11-27T15:23:58.248472+0000","last_clean":"2023-11-27T15:23:58.248472+0000","last_became_active":"2023-11-27T15:23:26.602372+0000","last_became_peered":"2023-11-27T15:23:26.602372+0000","last_unstale":"2023-11-27T15:23:58.248472+0000","last_undegraded":"2023-11-27T15:23:58.248472+0000","last_fullsized":"2023-11-27T15:23:58.248472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:50:18.169759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806555+0000","last_change":"2023-11-27T15:23:27.200479+0000","last_active":"2023-11-27T15:23:32.806555+0000","last_peered":"2023-11-27T15:23:32.806555+0000","last_clean":"2023-11-27T15:23:32.806555+0000","last_became_active":"2023-11-27T15:23:27.200155+0000","last_became_peered":"2023-11-27T15:23:27.200155+0000","last_unstale":"2023-11-27T15:23:32.806555+0000","last_undegraded":"2023-11-27T15:23:32.806555+0000","last_fullsized":"2023-11-27T15:23:32.806555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:00:32.990163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151529+0000","last_change":"2023-11-27T15:23:31.655255+0000","last_active":"2023-11-27T15:23:32.151529+0000","last_peered":"2023-11-27T15:23:32.151529+0000","last_clean":"2023-11-27T15:23:32.151529+0000","last_became_active":"2023-11-27T15:23:31.654789+0000","last_became_peered":"2023-11-27T15:23:31.654789+0000","last_unstale":"2023-11-27T15:23:32.151529+0000","last_undegraded":"2023-11-27T15:23:32.151529+0000","last_fullsized":"2023-11-27T15:23:32.151529+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:06:57.555131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.443790+0000","last_change":"2023-11-27T15:23:26.605927+0000","last_active":"2023-11-27T15:23:57.443790+0000","last_peered":"2023-11-27T15:23:57.443790+0000","last_clean":"2023-11-27T15:23:57.443790+0000","last_became_active":"2023-11-27T15:23:26.605515+0000","last_became_peered":"2023-11-27T15:23:26.605515+0000","last_unstale":"2023-11-27T15:23:57.443790+0000","last_undegraded":"2023-11-27T15:23:57.443790+0000","last_fullsized":"2023-11-27T15:23:57.443790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:39:06.131896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387649+0000","last_change":"2023-11-27T15:23:27.200237+0000","last_active":"2023-11-27T15:23:33.387649+0000","last_peered":"2023-11-27T15:23:33.387649+0000","last_clean":"2023-11-27T15:23:33.387649+0000","last_became_active":"2023-11-27T15:23:27.199952+0000","last_became_peered":"2023-11-27T15:23:27.199952+0000","last_unstale":"2023-11-27T15:23:33.387649+0000","last_undegraded":"2023-11-27T15:23:33.387649+0000","last_fullsized":"2023-11-27T15:23:33.387649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:25:38.280448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150647+0000","last_change":"2023-11-27T15:23:31.650718+0000","last_active":"2023-11-27T15:23:32.150647+0000","last_peered":"2023-11-27T15:23:32.150647+0000","last_clean":"2023-11-27T15:23:32.150647+0000","last_became_active":"2023-11-27T15:23:31.650144+0000","last_became_peered":"2023-11-27T15:23:31.650144+0000","last_unstale":"2023-11-27T15:23:32.150647+0000","last_undegraded":"2023-11-27T15:23:32.150647+0000","last_fullsized":"2023-11-27T15:23:32.150647+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:06:51.387716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:24:00.252935+0000","last_change":"2023-11-27T15:23:26.600962+0000","last_active":"2023-11-27T15:24:00.252935+0000","last_peered":"2023-11-27T15:24:00.252935+0000","last_clean":"2023-11-27T15:24:00.252935+0000","last_became_active":"2023-11-27T15:23:26.600628+0000","last_became_peered":"2023-11-27T15:23:26.600628+0000","last_unstale":"2023-11-27T15:24:00.252935+0000","last_undegraded":"2023-11-27T15:24:00.252935+0000","last_fullsized":"2023-11-27T15:24:00.252935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:13:05.116836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387225+0000","last_change":"2023-11-27T15:23:27.191939+0000","last_active":"2023-11-27T15:23:33.387225+0000","last_peered":"2023-11-27T15:23:33.387225+0000","last_clean":"2023-11-27T15:23:33.387225+0000","last_became_active":"2023-11-27T15:23:27.191634+0000","last_became_peered":"2023-11-27T15:23:27.191634+0000","last_unstale":"2023-11-27T15:23:33.387225+0000","last_undegraded":"2023-11-27T15:23:33.387225+0000","last_fullsized":"2023-11-27T15:23:33.387225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:09.985147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805930+0000","last_change":"2023-11-27T15:23:31.650733+0000","last_active":"2023-11-27T15:23:32.805930+0000","last_peered":"2023-11-27T15:23:32.805930+0000","last_clean":"2023-11-27T15:23:32.805930+0000","last_became_active":"2023-11-27T15:23:31.650280+0000","last_became_peered":"2023-11-27T15:23:31.650280+0000","last_unstale":"2023-11-27T15:23:32.805930+0000","last_undegraded":"2023-11-27T15:23:32.805930+0000","last_fullsized":"2023-11-27T15:23:32.805930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:34:35.055557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.249316+0000","last_change":"2023-11-27T15:23:26.615536+0000","last_active":"2023-11-27T15:23:58.249316+0000","last_peered":"2023-11-27T15:23:58.249316+0000","last_clean":"2023-11-27T15:23:58.249316+0000","last_became_active":"2023-11-27T15:23:26.615180+0000","last_became_peered":"2023-11-27T15:23:26.615180+0000","last_unstale":"2023-11-27T15:23:58.249316+0000","last_undegraded":"2023-11-27T15:23:58.249316+0000","last_fullsized":"2023-11-27T15:23:58.249316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:05:54.739431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806183+0000","last_change":"2023-11-27T15:23:27.189011+0000","last_active":"2023-11-27T15:23:32.806183+0000","last_peered":"2023-11-27T15:23:32.806183+0000","last_clean":"2023-11-27T15:23:32.806183+0000","last_became_active":"2023-11-27T15:23:27.188673+0000","last_became_peered":"2023-11-27T15:23:27.188673+0000","last_unstale":"2023-11-27T15:23:32.806183+0000","last_undegraded":"2023-11-27T15:23:32.806183+0000","last_fullsized":"2023-11-27T15:23:32.806183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:42:38.911696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806049+0000","last_change":"2023-11-27T15:23:31.644073+0000","last_active":"2023-11-27T15:23:32.806049+0000","last_peered":"2023-11-27T15:23:32.806049+0000","last_clean":"2023-11-27T15:23:32.806049+0000","last_became_active":"2023-11-27T15:23:31.643689+0000","last_became_peered":"2023-11-27T15:23:31.643689+0000","last_unstale":"2023-11-27T15:23:32.806049+0000","last_undegraded":"2023-11-27T15:23:32.806049+0000","last_fullsized":"2023-11-27T15:23:32.806049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:16:46.525106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806706+0000","last_change":"2023-11-27T15:23:26.604034+0000","last_active":"2023-11-27T15:23:32.806706+0000","last_peered":"2023-11-27T15:23:32.806706+0000","last_clean":"2023-11-27T15:23:32.806706+0000","last_became_active":"2023-11-27T15:23:26.603689+0000","last_became_peered":"2023-11-27T15:23:26.603689+0000","last_unstale":"2023-11-27T15:23:32.806706+0000","last_undegraded":"2023-11-27T15:23:32.806706+0000","last_fullsized":"2023-11-27T15:23:32.806706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:40:44.325758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151781+0000","last_change":"2023-11-27T15:23:27.192723+0000","last_active":"2023-11-27T15:23:32.151781+0000","last_peered":"2023-11-27T15:23:32.151781+0000","last_clean":"2023-11-27T15:23:32.151781+0000","last_became_active":"2023-11-27T15:23:27.192034+0000","last_became_peered":"2023-11-27T15:23:27.192034+0000","last_unstale":"2023-11-27T15:23:32.151781+0000","last_undegraded":"2023-11-27T15:23:32.151781+0000","last_fullsized":"2023-11-27T15:23:32.151781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:40:19.176841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807181+0000","last_change":"2023-11-27T15:23:31.652756+0000","last_active":"2023-11-27T15:23:32.807181+0000","last_peered":"2023-11-27T15:23:32.807181+0000","last_clean":"2023-11-27T15:23:32.807181+0000","last_became_active":"2023-11-27T15:23:31.652280+0000","last_became_peered":"2023-11-27T15:23:31.652280+0000","last_unstale":"2023-11-27T15:23:32.807181+0000","last_undegraded":"2023-11-27T15:23:32.807181+0000","last_fullsized":"2023-11-27T15:23:32.807181+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:42:39.820208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150390+0000","last_change":"2023-11-27T15:23:26.609927+0000","last_active":"2023-11-27T15:23:32.150390+0000","last_peered":"2023-11-27T15:23:32.150390+0000","last_clean":"2023-11-27T15:23:32.150390+0000","last_became_active":"2023-11-27T15:23:26.608928+0000","last_became_peered":"2023-11-27T15:23:26.608928+0000","last_unstale":"2023-11-27T15:23:32.150390+0000","last_undegraded":"2023-11-27T15:23:32.150390+0000","last_fullsized":"2023-11-27T15:23:32.150390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:22:11.858889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151949+0000","last_change":"2023-11-27T15:23:27.204619+0000","last_active":"2023-11-27T15:23:32.151949+0000","last_peered":"2023-11-27T15:23:32.151949+0000","last_clean":"2023-11-27T15:23:32.151949+0000","last_became_active":"2023-11-27T15:23:27.204298+0000","last_became_peered":"2023-11-27T15:23:27.204298+0000","last_unstale":"2023-11-27T15:23:32.151949+0000","last_undegraded":"2023-11-27T15:23:32.151949+0000","last_fullsized":"2023-11-27T15:23:32.151949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:04:25.075907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807083+0000","last_change":"2023-11-27T15:23:31.644763+0000","last_active":"2023-11-27T15:23:32.807083+0000","last_peered":"2023-11-27T15:23:32.807083+0000","last_clean":"2023-11-27T15:23:32.807083+0000","last_became_active":"2023-11-27T15:23:31.644106+0000","last_became_peered":"2023-11-27T15:23:31.644106+0000","last_unstale":"2023-11-27T15:23:32.807083+0000","last_undegraded":"2023-11-27T15:23:32.807083+0000","last_fullsized":"2023-11-27T15:23:32.807083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:04:07.977842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387293+0000","last_change":"2023-11-27T15:23:26.614703+0000","last_active":"2023-11-27T15:23:33.387293+0000","last_peered":"2023-11-27T15:23:33.387293+0000","last_clean":"2023-11-27T15:23:33.387293+0000","last_became_active":"2023-11-27T15:23:26.614324+0000","last_became_peered":"2023-11-27T15:23:26.614324+0000","last_unstale":"2023-11-27T15:23:33.387293+0000","last_undegraded":"2023-11-27T15:23:33.387293+0000","last_fullsized":"2023-11-27T15:23:33.387293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:49:23.116644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151572+0000","last_change":"2023-11-27T15:23:27.197288+0000","last_active":"2023-11-27T15:23:32.151572+0000","last_peered":"2023-11-27T15:23:32.151572+0000","last_clean":"2023-11-27T15:23:32.151572+0000","last_became_active":"2023-11-27T15:23:27.196925+0000","last_became_peered":"2023-11-27T15:23:27.196925+0000","last_unstale":"2023-11-27T15:23:32.151572+0000","last_undegraded":"2023-11-27T15:23:32.151572+0000","last_fullsized":"2023-11-27T15:23:32.151572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:43:50.170855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806017+0000","last_change":"2023-11-27T15:23:31.652701+0000","last_active":"2023-11-27T15:23:32.806017+0000","last_peered":"2023-11-27T15:23:32.806017+0000","last_clean":"2023-11-27T15:23:32.806017+0000","last_became_active":"2023-11-27T15:23:31.652259+0000","last_became_peered":"2023-11-27T15:23:31.652259+0000","last_unstale":"2023-11-27T15:23:32.806017+0000","last_undegraded":"2023-11-27T15:23:32.806017+0000","last_fullsized":"2023-11-27T15:23:32.806017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:51:09.415292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806638+0000","last_change":"2023-11-27T15:23:27.190103+0000","last_active":"2023-11-27T15:23:32.806638+0000","last_peered":"2023-11-27T15:23:32.806638+0000","last_clean":"2023-11-27T15:23:32.806638+0000","last_became_active":"2023-11-27T15:23:27.189803+0000","last_became_peered":"2023-11-27T15:23:27.189803+0000","last_unstale":"2023-11-27T15:23:32.806638+0000","last_undegraded":"2023-11-27T15:23:32.806638+0000","last_fullsized":"2023-11-27T15:23:32.806638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:11:38.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.250788+0000","last_change":"2023-11-27T15:23:26.606338+0000","last_active":"2023-11-27T15:23:58.250788+0000","last_peered":"2023-11-27T15:23:58.250788+0000","last_clean":"2023-11-27T15:23:58.250788+0000","last_became_active":"2023-11-27T15:23:26.601152+0000","last_became_peered":"2023-11-27T15:23:26.601152+0000","last_unstale":"2023-11-27T15:23:58.250788+0000","last_undegraded":"2023-11-27T15:23:58.250788+0000","last_fullsized":"2023-11-27T15:23:58.250788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:25:51.376468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387602+0000","last_change":"2023-11-27T15:23:31.646457+0000","last_active":"2023-11-27T15:23:33.387602+0000","last_peered":"2023-11-27T15:23:33.387602+0000","last_clean":"2023-11-27T15:23:33.387602+0000","last_became_active":"2023-11-27T15:23:31.646071+0000","last_became_peered":"2023-11-27T15:23:31.646071+0000","last_unstale":"2023-11-27T15:23:33.387602+0000","last_undegraded":"2023-11-27T15:23:33.387602+0000","last_fullsized":"2023-11-27T15:23:33.387602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:50:17.518444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445545+0000","last_change":"2023-11-27T15:23:27.201405+0000","last_active":"2023-11-27T15:23:32.445545+0000","last_peered":"2023-11-27T15:23:32.445545+0000","last_clean":"2023-11-27T15:23:32.445545+0000","last_became_active":"2023-11-27T15:23:27.200951+0000","last_became_peered":"2023-11-27T15:23:27.200951+0000","last_unstale":"2023-11-27T15:23:32.445545+0000","last_undegraded":"2023-11-27T15:23:32.445545+0000","last_fullsized":"2023-11-27T15:23:32.445545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:43:06.543689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.250588+0000","last_change":"2023-11-27T15:23:26.604328+0000","last_active":"2023-11-27T15:23:58.250588+0000","last_peered":"2023-11-27T15:23:58.250588+0000","last_clean":"2023-11-27T15:23:58.250588+0000","last_became_active":"2023-11-27T15:23:26.603937+0000","last_became_peered":"2023-11-27T15:23:26.603937+0000","last_unstale":"2023-11-27T15:23:58.250588+0000","last_undegraded":"2023-11-27T15:23:58.250588+0000","last_fullsized":"2023-11-27T15:23:58.250588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:13:28.104497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.386998+0000","last_change":"2023-11-27T15:22:17.949235+0000","last_active":"2023-11-27T15:23:33.386998+0000","last_peered":"2023-11-27T15:23:33.386998+0000","last_clean":"2023-11-27T15:23:33.386998+0000","last_became_active":"2023-11-27T15:22:17.948822+0000","last_became_peered":"2023-11-27T15:22:17.948822+0000","last_unstale":"2023-11-27T15:23:33.386998+0000","last_undegraded":"2023-11-27T15:23:33.386998+0000","last_fullsized":"2023-11-27T15:23:33.386998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:19:02.909930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:19:02.909930+0000","last_clean_scrub_stamp":"2023-11-27T15:19:02.909930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:35:54.127781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807234+0000","last_change":"2023-11-27T15:23:31.654325+0000","last_active":"2023-11-27T15:23:32.807234+0000","last_peered":"2023-11-27T15:23:32.807234+0000","last_clean":"2023-11-27T15:23:32.807234+0000","last_became_active":"2023-11-27T15:23:31.653857+0000","last_became_peered":"2023-11-27T15:23:31.653857+0000","last_unstale":"2023-11-27T15:23:32.807234+0000","last_undegraded":"2023-11-27T15:23:32.807234+0000","last_fullsized":"2023-11-27T15:23:32.807234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:50:07.042580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805768+0000","last_change":"2023-11-27T15:23:27.192623+0000","last_active":"2023-11-27T15:23:32.805768+0000","last_peered":"2023-11-27T15:23:32.805768+0000","last_clean":"2023-11-27T15:23:32.805768+0000","last_became_active":"2023-11-27T15:23:27.192266+0000","last_became_peered":"2023-11-27T15:23:27.192266+0000","last_unstale":"2023-11-27T15:23:32.805768+0000","last_undegraded":"2023-11-27T15:23:32.805768+0000","last_fullsized":"2023-11-27T15:23:32.805768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:19:04.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.248186+0000","last_change":"2023-11-27T15:23:26.603078+0000","last_active":"2023-11-27T15:23:58.248186+0000","last_peered":"2023-11-27T15:23:58.248186+0000","last_clean":"2023-11-27T15:23:58.248186+0000","last_became_active":"2023-11-27T15:23:26.602758+0000","last_became_peered":"2023-11-27T15:23:26.602758+0000","last_unstale":"2023-11-27T15:23:58.248186+0000","last_undegraded":"2023-11-27T15:23:58.248186+0000","last_fullsized":"2023-11-27T15:23:58.248186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:53:27.360807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151477+0000","last_change":"2023-11-27T15:23:31.648266+0000","last_active":"2023-11-27T15:23:32.151477+0000","last_peered":"2023-11-27T15:23:32.151477+0000","last_clean":"2023-11-27T15:23:32.151477+0000","last_became_active":"2023-11-27T15:23:31.647804+0000","last_became_peered":"2023-11-27T15:23:31.647804+0000","last_unstale":"2023-11-27T15:23:32.151477+0000","last_undegraded":"2023-11-27T15:23:32.151477+0000","last_fullsized":"2023-11-27T15:23:32.151477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:14:46.320009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.440496+0000","last_change":"2023-11-27T15:23:26.615577+0000","last_active":"2023-11-27T15:23:57.440496+0000","last_peered":"2023-11-27T15:23:57.440496+0000","last_clean":"2023-11-27T15:23:57.440496+0000","last_became_active":"2023-11-27T15:23:26.615287+0000","last_became_peered":"2023-11-27T15:23:26.615287+0000","last_unstale":"2023-11-27T15:23:57.440496+0000","last_undegraded":"2023-11-27T15:23:57.440496+0000","last_fullsized":"2023-11-27T15:23:57.440496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:17:38.255280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807155+0000","last_change":"2023-11-27T15:23:27.203162+0000","last_active":"2023-11-27T15:23:32.807155+0000","last_peered":"2023-11-27T15:23:32.807155+0000","last_clean":"2023-11-27T15:23:32.807155+0000","last_became_active":"2023-11-27T15:23:27.201725+0000","last_became_peered":"2023-11-27T15:23:27.201725+0000","last_unstale":"2023-11-27T15:23:32.807155+0000","last_undegraded":"2023-11-27T15:23:32.807155+0000","last_fullsized":"2023-11-27T15:23:32.807155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:26:49.344996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445343+0000","last_change":"2023-11-27T15:23:31.653253+0000","last_active":"2023-11-27T15:23:32.445343+0000","last_peered":"2023-11-27T15:23:32.445343+0000","last_clean":"2023-11-27T15:23:32.445343+0000","last_became_active":"2023-11-27T15:23:31.652777+0000","last_became_peered":"2023-11-27T15:23:31.652777+0000","last_unstale":"2023-11-27T15:23:32.445343+0000","last_undegraded":"2023-11-27T15:23:32.445343+0000","last_fullsized":"2023-11-27T15:23:32.445343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:42:04.663789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.249981+0000","last_change":"2023-11-27T15:23:26.610599+0000","last_active":"2023-11-27T15:23:58.249981+0000","last_peered":"2023-11-27T15:23:58.249981+0000","last_clean":"2023-11-27T15:23:58.249981+0000","last_became_active":"2023-11-27T15:23:26.610205+0000","last_became_peered":"2023-11-27T15:23:26.610205+0000","last_unstale":"2023-11-27T15:23:58.249981+0000","last_undegraded":"2023-11-27T15:23:58.249981+0000","last_fullsized":"2023-11-27T15:23:58.249981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:14:37.484874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445743+0000","last_change":"2023-11-27T15:23:27.203367+0000","last_active":"2023-11-27T15:23:32.445743+0000","last_peered":"2023-11-27T15:23:32.445743+0000","last_clean":"2023-11-27T15:23:32.445743+0000","last_became_active":"2023-11-27T15:23:27.203054+0000","last_became_peered":"2023-11-27T15:23:27.203054+0000","last_unstale":"2023-11-27T15:23:32.445743+0000","last_undegraded":"2023-11-27T15:23:32.445743+0000","last_fullsized":"2023-11-27T15:23:32.445743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:36:56.852033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805939+0000","last_change":"2023-11-27T15:23:31.638372+0000","last_active":"2023-11-27T15:23:32.805939+0000","last_peered":"2023-11-27T15:23:32.805939+0000","last_clean":"2023-11-27T15:23:32.805939+0000","last_became_active":"2023-11-27T15:23:31.637973+0000","last_became_peered":"2023-11-27T15:23:31.637973+0000","last_unstale":"2023-11-27T15:23:32.805939+0000","last_undegraded":"2023-11-27T15:23:32.805939+0000","last_fullsized":"2023-11-27T15:23:32.805939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:01:01.382773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.441204+0000","last_change":"2023-11-27T15:23:26.594218+0000","last_active":"2023-11-27T15:23:57.441204+0000","last_peered":"2023-11-27T15:23:57.441204+0000","last_clean":"2023-11-27T15:23:57.441204+0000","last_became_active":"2023-11-27T15:23:26.593842+0000","last_became_peered":"2023-11-27T15:23:26.593842+0000","last_unstale":"2023-11-27T15:23:57.441204+0000","last_undegraded":"2023-11-27T15:23:57.441204+0000","last_fullsized":"2023-11-27T15:23:57.441204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:10:10.686777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807263+0000","last_change":"2023-11-27T15:23:27.211258+0000","last_active":"2023-11-27T15:23:32.807263+0000","last_peered":"2023-11-27T15:23:32.807263+0000","last_clean":"2023-11-27T15:23:32.807263+0000","last_became_active":"2023-11-27T15:23:27.210963+0000","last_became_peered":"2023-11-27T15:23:27.210963+0000","last_unstale":"2023-11-27T15:23:32.807263+0000","last_undegraded":"2023-11-27T15:23:32.807263+0000","last_fullsized":"2023-11-27T15:23:32.807263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:29:21.488753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240782+0000","last_change":"2023-11-27T15:23:31.652938+0000","last_active":"2023-11-27T15:23:32.240782+0000","last_peered":"2023-11-27T15:23:32.240782+0000","last_clean":"2023-11-27T15:23:32.240782+0000","last_became_active":"2023-11-27T15:23:31.652504+0000","last_became_peered":"2023-11-27T15:23:31.652504+0000","last_unstale":"2023-11-27T15:23:32.240782+0000","last_undegraded":"2023-11-27T15:23:32.240782+0000","last_fullsized":"2023-11-27T15:23:32.240782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:59:59.372384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.443570+0000","last_change":"2023-11-27T15:23:26.602328+0000","last_active":"2023-11-27T15:23:57.443570+0000","last_peered":"2023-11-27T15:23:57.443570+0000","last_clean":"2023-11-27T15:23:57.443570+0000","last_became_active":"2023-11-27T15:23:26.601974+0000","last_became_peered":"2023-11-27T15:23:26.601974+0000","last_unstale":"2023-11-27T15:23:57.443570+0000","last_undegraded":"2023-11-27T15:23:57.443570+0000","last_fullsized":"2023-11-27T15:23:57.443570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:46:03.610233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805397+0000","last_change":"2023-11-27T15:23:27.194577+0000","last_active":"2023-11-27T15:23:32.805397+0000","last_peered":"2023-11-27T15:23:32.805397+0000","last_clean":"2023-11-27T15:23:32.805397+0000","last_became_active":"2023-11-27T15:23:27.194186+0000","last_became_peered":"2023-11-27T15:23:27.194186+0000","last_unstale":"2023-11-27T15:23:32.805397+0000","last_undegraded":"2023-11-27T15:23:32.805397+0000","last_fullsized":"2023-11-27T15:23:32.805397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:01:42.662103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150738+0000","last_change":"2023-11-27T15:23:31.649019+0000","last_active":"2023-11-27T15:23:32.150738+0000","last_peered":"2023-11-27T15:23:32.150738+0000","last_clean":"2023-11-27T15:23:32.150738+0000","last_became_active":"2023-11-27T15:23:31.647182+0000","last_became_peered":"2023-11-27T15:23:31.647182+0000","last_unstale":"2023-11-27T15:23:32.150738+0000","last_undegraded":"2023-11-27T15:23:32.150738+0000","last_fullsized":"2023-11-27T15:23:32.150738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:13:57.191561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.444377+0000","last_change":"2023-11-27T15:23:26.611336+0000","last_active":"2023-11-27T15:23:57.444377+0000","last_peered":"2023-11-27T15:23:57.444377+0000","last_clean":"2023-11-27T15:23:57.444377+0000","last_became_active":"2023-11-27T15:23:26.610819+0000","last_became_peered":"2023-11-27T15:23:26.610819+0000","last_unstale":"2023-11-27T15:23:57.444377+0000","last_undegraded":"2023-11-27T15:23:57.444377+0000","last_fullsized":"2023-11-27T15:23:57.444377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:35:57.668215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151229+0000","last_change":"2023-11-27T15:23:27.198336+0000","last_active":"2023-11-27T15:23:32.151229+0000","last_peered":"2023-11-27T15:23:32.151229+0000","last_clean":"2023-11-27T15:23:32.151229+0000","last_became_active":"2023-11-27T15:23:27.198062+0000","last_became_peered":"2023-11-27T15:23:27.198062+0000","last_unstale":"2023-11-27T15:23:32.151229+0000","last_undegraded":"2023-11-27T15:23:32.151229+0000","last_fullsized":"2023-11-27T15:23:32.151229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:10:31.709614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240780+0000","last_change":"2023-11-27T15:23:31.655526+0000","last_active":"2023-11-27T15:23:32.240780+0000","last_peered":"2023-11-27T15:23:32.240780+0000","last_clean":"2023-11-27T15:23:32.240780+0000","last_became_active":"2023-11-27T15:23:31.655011+0000","last_became_peered":"2023-11-27T15:23:31.655011+0000","last_unstale":"2023-11-27T15:23:32.240780+0000","last_undegraded":"2023-11-27T15:23:32.240780+0000","last_fullsized":"2023-11-27T15:23:32.240780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:37:13.232844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806016+0000","last_change":"2023-11-27T15:23:26.604536+0000","last_active":"2023-11-27T15:23:32.806016+0000","last_peered":"2023-11-27T15:23:32.806016+0000","last_clean":"2023-11-27T15:23:32.806016+0000","last_became_active":"2023-11-27T15:23:26.604239+0000","last_became_peered":"2023-11-27T15:23:26.604239+0000","last_unstale":"2023-11-27T15:23:32.806016+0000","last_undegraded":"2023-11-27T15:23:32.806016+0000","last_fullsized":"2023-11-27T15:23:32.806016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:48:36.835527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445522+0000","last_change":"2023-11-27T15:23:27.201159+0000","last_active":"2023-11-27T15:23:32.445522+0000","last_peered":"2023-11-27T15:23:32.445522+0000","last_clean":"2023-11-27T15:23:32.445522+0000","last_became_active":"2023-11-27T15:23:27.200800+0000","last_became_peered":"2023-11-27T15:23:27.200800+0000","last_unstale":"2023-11-27T15:23:32.445522+0000","last_undegraded":"2023-11-27T15:23:32.445522+0000","last_fullsized":"2023-11-27T15:23:32.445522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:05:44.391748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240746+0000","last_change":"2023-11-27T15:23:31.652835+0000","last_active":"2023-11-27T15:23:32.240746+0000","last_peered":"2023-11-27T15:23:32.240746+0000","last_clean":"2023-11-27T15:23:32.240746+0000","last_became_active":"2023-11-27T15:23:31.652425+0000","last_became_peered":"2023-11-27T15:23:31.652425+0000","last_unstale":"2023-11-27T15:23:32.240746+0000","last_undegraded":"2023-11-27T15:23:32.240746+0000","last_fullsized":"2023-11-27T15:23:32.240746+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:32:00.109457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.442980+0000","last_change":"2023-11-27T15:23:26.606303+0000","last_active":"2023-11-27T15:23:57.442980+0000","last_peered":"2023-11-27T15:23:57.442980+0000","last_clean":"2023-11-27T15:23:57.442980+0000","last_became_active":"2023-11-27T15:23:26.605827+0000","last_became_peered":"2023-11-27T15:23:26.605827+0000","last_unstale":"2023-11-27T15:23:57.442980+0000","last_undegraded":"2023-11-27T15:23:57.442980+0000","last_fullsized":"2023-11-27T15:23:57.442980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:01:59.336718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387434+0000","last_change":"2023-11-27T15:23:27.198422+0000","last_active":"2023-11-27T15:23:33.387434+0000","last_peered":"2023-11-27T15:23:33.387434+0000","last_clean":"2023-11-27T15:23:33.387434+0000","last_became_active":"2023-11-27T15:23:27.198116+0000","last_became_peered":"2023-11-27T15:23:27.198116+0000","last_unstale":"2023-11-27T15:23:33.387434+0000","last_undegraded":"2023-11-27T15:23:33.387434+0000","last_fullsized":"2023-11-27T15:23:33.387434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:06:48.770689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805383+0000","last_change":"2023-11-27T15:23:31.647798+0000","last_active":"2023-11-27T15:23:32.805383+0000","last_peered":"2023-11-27T15:23:32.805383+0000","last_clean":"2023-11-27T15:23:32.805383+0000","last_became_active":"2023-11-27T15:23:31.647087+0000","last_became_peered":"2023-11-27T15:23:31.647087+0000","last_unstale":"2023-11-27T15:23:32.805383+0000","last_undegraded":"2023-11-27T15:23:32.805383+0000","last_fullsized":"2023-11-27T15:23:32.805383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:15:38.922135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.251072+0000","last_change":"2023-11-27T15:23:26.603373+0000","last_active":"2023-11-27T15:23:58.251072+0000","last_peered":"2023-11-27T15:23:58.251072+0000","last_clean":"2023-11-27T15:23:58.251072+0000","last_became_active":"2023-11-27T15:23:26.603049+0000","last_became_peered":"2023-11-27T15:23:26.603049+0000","last_unstale":"2023-11-27T15:23:58.251072+0000","last_undegraded":"2023-11-27T15:23:58.251072+0000","last_fullsized":"2023-11-27T15:23:58.251072+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:06:36.526595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387805+0000","last_change":"2023-11-27T15:23:27.186270+0000","last_active":"2023-11-27T15:23:33.387805+0000","last_peered":"2023-11-27T15:23:33.387805+0000","last_clean":"2023-11-27T15:23:33.387805+0000","last_became_active":"2023-11-27T15:23:27.185952+0000","last_became_peered":"2023-11-27T15:23:27.185952+0000","last_unstale":"2023-11-27T15:23:33.387805+0000","last_undegraded":"2023-11-27T15:23:33.387805+0000","last_fullsized":"2023-11-27T15:23:33.387805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:25:26.018331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805925+0000","last_change":"2023-11-27T15:23:31.648666+0000","last_active":"2023-11-27T15:23:32.805925+0000","last_peered":"2023-11-27T15:23:32.805925+0000","last_clean":"2023-11-27T15:23:32.805925+0000","last_became_active":"2023-11-27T15:23:31.647245+0000","last_became_peered":"2023-11-27T15:23:31.647245+0000","last_unstale":"2023-11-27T15:23:32.805925+0000","last_undegraded":"2023-11-27T15:23:32.805925+0000","last_fullsized":"2023-11-27T15:23:32.805925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:28:31.010202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807088+0000","last_change":"2023-11-27T15:23:26.606886+0000","last_active":"2023-11-27T15:23:32.807088+0000","last_peered":"2023-11-27T15:23:32.807088+0000","last_clean":"2023-11-27T15:23:32.807088+0000","last_became_active":"2023-11-27T15:23:26.606556+0000","last_became_peered":"2023-11-27T15:23:26.606556+0000","last_unstale":"2023-11-27T15:23:32.807088+0000","last_undegraded":"2023-11-27T15:23:32.807088+0000","last_fullsized":"2023-11-27T15:23:32.807088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:44:25.371619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387474+0000","last_change":"2023-11-27T15:23:27.194532+0000","last_active":"2023-11-27T15:23:33.387474+0000","last_peered":"2023-11-27T15:23:33.387474+0000","last_clean":"2023-11-27T15:23:33.387474+0000","last_became_active":"2023-11-27T15:23:27.194202+0000","last_became_peered":"2023-11-27T15:23:27.194202+0000","last_unstale":"2023-11-27T15:23:33.387474+0000","last_undegraded":"2023-11-27T15:23:33.387474+0000","last_fullsized":"2023-11-27T15:23:33.387474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:48:31.124655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806037+0000","last_change":"2023-11-27T15:23:31.638466+0000","last_active":"2023-11-27T15:23:32.806037+0000","last_peered":"2023-11-27T15:23:32.806037+0000","last_clean":"2023-11-27T15:23:32.806037+0000","last_became_active":"2023-11-27T15:23:31.638103+0000","last_became_peered":"2023-11-27T15:23:31.638103+0000","last_unstale":"2023-11-27T15:23:32.806037+0000","last_undegraded":"2023-11-27T15:23:32.806037+0000","last_fullsized":"2023-11-27T15:23:32.806037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:46:01.956128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805894+0000","last_change":"2023-11-27T15:23:26.613616+0000","last_active":"2023-11-27T15:23:32.805894+0000","last_peered":"2023-11-27T15:23:32.805894+0000","last_clean":"2023-11-27T15:23:32.805894+0000","last_became_active":"2023-11-27T15:23:26.608240+0000","last_became_peered":"2023-11-27T15:23:26.608240+0000","last_unstale":"2023-11-27T15:23:32.805894+0000","last_undegraded":"2023-11-27T15:23:32.805894+0000","last_fullsized":"2023-11-27T15:23:32.805894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:15:18.962601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387750+0000","last_change":"2023-11-27T15:23:27.185677+0000","last_active":"2023-11-27T15:23:33.387750+0000","last_peered":"2023-11-27T15:23:33.387750+0000","last_clean":"2023-11-27T15:23:33.387750+0000","last_became_active":"2023-11-27T15:23:27.185321+0000","last_became_peered":"2023-11-27T15:23:27.185321+0000","last_unstale":"2023-11-27T15:23:33.387750+0000","last_undegraded":"2023-11-27T15:23:33.387750+0000","last_fullsized":"2023-11-27T15:23:33.387750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:41:04.564578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151397+0000","last_change":"2023-11-27T15:23:27.195847+0000","last_active":"2023-11-27T15:23:32.151397+0000","last_peered":"2023-11-27T15:23:32.151397+0000","last_clean":"2023-11-27T15:23:32.151397+0000","last_became_active":"2023-11-27T15:23:27.195421+0000","last_became_peered":"2023-11-27T15:23:27.195421+0000","last_unstale":"2023-11-27T15:23:32.151397+0000","last_undegraded":"2023-11-27T15:23:32.151397+0000","last_fullsized":"2023-11-27T15:23:32.151397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:43:55.522227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806546+0000","last_change":"2023-11-27T15:23:27.181352+0000","last_active":"2023-11-27T15:23:32.806546+0000","last_peered":"2023-11-27T15:23:32.806546+0000","last_clean":"2023-11-27T15:23:32.806546+0000","last_became_active":"2023-11-27T15:23:27.181042+0000","last_became_peered":"2023-11-27T15:23:27.181042+0000","last_unstale":"2023-11-27T15:23:32.806546+0000","last_undegraded":"2023-11-27T15:23:32.806546+0000","last_fullsized":"2023-11-27T15:23:32.806546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:59:34.485652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805780+0000","last_change":"2023-11-27T15:23:27.199915+0000","last_active":"2023-11-27T15:23:32.805780+0000","last_peered":"2023-11-27T15:23:32.805780+0000","last_clean":"2023-11-27T15:23:32.805780+0000","last_became_active":"2023-11-27T15:23:27.199595+0000","last_became_peered":"2023-11-27T15:23:27.199595+0000","last_unstale":"2023-11-27T15:23:32.805780+0000","last_undegraded":"2023-11-27T15:23:32.805780+0000","last_fullsized":"2023-11-27T15:23:32.805780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:38:25.307144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806283+0000","last_change":"2023-11-27T15:23:27.210258+0000","last_active":"2023-11-27T15:23:32.806283+0000","last_peered":"2023-11-27T15:23:32.806283+0000","last_clean":"2023-11-27T15:23:32.806283+0000","last_became_active":"2023-11-27T15:23:27.208028+0000","last_became_peered":"2023-11-27T15:23:27.208028+0000","last_unstale":"2023-11-27T15:23:32.806283+0000","last_undegraded":"2023-11-27T15:23:32.806283+0000","last_fullsized":"2023-11-27T15:23:32.806283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:23:01.035064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806936+0000","last_change":"2023-11-27T15:23:27.211759+0000","last_active":"2023-11-27T15:23:32.806936+0000","last_peered":"2023-11-27T15:23:32.806936+0000","last_clean":"2023-11-27T15:23:32.806936+0000","last_became_active":"2023-11-27T15:23:27.211445+0000","last_became_peered":"2023-11-27T15:23:27.211445+0000","last_unstale":"2023-11-27T15:23:32.806936+0000","last_undegraded":"2023-11-27T15:23:32.806936+0000","last_fullsized":"2023-11-27T15:23:32.806936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:42:20.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151808+0000","last_change":"2023-11-27T15:23:27.185855+0000","last_active":"2023-11-27T15:23:32.151808+0000","last_peered":"2023-11-27T15:23:32.151808+0000","last_clean":"2023-11-27T15:23:32.151808+0000","last_became_active":"2023-11-27T15:23:27.185535+0000","last_became_peered":"2023-11-27T15:23:27.185535+0000","last_unstale":"2023-11-27T15:23:32.151808+0000","last_undegraded":"2023-11-27T15:23:32.151808+0000","last_fullsized":"2023-11-27T15:23:32.151808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:46:29.055191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806865+0000","last_change":"2023-11-27T15:23:27.190163+0000","last_active":"2023-11-27T15:23:32.806865+0000","last_peered":"2023-11-27T15:23:32.806865+0000","last_clean":"2023-11-27T15:23:32.806865+0000","last_became_active":"2023-11-27T15:23:27.188729+0000","last_became_peered":"2023-11-27T15:23:27.188729+0000","last_unstale":"2023-11-27T15:23:32.806865+0000","last_undegraded":"2023-11-27T15:23:32.806865+0000","last_fullsized":"2023-11-27T15:23:32.806865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:30:20.263241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445896+0000","last_change":"2023-11-27T15:23:27.203904+0000","last_active":"2023-11-27T15:23:32.445896+0000","last_peered":"2023-11-27T15:23:32.445896+0000","last_clean":"2023-11-27T15:23:32.445896+0000","last_became_active":"2023-11-27T15:23:27.203578+0000","last_became_peered":"2023-11-27T15:23:27.203578+0000","last_unstale":"2023-11-27T15:23:32.445896+0000","last_undegraded":"2023-11-27T15:23:32.445896+0000","last_fullsized":"2023-11-27T15:23:32.445896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:49:47.755682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806592+0000","last_change":"2023-11-27T15:23:27.206791+0000","last_active":"2023-11-27T15:23:32.806592+0000","last_peered":"2023-11-27T15:23:32.806592+0000","last_clean":"2023-11-27T15:23:32.806592+0000","last_became_active":"2023-11-27T15:23:27.206529+0000","last_became_peered":"2023-11-27T15:23:27.206529+0000","last_unstale":"2023-11-27T15:23:32.806592+0000","last_undegraded":"2023-11-27T15:23:32.806592+0000","last_fullsized":"2023-11-27T15:23:32.806592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:39:29.543511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445442+0000","last_change":"2023-11-27T15:23:27.197562+0000","last_active":"2023-11-27T15:23:32.445442+0000","last_peered":"2023-11-27T15:23:32.445442+0000","last_clean":"2023-11-27T15:23:32.445442+0000","last_became_active":"2023-11-27T15:23:27.197233+0000","last_became_peered":"2023-11-27T15:23:27.197233+0000","last_unstale":"2023-11-27T15:23:32.445442+0000","last_undegraded":"2023-11-27T15:23:32.445442+0000","last_fullsized":"2023-11-27T15:23:32.445442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:21:31.977127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805566+0000","last_change":"2023-11-27T15:23:27.191281+0000","last_active":"2023-11-27T15:23:32.805566+0000","last_peered":"2023-11-27T15:23:32.805566+0000","last_clean":"2023-11-27T15:23:32.805566+0000","last_became_active":"2023-11-27T15:23:27.190973+0000","last_became_peered":"2023-11-27T15:23:27.190973+0000","last_unstale":"2023-11-27T15:23:32.805566+0000","last_undegraded":"2023-11-27T15:23:32.805566+0000","last_fullsized":"2023-11-27T15:23:32.805566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:10:22.602292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806751+0000","last_change":"2023-11-27T15:23:27.199251+0000","last_active":"2023-11-27T15:23:32.806751+0000","last_peered":"2023-11-27T15:23:32.806751+0000","last_clean":"2023-11-27T15:23:32.806751+0000","last_became_active":"2023-11-27T15:23:27.198926+0000","last_became_peered":"2023-11-27T15:23:27.198926+0000","last_unstale":"2023-11-27T15:23:32.806751+0000","last_undegraded":"2023-11-27T15:23:32.806751+0000","last_fullsized":"2023-11-27T15:23:32.806751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:18:22.223705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151530+0000","last_change":"2023-11-27T15:23:27.192227+0000","last_active":"2023-11-27T15:23:32.151530+0000","last_peered":"2023-11-27T15:23:32.151530+0000","last_clean":"2023-11-27T15:23:32.151530+0000","last_became_active":"2023-11-27T15:23:27.191852+0000","last_became_peered":"2023-11-27T15:23:27.191852+0000","last_unstale":"2023-11-27T15:23:32.151530+0000","last_undegraded":"2023-11-27T15:23:32.151530+0000","last_fullsized":"2023-11-27T15:23:32.151530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:01:52.455108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":28672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":25469,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":137,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":76276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6072,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739080,"statfs":{"total":95995035648,"available":95988817920,"internally_reserved":0,"allocated":966656,"data_stored":467705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49399999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52700000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.629}]},{"osd":5,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.66400000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.73599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.82899999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.85099999999999998}]}]},{"osd":10,"up_from":72,"seq":309237645336,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6964,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738188,"statfs":{"total":95995035648,"available":95987904512,"internally_reserved":0,"allocated":1617920,"data_stored":1080255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.63800000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.73099999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.93400000000000005}]},{"osd":4,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68999999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1259999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65500000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.95099999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.0069999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:23:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.175}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6284,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738868,"statfs":{"total":95995035648,"available":95988600832,"internally_reserved":0,"allocated":987136,"data_stored":472037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.56100000000000005,"15min":0.57599999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64900000000000002,"5min":1.054,"15min":1.054},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45100000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68000000000000005,"15min":0.69099999999999995},"min":{"1min":0.44400000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.0029999999999999,"5min":1.046,"15min":1.046},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.044,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.751}]},{"osd":2,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.079,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.505},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.53100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97199999999999998,"5min":1.294,"15min":1.294},"last":0.52300000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.73699999999999999,"15min":0.755},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.3520000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.58999999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.82399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.55800000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.88800000000000001,"15min":0.90400000000000003},"min":{"1min":0.64800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3129999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.85899999999999999},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Mon Nov 27 15:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.84299999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.94799999999999995,"15min":0.97199999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.429,"5min":2.069,"15min":2.069},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.95599999999999996,"15min":0.98399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.702,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.72499999999999998}]},{"osd":8,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.63900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1990000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.872,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.478,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.629}]},{"osd":10,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6404,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738748,"statfs":{"total":95995035648,"available":95988477952,"internally_reserved":0,"allocated":978944,"data_stored":467530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80100000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56000000000000005}]},{"osd":1,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.623,"15min":0.623},"min":{"1min":0.34799999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.873,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.85999999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48499999999999999,"15min":0.48599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70499999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59199999999999997}]},{"osd":3,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95699999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":3.3940000000000001,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.81599999999999995}]},{"osd":4,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.40799999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.91700000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.38100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88600000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73399999999999999,"15min":0.73699999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81999999999999995,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.88200000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.996},{"interface":"front","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0429999999999999,"5min":1.069,"15min":1.069},"last":0.93799999999999994}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.73699999999999999,"15min":0.72199999999999998},"min":{"1min":0.63200000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.59399999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.117,"5min":1.119,"15min":1.119},"last":0.83399999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.90800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7120,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738032,"statfs":{"total":95995035648,"available":95987744768,"internally_reserved":0,"allocated":1581056,"data_stored":1068159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.35799999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.26400000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.48099999999999998,"15min":0.47799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.50700000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89900000000000002,"5min":1.03,"15min":1.03},"last":0.68899999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.64200000000000002,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.378,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84899999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53700000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93100000000000005,"5min":1.018,"15min":1.018},"last":0.72299999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.74199999999999999,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.81799999999999995,"15min":0.78300000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":6.1349999999999998,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":0.65300000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:23:23 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.79500000000000004,"15min":0.76400000000000001},"min":{"1min":0.505,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":6.1050000000000004,"5min":6.1050000000000004,"15min":6.1050000000000004},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.81399999999999995,"15min":0.78400000000000003},"min":{"1min":0.53800000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":6.0330000000000004,"5min":6.0330000000000004,"15min":6.0330000000000004},"last":0.82999999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.54600000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98299999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.82299999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6392,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738760,"statfs":{"total":95995035648,"available":95988490240,"internally_reserved":0,"allocated":966656,"data_stored":467213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.504,"15min":0.49199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.57699999999999996}]},{"osd":1,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.752},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.42699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69899999999999995,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0149999999999999,"5min":1.101,"15min":1.101},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.77700000000000002,"15min":0.75700000000000001},"min":{"1min":0.59399999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":0.69999999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65800000000000003,"15min":0.65200000000000002},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.62},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81599999999999995,"15min":0.80500000000000005},"min":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75900000000000001,"15min":0.748},"min":{"1min":0.53200000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.66400000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78500000000000003,"15min":0.78200000000000003},"min":{"1min":0.45600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.093,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1080000000000001,"5min":1.581,"15min":1.581},"last":0.874}]},{"osd":7,"last update":"Mon Nov 27 15:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82199999999999995,"15min":0.81699999999999995},"min":{"1min":0.51100000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.159,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.53000000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.107,"5min":1.175,"15min":1.175},"last":0.83599999999999997}]},{"osd":8,"last update":"Mon Nov 27 15:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85499999999999998,"15min":0.85899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.874,"15min":0.88},"min":{"1min":0.63600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1299999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.91400000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.57499999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":0.93799999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.85699999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.90000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7360,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737792,"statfs":{"total":95995035648,"available":95987499008,"internally_reserved":0,"allocated":974848,"data_stored":470917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.94799999999999995,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.54600000000000004},"min":{"1min":0.41099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.89400000000000002,"5min":6.2839999999999998,"15min":6.2839999999999998},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.54700000000000004,"15min":0.51800000000000002},"min":{"1min":0.38700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85999999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.56100000000000005,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72199999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.441},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71699999999999997,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.8100000000000001,"5min":6.3179999999999996,"15min":6.3179999999999996},"last":0.72299999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46899999999999997,"15min":0.42099999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.5,"15min":0.45800000000000002},"min":{"1min":0.308,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.41999999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65300000000000002,"15min":0.623},"min":{"1min":0.46400000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93000000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.66000000000000003,"15min":0.625},"min":{"1min":0.52000000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.80400000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72999999999999998,"15min":0.72799999999999998},"min":{"1min":0.498,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0289999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.53100000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91400000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Nov 27 15:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.377,"5min":0.316,"15min":0.316},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.70399999999999996,"15min":0.68500000000000005},"min":{"1min":0.42499999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.82699999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.51900000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59999999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.86699999999999999,"15min":0.879},"min":{"1min":0.51600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1020000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.91500000000000004,"15min":0.92500000000000004},"min":{"1min":0.59599999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.876}]},{"osd":10,"last update":"Mon Nov 27 15:23:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.86699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7372,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737780,"statfs":{"total":95995035648,"available":95987486720,"internally_reserved":0,"allocated":987136,"data_stored":468270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.54500000000000004},"min":{"1min":0.439,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93400000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.504},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59699999999999998,"15min":0.55300000000000005},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.95699999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53700000000000003}]},{"osd":2,"last update":"Mon Nov 27 15:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.503,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59899999999999998,"15min":0.57599999999999996},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66600000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.379,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.59299999999999997,"15min":0.54400000000000004},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.76400000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67400000000000004,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92200000000000004,"5min":0.998,"15min":0.998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.98299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.89300000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.45900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93999999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":1.115,"15min":1.115},"last":0.82799999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.76600000000000001,"15min":0.751},"min":{"1min":0.59299999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0720000000000001,"5min":1.147,"15min":1.147},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.76900000000000002,"15min":0.753},"min":{"1min":0.61299999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.107,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.85999999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:23:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77800000000000002},"min":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.874},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.079,"5min":1.22,"15min":1.22},"last":0.93500000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.89600000000000002,"15min":0.91100000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":0.874},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.032,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.92400000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.98399999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:23:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":1.0229999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7480,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737672,"statfs":{"total":95995035648,"available":95987376128,"internally_reserved":0,"allocated":1032192,"data_stored":507057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.85399999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.34999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79500000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58699999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54800000000000004,"15min":0.502},"min":{"1min":0.439,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1479999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.55300000000000005,"15min":0.501},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0700000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.74399999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72299999999999998,"15min":0.82899999999999996},"min":{"1min":0.38600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.874,"5min":7.1189999999999998,"15min":7.1189999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.91300000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.36699999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.42599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77500000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.44800000000000001},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.62},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.47999999999999998},"min":{"1min":0.30499999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.754,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72099999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:23:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65600000000000003,"15min":0.63200000000000001},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.96199999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.53500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.90000000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":1.044},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.82299999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":1.0189999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6652,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738500,"statfs":{"total":95995035648,"available":95988224000,"internally_reserved":0,"allocated":970752,"data_stored":467227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.41799999999999998},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67500000000000004,"5min":1.159,"15min":1.159},"last":0.437},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.34000000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67600000000000005,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.40600000000000003}]},{"osd":1,"last update":"Mon Nov 27 15:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84799999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64200000000000002,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81499999999999995,"5min":1.115,"15min":1.115},"last":0.77300000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89200000000000002,"5min":1.143,"15min":1.143},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65500000000000003,"15min":0.63500000000000001},"min":{"1min":0.46700000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85499999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.48899999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.63400000000000001,"15min":0.60799999999999998},"min":{"1min":0.54700000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.032,"5min":1.034,"15min":1.034},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.74399999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:23:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69299999999999995,"15min":0.67500000000000004},"min":{"1min":0.53600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0720000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.52600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97099999999999997,"5min":1.306,"15min":1.306},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88,"5min":1.284,"15min":1.284},"last":0.998}]},{"osd":7,"last update":"Mon Nov 27 15:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.81899999999999995,"15min":0.80700000000000005},"min":{"1min":0.622,"5min":0.442,"15min":0.442},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.64800000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":1.0209999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:23:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.81399999999999995},"min":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.84699999999999998,"15min":0.84999999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.80500000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.86499999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:23:14 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.92900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":970752,"data_stored":467778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.58399999999999996}]},{"osd":1,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.375},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.70899999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.436,"15min":0.39600000000000002},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72199999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61099999999999999,"15min":0.59599999999999997},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.62,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":0.85499999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.58699999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":1.329,"15min":1.329},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63200000000000001,"15min":0.59999999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92700000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.93899999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.46800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97599999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.84699999999999998,"5min":1.288,"15min":1.288},"last":0.83599999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.88600000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97799999999999998,"5min":1.038,"15min":1.038},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.48399999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.05,"5min":1.107,"15min":1.107},"last":0.91300000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1599999999999999,"5min":1.222,"15min":1.222},"last":0.998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.84099999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.96899999999999997,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.87}]},{"osd":10,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.92700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724591,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7256,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737896,"statfs":{"total":95995035648,"available":95987605504,"internally_reserved":0,"allocated":1589248,"data_stored":1063405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.61699999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84799999999999998,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85799999999999998,"5min":0.873,"15min":0.873},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.42999999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84599999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.52200000000000002},"min":{"1min":0.44700000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75900000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.39800000000000002},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.437,"15min":0.42399999999999999},"min":{"1min":0.33800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.35799999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.41099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.875,"15min":0.875},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82699999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89600000000000002,"5min":2.097,"15min":2.097},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58899999999999997,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97099999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.68300000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90100000000000002,"5min":1.004,"15min":1.004},"last":0.62},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91500000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.66000000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.48499999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71399999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:22:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69799999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":40207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":23095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:24:02.709 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:24:02.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:02 smithi060 ceph-mon[131518]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:02.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:02 smithi060 ceph-mon[131518]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:02.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:02 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/876045884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:02.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:02 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2678733243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:02.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:02 smithi029 ceph-mon[131166]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:02.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:02 smithi029 ceph-mon[131166]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:24:02.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:02 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/876045884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:02.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:02 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2678733243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:02.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.942+0000 7efda57ff700 1 -- 172.21.15.8:0/1500196808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda0073550 msgr2=0x7efda00739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.942+0000 7efda57ff700 1 --2- 172.21.15.8:0/1500196808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda0073550 0x7efda00739c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd88009990 tx=0x7efd88009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.943+0000 7efda57ff700 1 -- 172.21.15.8:0/1500196808 shutdown_connections 2023-11-27T15:24:02.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.943+0000 7efda57ff700 1 --2- 172.21.15.8:0/1500196808 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0074b30 0x7efda0072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.943+0000 7efda57ff700 1 --2- 172.21.15.8:0/1500196808 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda0108190 0x7efda010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.943+0000 7efda57ff700 1 --2- 172.21.15.8:0/1500196808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda0073550 0x7efda00739c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.943+0000 7efda57ff700 1 -- 172.21.15.8:0/1500196808 >> 172.21.15.8:0/1500196808 conn(0x7efda00fb8b0 msgr2=0x7efda00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.943+0000 7efda57ff700 1 -- 172.21.15.8:0/1500196808 shutdown_connections 2023-11-27T15:24:02.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.943+0000 7efda57ff700 1 -- 172.21.15.8:0/1500196808 wait complete. 2023-11-27T15:24:02.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.944+0000 7efda57ff700 1 Processor -- start 2023-11-27T15:24:02.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.944+0000 7efda57ff700 1 -- start start 2023-11-27T15:24:02.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.944+0000 7efda57ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0073550 0x7efda0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.944+0000 7efda57ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda0074b30 0x7efda01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.944+0000 7efda57ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda0108190 0x7efda019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.944+0000 7efda57ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda019fe10 con 0x7efda0074b30 2023-11-27T15:24:02.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.944+0000 7efda57ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda019ff50 con 0x7efda0108190 2023-11-27T15:24:02.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.944+0000 7efda57ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda01a00e0 con 0x7efda0073550 2023-11-27T15:24:02.950 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.944+0000 7efd97fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda0074b30 0x7efda01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd9effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0073550 0x7efda0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd97fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda0074b30 0x7efda01987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53598/0 (socket says 172.21.15.8:53598) 2023-11-27T15:24:02.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd97fff700 1 -- 172.21.15.8:0/4061116806 learned_addr learned my addr 172.21.15.8:0/4061116806 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:02.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd9f7fe700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda0108190 0x7efda019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd97fff700 1 -- 172.21.15.8:0/4061116806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0073550 msgr2=0x7efda0198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd97fff700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0073550 0x7efda0198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd97fff700 1 -- 172.21.15.8:0/4061116806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda0108190 msgr2=0x7efda019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd97fff700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda0108190 0x7efda019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd97fff700 1 -- 172.21.15.8:0/4061116806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880095f0 con 0x7efda0074b30 2023-11-27T15:24:02.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd9f7fe700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda0108190 0x7efda019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:02.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd97fff700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda0074b30 0x7efda01987d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efd88004710 tx=0x7efd88009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd9cff9700 1 -- 172.21.15.8:0/4061116806 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8801d070 con 0x7efda0074b30 2023-11-27T15:24:02.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd9cff9700 1 -- 172.21.15.8:0/4061116806 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd88021410 con 0x7efda0074b30 2023-11-27T15:24:02.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efda57ff700 1 -- 172.21.15.8:0/4061116806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda010deb0 con 0x7efda0074b30 2023-11-27T15:24:02.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.945+0000 7efd9cff9700 1 -- 172.21.15.8:0/4061116806 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88017650 con 0x7efda0074b30 2023-11-27T15:24:02.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.946+0000 7efda57ff700 1 -- 172.21.15.8:0/4061116806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda010dd20 con 0x7efda0074b30 2023-11-27T15:24:02.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.947+0000 7efda57ff700 1 -- 172.21.15.8:0/4061116806 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda0106840 con 0x7efda0074b30 2023-11-27T15:24:02.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.948+0000 7efd9cff9700 1 -- 172.21.15.8:0/4061116806 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7efd88021580 con 0x7efda0074b30 2023-11-27T15:24:02.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.950+0000 7efd9cff9700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efd8c0a11e0 0x7efd8c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.950+0000 7efd9cff9700 1 -- 172.21.15.8:0/4061116806 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efd880ce170 con 0x7efda0074b30 2023-11-27T15:24:02.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.950+0000 7efd9effd700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efd8c0a11e0 0x7efd8c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.951+0000 7efd9effd700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efd8c0a11e0 0x7efd8c0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efda0074f00 tx=0x7efd90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:02.954+0000 7efd9cff9700 1 -- 172.21.15.8:0/4061116806 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efd880975a0 con 0x7efda0074b30 2023-11-27T15:24:03.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.161+0000 7efda57ff700 1 -- 172.21.15.8:0/4061116806 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efda0061980 con 0x7efd8c0a11e0 2023-11-27T15:24:03.178 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.175+0000 7efd9cff9700 1 -- 172.21.15.8:0/4061116806 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422357 (secure 0 0 0) 0x7efda0061980 con 0x7efd8c0a11e0 2023-11-27T15:24:03.186 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:03.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 -- 172.21.15.8:0/4061116806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efd8c0a11e0 msgr2=0x7efd8c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:03.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efd8c0a11e0 0x7efd8c0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efda0074f00 tx=0x7efd90009260 comp rx=0 tx=0).stop 2023-11-27T15:24:03.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 -- 172.21.15.8:0/4061116806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda0074b30 msgr2=0x7efda01987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:03.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda0074b30 0x7efda01987d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efd88004710 tx=0x7efd88009580 comp rx=0 tx=0).stop 2023-11-27T15:24:03.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 -- 172.21.15.8:0/4061116806 shutdown_connections 2023-11-27T15:24:03.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7efd8c0a11e0 0x7efd8c0a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:03.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7efda0073550 0x7efda0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:03.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7efda0108190 0x7efda019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:03.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 --2- 172.21.15.8:0/4061116806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efda0074b30 0x7efda01987d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:03.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 -- 172.21.15.8:0/4061116806 >> 172.21.15.8:0/4061116806 conn(0x7efda00fb8b0 msgr2=0x7efda00fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:03.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 -- 172.21.15.8:0/4061116806 shutdown_connections 2023-11-27T15:24:03.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:03.180+0000 7efda57ff700 1 -- 172.21.15.8:0/4061116806 wait complete. 2023-11-27T15:24:03.193 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-11-27T15:24:03.203 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-11-27T15:24:02.334942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616083,"num_objects":63,"num_object_clones":0,"num_object_copies":191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":158,"num_read_kb":137,"num_write":320,"num_write_kb":3005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":259,"ondisk_log_size":259,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":81880,"kb_used_data":13304,"kb_used_omap":0,"kb_used_meta":68480,"kb_avail":1124859944,"statfs":{"total":1151940427776,"available":1151856582656,"internally_reserved":0,"allocated":13623296,"data_stored":7467553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70123520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21123,"num_objects":39,"num_object_clones":0,"num_object_copies":119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":90,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003826"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.152009+0000","last_change":"2023-11-27T15:23:27.211699+0000","last_active":"2023-11-27T15:23:32.152009+0000","last_peered":"2023-11-27T15:23:32.152009+0000","last_clean":"2023-11-27T15:23:32.152009+0000","last_became_active":"2023-11-27T15:23:27.211372+0000","last_became_peered":"2023-11-27T15:23:27.211372+0000","last_unstale":"2023-11-27T15:23:32.152009+0000","last_undegraded":"2023-11-27T15:23:32.152009+0000","last_fullsized":"2023-11-27T15:23:32.152009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:36:00.909796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807370+0000","last_change":"2023-11-27T15:23:27.210567+0000","last_active":"2023-11-27T15:23:32.807370+0000","last_peered":"2023-11-27T15:23:32.807370+0000","last_clean":"2023-11-27T15:23:32.807370+0000","last_became_active":"2023-11-27T15:23:27.208340+0000","last_became_peered":"2023-11-27T15:23:27.208340+0000","last_unstale":"2023-11-27T15:23:32.807370+0000","last_undegraded":"2023-11-27T15:23:32.807370+0000","last_fullsized":"2023-11-27T15:23:32.807370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:10:44.168450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806815+0000","last_change":"2023-11-27T15:23:27.196040+0000","last_active":"2023-11-27T15:23:32.806815+0000","last_peered":"2023-11-27T15:23:32.806815+0000","last_clean":"2023-11-27T15:23:32.806815+0000","last_became_active":"2023-11-27T15:23:27.195195+0000","last_became_peered":"2023-11-27T15:23:27.195195+0000","last_unstale":"2023-11-27T15:23:32.806815+0000","last_undegraded":"2023-11-27T15:23:32.806815+0000","last_fullsized":"2023-11-27T15:23:32.806815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:57:28.442265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240980+0000","last_change":"2023-11-27T15:23:27.190096+0000","last_active":"2023-11-27T15:23:32.240980+0000","last_peered":"2023-11-27T15:23:32.240980+0000","last_clean":"2023-11-27T15:23:32.240980+0000","last_became_active":"2023-11-27T15:23:27.189768+0000","last_became_peered":"2023-11-27T15:23:27.189768+0000","last_unstale":"2023-11-27T15:23:32.240980+0000","last_undegraded":"2023-11-27T15:23:32.240980+0000","last_fullsized":"2023-11-27T15:23:32.240980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:16:12.429083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805525+0000","last_change":"2023-11-27T15:23:27.200974+0000","last_active":"2023-11-27T15:23:32.805525+0000","last_peered":"2023-11-27T15:23:32.805525+0000","last_clean":"2023-11-27T15:23:32.805525+0000","last_became_active":"2023-11-27T15:23:27.200647+0000","last_became_peered":"2023-11-27T15:23:27.200647+0000","last_unstale":"2023-11-27T15:23:32.805525+0000","last_undegraded":"2023-11-27T15:23:32.805525+0000","last_fullsized":"2023-11-27T15:23:32.805525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:53:56.304270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151519+0000","last_change":"2023-11-27T15:23:27.188417+0000","last_active":"2023-11-27T15:23:32.151519+0000","last_peered":"2023-11-27T15:23:32.151519+0000","last_clean":"2023-11-27T15:23:32.151519+0000","last_became_active":"2023-11-27T15:23:27.188079+0000","last_became_peered":"2023-11-27T15:23:27.188079+0000","last_unstale":"2023-11-27T15:23:32.151519+0000","last_undegraded":"2023-11-27T15:23:32.151519+0000","last_fullsized":"2023-11-27T15:23:32.151519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:53:23.107972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387140+0000","last_change":"2023-11-27T15:23:27.192410+0000","last_active":"2023-11-27T15:23:33.387140+0000","last_peered":"2023-11-27T15:23:33.387140+0000","last_clean":"2023-11-27T15:23:33.387140+0000","last_became_active":"2023-11-27T15:23:27.192026+0000","last_became_peered":"2023-11-27T15:23:27.192026+0000","last_unstale":"2023-11-27T15:23:33.387140+0000","last_undegraded":"2023-11-27T15:23:33.387140+0000","last_fullsized":"2023-11-27T15:23:33.387140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:35:50.652363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445258+0000","last_change":"2023-11-27T15:23:27.205778+0000","last_active":"2023-11-27T15:23:32.445258+0000","last_peered":"2023-11-27T15:23:32.445258+0000","last_clean":"2023-11-27T15:23:32.445258+0000","last_became_active":"2023-11-27T15:23:27.205467+0000","last_became_peered":"2023-11-27T15:23:27.205467+0000","last_unstale":"2023-11-27T15:23:32.445258+0000","last_undegraded":"2023-11-27T15:23:32.445258+0000","last_fullsized":"2023-11-27T15:23:32.445258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:35:33.177488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805937+0000","last_change":"2023-11-27T15:23:27.187920+0000","last_active":"2023-11-27T15:23:32.805937+0000","last_peered":"2023-11-27T15:23:32.805937+0000","last_clean":"2023-11-27T15:23:32.805937+0000","last_became_active":"2023-11-27T15:23:27.186989+0000","last_became_peered":"2023-11-27T15:23:27.186989+0000","last_unstale":"2023-11-27T15:23:32.805937+0000","last_undegraded":"2023-11-27T15:23:32.805937+0000","last_fullsized":"2023-11-27T15:23:32.805937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:17:57.515405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.152137+0000","last_change":"2023-11-27T15:23:27.181330+0000","last_active":"2023-11-27T15:23:32.152137+0000","last_peered":"2023-11-27T15:23:32.152137+0000","last_clean":"2023-11-27T15:23:32.152137+0000","last_became_active":"2023-11-27T15:23:27.180986+0000","last_became_peered":"2023-11-27T15:23:27.180986+0000","last_unstale":"2023-11-27T15:23:32.152137+0000","last_undegraded":"2023-11-27T15:23:32.152137+0000","last_fullsized":"2023-11-27T15:23:32.152137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:32:05.556276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387781+0000","last_change":"2023-11-27T15:23:27.190673+0000","last_active":"2023-11-27T15:23:33.387781+0000","last_peered":"2023-11-27T15:23:33.387781+0000","last_clean":"2023-11-27T15:23:33.387781+0000","last_became_active":"2023-11-27T15:23:27.186256+0000","last_became_peered":"2023-11-27T15:23:27.186256+0000","last_unstale":"2023-11-27T15:23:33.387781+0000","last_undegraded":"2023-11-27T15:23:33.387781+0000","last_fullsized":"2023-11-27T15:23:33.387781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:59:37.848691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.241078+0000","last_change":"2023-11-27T15:23:27.185077+0000","last_active":"2023-11-27T15:23:32.241078+0000","last_peered":"2023-11-27T15:23:32.241078+0000","last_clean":"2023-11-27T15:23:32.241078+0000","last_became_active":"2023-11-27T15:23:27.184227+0000","last_became_peered":"2023-11-27T15:23:27.184227+0000","last_unstale":"2023-11-27T15:23:32.241078+0000","last_undegraded":"2023-11-27T15:23:32.241078+0000","last_fullsized":"2023-11-27T15:23:32.241078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:51.267197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806664+0000","last_change":"2023-11-27T15:23:27.190146+0000","last_active":"2023-11-27T15:23:32.806664+0000","last_peered":"2023-11-27T15:23:32.806664+0000","last_clean":"2023-11-27T15:23:32.806664+0000","last_became_active":"2023-11-27T15:23:27.189133+0000","last_became_peered":"2023-11-27T15:23:27.189133+0000","last_unstale":"2023-11-27T15:23:32.806664+0000","last_undegraded":"2023-11-27T15:23:32.806664+0000","last_fullsized":"2023-11-27T15:23:32.806664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:08:16.740298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806059+0000","last_change":"2023-11-27T15:23:27.207461+0000","last_active":"2023-11-27T15:23:32.806059+0000","last_peered":"2023-11-27T15:23:32.806059+0000","last_clean":"2023-11-27T15:23:32.806059+0000","last_became_active":"2023-11-27T15:23:27.206837+0000","last_became_peered":"2023-11-27T15:23:27.206837+0000","last_unstale":"2023-11-27T15:23:32.806059+0000","last_undegraded":"2023-11-27T15:23:32.806059+0000","last_fullsized":"2023-11-27T15:23:32.806059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:58:03.194278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805991+0000","last_change":"2023-11-27T15:23:27.211206+0000","last_active":"2023-11-27T15:23:32.805991+0000","last_peered":"2023-11-27T15:23:32.805991+0000","last_clean":"2023-11-27T15:23:32.805991+0000","last_became_active":"2023-11-27T15:23:27.209933+0000","last_became_peered":"2023-11-27T15:23:27.209933+0000","last_unstale":"2023-11-27T15:23:32.805991+0000","last_undegraded":"2023-11-27T15:23:32.805991+0000","last_fullsized":"2023-11-27T15:23:32.805991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:23:51.245058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805184+0000","last_change":"2023-11-27T15:23:27.188863+0000","last_active":"2023-11-27T15:23:32.805184+0000","last_peered":"2023-11-27T15:23:32.805184+0000","last_clean":"2023-11-27T15:23:32.805184+0000","last_became_active":"2023-11-27T15:23:27.188534+0000","last_became_peered":"2023-11-27T15:23:27.188534+0000","last_unstale":"2023-11-27T15:23:32.805184+0000","last_undegraded":"2023-11-27T15:23:32.805184+0000","last_fullsized":"2023-11-27T15:23:32.805184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:27:47.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151901+0000","last_change":"2023-11-27T15:23:27.211786+0000","last_active":"2023-11-27T15:23:32.151901+0000","last_peered":"2023-11-27T15:23:32.151901+0000","last_clean":"2023-11-27T15:23:32.151901+0000","last_became_active":"2023-11-27T15:23:27.211475+0000","last_became_peered":"2023-11-27T15:23:27.211475+0000","last_unstale":"2023-11-27T15:23:32.151901+0000","last_undegraded":"2023-11-27T15:23:32.151901+0000","last_fullsized":"2023-11-27T15:23:32.151901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:34:31.524704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150698+0000","last_change":"2023-11-27T15:23:27.183246+0000","last_active":"2023-11-27T15:23:32.150698+0000","last_peered":"2023-11-27T15:23:32.150698+0000","last_clean":"2023-11-27T15:23:32.150698+0000","last_became_active":"2023-11-27T15:23:27.182454+0000","last_became_peered":"2023-11-27T15:23:27.182454+0000","last_unstale":"2023-11-27T15:23:32.150698+0000","last_undegraded":"2023-11-27T15:23:32.150698+0000","last_fullsized":"2023-11-27T15:23:32.150698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:18:44.304123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806853+0000","last_change":"2023-11-27T15:23:27.196461+0000","last_active":"2023-11-27T15:23:32.806853+0000","last_peered":"2023-11-27T15:23:32.806853+0000","last_clean":"2023-11-27T15:23:32.806853+0000","last_became_active":"2023-11-27T15:23:27.196122+0000","last_became_peered":"2023-11-27T15:23:27.196122+0000","last_unstale":"2023-11-27T15:23:32.806853+0000","last_undegraded":"2023-11-27T15:23:32.806853+0000","last_fullsized":"2023-11-27T15:23:32.806853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:25:09.594663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806301+0000","last_change":"2023-11-27T15:23:31.649582+0000","last_active":"2023-11-27T15:23:32.806301+0000","last_peered":"2023-11-27T15:23:32.806301+0000","last_clean":"2023-11-27T15:23:32.806301+0000","last_became_active":"2023-11-27T15:23:31.649172+0000","last_became_peered":"2023-11-27T15:23:31.649172+0000","last_unstale":"2023-11-27T15:23:32.806301+0000","last_undegraded":"2023-11-27T15:23:32.806301+0000","last_fullsized":"2023-11-27T15:23:32.806301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:11:01.675107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.249201+0000","last_change":"2023-11-27T15:23:26.609457+0000","last_active":"2023-11-27T15:23:58.249201+0000","last_peered":"2023-11-27T15:23:58.249201+0000","last_clean":"2023-11-27T15:23:58.249201+0000","last_became_active":"2023-11-27T15:23:26.609144+0000","last_became_peered":"2023-11-27T15:23:26.609144+0000","last_unstale":"2023-11-27T15:23:58.249201+0000","last_undegraded":"2023-11-27T15:23:58.249201+0000","last_fullsized":"2023-11-27T15:23:58.249201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:24:37.816167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445664+0000","last_change":"2023-11-27T15:23:27.204423+0000","last_active":"2023-11-27T15:23:32.445664+0000","last_peered":"2023-11-27T15:23:32.445664+0000","last_clean":"2023-11-27T15:23:32.445664+0000","last_became_active":"2023-11-27T15:23:27.204104+0000","last_became_peered":"2023-11-27T15:23:27.204104+0000","last_unstale":"2023-11-27T15:23:32.445664+0000","last_undegraded":"2023-11-27T15:23:32.445664+0000","last_fullsized":"2023-11-27T15:23:32.445664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:23:10.395677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240285+0000","last_change":"2023-11-27T15:23:31.655641+0000","last_active":"2023-11-27T15:23:32.240285+0000","last_peered":"2023-11-27T15:23:32.240285+0000","last_clean":"2023-11-27T15:23:32.240285+0000","last_became_active":"2023-11-27T15:23:31.654991+0000","last_became_peered":"2023-11-27T15:23:31.654991+0000","last_unstale":"2023-11-27T15:23:32.240285+0000","last_undegraded":"2023-11-27T15:23:32.240285+0000","last_fullsized":"2023-11-27T15:23:32.240285+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:11:50.509562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"83'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445960+0000","last_change":"2023-11-27T15:23:26.620780+0000","last_active":"2023-11-27T15:23:32.445960+0000","last_peered":"2023-11-27T15:23:32.445960+0000","last_clean":"2023-11-27T15:23:32.445960+0000","last_became_active":"2023-11-27T15:23:26.620438+0000","last_became_peered":"2023-11-27T15:23:26.620438+0000","last_unstale":"2023-11-27T15:23:32.445960+0000","last_undegraded":"2023-11-27T15:23:32.445960+0000","last_fullsized":"2023-11-27T15:23:32.445960+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:12:21.282175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807271+0000","last_change":"2023-11-27T15:23:27.195581+0000","last_active":"2023-11-27T15:23:32.807271+0000","last_peered":"2023-11-27T15:23:32.807271+0000","last_clean":"2023-11-27T15:23:32.807271+0000","last_became_active":"2023-11-27T15:23:27.195260+0000","last_became_peered":"2023-11-27T15:23:27.195260+0000","last_unstale":"2023-11-27T15:23:32.807271+0000","last_undegraded":"2023-11-27T15:23:32.807271+0000","last_fullsized":"2023-11-27T15:23:32.807271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:05:13.244643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240518+0000","last_change":"2023-11-27T15:23:31.655464+0000","last_active":"2023-11-27T15:23:32.240518+0000","last_peered":"2023-11-27T15:23:32.240518+0000","last_clean":"2023-11-27T15:23:32.240518+0000","last_became_active":"2023-11-27T15:23:31.654952+0000","last_became_peered":"2023-11-27T15:23:31.654952+0000","last_unstale":"2023-11-27T15:23:32.240518+0000","last_undegraded":"2023-11-27T15:23:32.240518+0000","last_fullsized":"2023-11-27T15:23:32.240518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:47:45.892988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.247586+0000","last_change":"2023-11-27T15:23:26.606593+0000","last_active":"2023-11-27T15:23:58.247586+0000","last_peered":"2023-11-27T15:23:58.247586+0000","last_clean":"2023-11-27T15:23:58.247586+0000","last_became_active":"2023-11-27T15:23:26.605719+0000","last_became_peered":"2023-11-27T15:23:26.605719+0000","last_unstale":"2023-11-27T15:23:58.247586+0000","last_undegraded":"2023-11-27T15:23:58.247586+0000","last_fullsized":"2023-11-27T15:23:58.247586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:24.130402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805664+0000","last_change":"2023-11-27T15:23:27.192629+0000","last_active":"2023-11-27T15:23:32.805664+0000","last_peered":"2023-11-27T15:23:32.805664+0000","last_clean":"2023-11-27T15:23:32.805664+0000","last_became_active":"2023-11-27T15:23:27.192327+0000","last_became_peered":"2023-11-27T15:23:27.192327+0000","last_unstale":"2023-11-27T15:23:32.805664+0000","last_undegraded":"2023-11-27T15:23:32.805664+0000","last_fullsized":"2023-11-27T15:23:32.805664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:50:01.298879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240541+0000","last_change":"2023-11-27T15:23:31.648296+0000","last_active":"2023-11-27T15:23:32.240541+0000","last_peered":"2023-11-27T15:23:32.240541+0000","last_clean":"2023-11-27T15:23:32.240541+0000","last_became_active":"2023-11-27T15:23:31.647756+0000","last_became_peered":"2023-11-27T15:23:31.647756+0000","last_unstale":"2023-11-27T15:23:32.240541+0000","last_undegraded":"2023-11-27T15:23:32.240541+0000","last_fullsized":"2023-11-27T15:23:32.240541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:36:20.124660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806078+0000","last_change":"2023-11-27T15:23:26.604061+0000","last_active":"2023-11-27T15:23:32.806078+0000","last_peered":"2023-11-27T15:23:32.806078+0000","last_clean":"2023-11-27T15:23:32.806078+0000","last_became_active":"2023-11-27T15:23:26.603725+0000","last_became_peered":"2023-11-27T15:23:26.603725+0000","last_unstale":"2023-11-27T15:23:32.806078+0000","last_undegraded":"2023-11-27T15:23:32.806078+0000","last_fullsized":"2023-11-27T15:23:32.806078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:57:33.099922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806841+0000","last_change":"2023-11-27T15:23:27.189389+0000","last_active":"2023-11-27T15:23:32.806841+0000","last_peered":"2023-11-27T15:23:32.806841+0000","last_clean":"2023-11-27T15:23:32.806841+0000","last_became_active":"2023-11-27T15:23:27.189072+0000","last_became_peered":"2023-11-27T15:23:27.189072+0000","last_unstale":"2023-11-27T15:23:32.806841+0000","last_undegraded":"2023-11-27T15:23:32.806841+0000","last_fullsized":"2023-11-27T15:23:32.806841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:31:14.484755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806619+0000","last_change":"2023-11-27T15:23:31.639042+0000","last_active":"2023-11-27T15:23:32.806619+0000","last_peered":"2023-11-27T15:23:32.806619+0000","last_clean":"2023-11-27T15:23:32.806619+0000","last_became_active":"2023-11-27T15:23:31.638611+0000","last_became_peered":"2023-11-27T15:23:31.638611+0000","last_unstale":"2023-11-27T15:23:32.806619+0000","last_undegraded":"2023-11-27T15:23:32.806619+0000","last_fullsized":"2023-11-27T15:23:32.806619+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:30:47.533959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:24:00.246985+0000","last_change":"2023-11-27T15:23:26.602979+0000","last_active":"2023-11-27T15:24:00.246985+0000","last_peered":"2023-11-27T15:24:00.246985+0000","last_clean":"2023-11-27T15:24:00.246985+0000","last_became_active":"2023-11-27T15:23:26.602681+0000","last_became_peered":"2023-11-27T15:23:26.602681+0000","last_unstale":"2023-11-27T15:24:00.246985+0000","last_undegraded":"2023-11-27T15:24:00.246985+0000","last_fullsized":"2023-11-27T15:24:00.246985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:21:25.923749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807277+0000","last_change":"2023-11-27T15:23:27.187294+0000","last_active":"2023-11-27T15:23:32.807277+0000","last_peered":"2023-11-27T15:23:32.807277+0000","last_clean":"2023-11-27T15:23:32.807277+0000","last_became_active":"2023-11-27T15:23:27.186445+0000","last_became_peered":"2023-11-27T15:23:27.186445+0000","last_unstale":"2023-11-27T15:23:32.807277+0000","last_undegraded":"2023-11-27T15:23:32.807277+0000","last_fullsized":"2023-11-27T15:23:32.807277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:37:31.842245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806339+0000","last_change":"2023-11-27T15:23:31.647523+0000","last_active":"2023-11-27T15:23:32.806339+0000","last_peered":"2023-11-27T15:23:32.806339+0000","last_clean":"2023-11-27T15:23:32.806339+0000","last_became_active":"2023-11-27T15:23:31.646945+0000","last_became_peered":"2023-11-27T15:23:31.646945+0000","last_unstale":"2023-11-27T15:23:32.806339+0000","last_undegraded":"2023-11-27T15:23:32.806339+0000","last_fullsized":"2023-11-27T15:23:32.806339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:40:32.187956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.248958+0000","last_change":"2023-11-27T15:23:26.602182+0000","last_active":"2023-11-27T15:23:58.248958+0000","last_peered":"2023-11-27T15:23:58.248958+0000","last_clean":"2023-11-27T15:23:58.248958+0000","last_became_active":"2023-11-27T15:23:26.601828+0000","last_became_peered":"2023-11-27T15:23:26.601828+0000","last_unstale":"2023-11-27T15:23:58.248958+0000","last_undegraded":"2023-11-27T15:23:58.248958+0000","last_fullsized":"2023-11-27T15:23:58.248958+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:41:27.271885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807145+0000","last_change":"2023-11-27T15:23:27.204874+0000","last_active":"2023-11-27T15:23:32.807145+0000","last_peered":"2023-11-27T15:23:32.807145+0000","last_clean":"2023-11-27T15:23:32.807145+0000","last_became_active":"2023-11-27T15:23:27.204588+0000","last_became_peered":"2023-11-27T15:23:27.204588+0000","last_unstale":"2023-11-27T15:23:32.807145+0000","last_undegraded":"2023-11-27T15:23:32.807145+0000","last_fullsized":"2023-11-27T15:23:32.807145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:51:34.799883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151899+0000","last_change":"2023-11-27T15:23:31.655262+0000","last_active":"2023-11-27T15:23:32.151899+0000","last_peered":"2023-11-27T15:23:32.151899+0000","last_clean":"2023-11-27T15:23:32.151899+0000","last_became_active":"2023-11-27T15:23:31.654840+0000","last_became_peered":"2023-11-27T15:23:31.654840+0000","last_unstale":"2023-11-27T15:23:32.151899+0000","last_undegraded":"2023-11-27T15:23:32.151899+0000","last_fullsized":"2023-11-27T15:23:32.151899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:48:46.408391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.250179+0000","last_change":"2023-11-27T15:23:26.610208+0000","last_active":"2023-11-27T15:23:58.250179+0000","last_peered":"2023-11-27T15:23:58.250179+0000","last_clean":"2023-11-27T15:23:58.250179+0000","last_became_active":"2023-11-27T15:23:26.609922+0000","last_became_peered":"2023-11-27T15:23:26.609922+0000","last_unstale":"2023-11-27T15:23:58.250179+0000","last_undegraded":"2023-11-27T15:23:58.250179+0000","last_fullsized":"2023-11-27T15:23:58.250179+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:42:52.139587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445392+0000","last_change":"2023-11-27T15:23:27.201533+0000","last_active":"2023-11-27T15:23:32.445392+0000","last_peered":"2023-11-27T15:23:32.445392+0000","last_clean":"2023-11-27T15:23:32.445392+0000","last_became_active":"2023-11-27T15:23:27.200516+0000","last_became_peered":"2023-11-27T15:23:27.200516+0000","last_unstale":"2023-11-27T15:23:32.445392+0000","last_undegraded":"2023-11-27T15:23:32.445392+0000","last_fullsized":"2023-11-27T15:23:32.445392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:11:52.020082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807126+0000","last_change":"2023-11-27T15:23:31.642858+0000","last_active":"2023-11-27T15:23:32.807126+0000","last_peered":"2023-11-27T15:23:32.807126+0000","last_clean":"2023-11-27T15:23:32.807126+0000","last_became_active":"2023-11-27T15:23:31.642367+0000","last_became_peered":"2023-11-27T15:23:31.642367+0000","last_unstale":"2023-11-27T15:23:32.807126+0000","last_undegraded":"2023-11-27T15:23:32.807126+0000","last_fullsized":"2023-11-27T15:23:32.807126+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:48:26.840176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806517+0000","last_change":"2023-11-27T15:23:26.602865+0000","last_active":"2023-11-27T15:23:32.806517+0000","last_peered":"2023-11-27T15:23:32.806517+0000","last_clean":"2023-11-27T15:23:32.806517+0000","last_became_active":"2023-11-27T15:23:26.602591+0000","last_became_peered":"2023-11-27T15:23:26.602591+0000","last_unstale":"2023-11-27T15:23:32.806517+0000","last_undegraded":"2023-11-27T15:23:32.806517+0000","last_fullsized":"2023-11-27T15:23:32.806517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:16:04.355452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150163+0000","last_change":"2023-11-27T15:23:27.186551+0000","last_active":"2023-11-27T15:23:32.150163+0000","last_peered":"2023-11-27T15:23:32.150163+0000","last_clean":"2023-11-27T15:23:32.150163+0000","last_became_active":"2023-11-27T15:23:27.185713+0000","last_became_peered":"2023-11-27T15:23:27.185713+0000","last_unstale":"2023-11-27T15:23:32.150163+0000","last_undegraded":"2023-11-27T15:23:32.150163+0000","last_fullsized":"2023-11-27T15:23:32.150163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:01:49.805801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445260+0000","last_change":"2023-11-27T15:23:31.641039+0000","last_active":"2023-11-27T15:23:32.445260+0000","last_peered":"2023-11-27T15:23:32.445260+0000","last_clean":"2023-11-27T15:23:32.445260+0000","last_became_active":"2023-11-27T15:23:31.639234+0000","last_became_peered":"2023-11-27T15:23:31.639234+0000","last_unstale":"2023-11-27T15:23:32.445260+0000","last_undegraded":"2023-11-27T15:23:32.445260+0000","last_fullsized":"2023-11-27T15:23:32.445260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:19:05.470708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.240994+0000","last_change":"2023-11-27T15:23:26.606409+0000","last_active":"2023-11-27T15:23:58.240994+0000","last_peered":"2023-11-27T15:23:58.240994+0000","last_clean":"2023-11-27T15:23:58.240994+0000","last_became_active":"2023-11-27T15:23:26.606060+0000","last_became_peered":"2023-11-27T15:23:26.606060+0000","last_unstale":"2023-11-27T15:23:58.240994+0000","last_undegraded":"2023-11-27T15:23:58.240994+0000","last_fullsized":"2023-11-27T15:23:58.240994+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:34:52.933461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805905+0000","last_change":"2023-11-27T15:23:27.180766+0000","last_active":"2023-11-27T15:23:32.805905+0000","last_peered":"2023-11-27T15:23:32.805905+0000","last_clean":"2023-11-27T15:23:32.805905+0000","last_became_active":"2023-11-27T15:23:27.180382+0000","last_became_peered":"2023-11-27T15:23:27.180382+0000","last_unstale":"2023-11-27T15:23:32.805905+0000","last_undegraded":"2023-11-27T15:23:32.805905+0000","last_fullsized":"2023-11-27T15:23:32.805905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:50:04.142745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.149973+0000","last_change":"2023-11-27T15:23:31.641697+0000","last_active":"2023-11-27T15:23:32.149973+0000","last_peered":"2023-11-27T15:23:32.149973+0000","last_clean":"2023-11-27T15:23:32.149973+0000","last_became_active":"2023-11-27T15:23:31.641246+0000","last_became_peered":"2023-11-27T15:23:31.641246+0000","last_unstale":"2023-11-27T15:23:32.149973+0000","last_undegraded":"2023-11-27T15:23:32.149973+0000","last_fullsized":"2023-11-27T15:23:32.149973+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:51:50.437000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.444150+0000","last_change":"2023-11-27T15:23:26.605307+0000","last_active":"2023-11-27T15:23:57.444150+0000","last_peered":"2023-11-27T15:23:57.444150+0000","last_clean":"2023-11-27T15:23:57.444150+0000","last_became_active":"2023-11-27T15:23:26.605047+0000","last_became_peered":"2023-11-27T15:23:26.605047+0000","last_unstale":"2023-11-27T15:23:57.444150+0000","last_undegraded":"2023-11-27T15:23:57.444150+0000","last_fullsized":"2023-11-27T15:23:57.444150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:43:04.881701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807258+0000","last_change":"2023-11-27T15:23:27.194469+0000","last_active":"2023-11-27T15:23:32.807258+0000","last_peered":"2023-11-27T15:23:32.807258+0000","last_clean":"2023-11-27T15:23:32.807258+0000","last_became_active":"2023-11-27T15:23:27.193930+0000","last_became_peered":"2023-11-27T15:23:27.193930+0000","last_unstale":"2023-11-27T15:23:32.807258+0000","last_undegraded":"2023-11-27T15:23:32.807258+0000","last_fullsized":"2023-11-27T15:23:32.807258+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:42:19.160539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240743+0000","last_change":"2023-11-27T15:23:31.655787+0000","last_active":"2023-11-27T15:23:32.240743+0000","last_peered":"2023-11-27T15:23:32.240743+0000","last_clean":"2023-11-27T15:23:32.240743+0000","last_became_active":"2023-11-27T15:23:31.654847+0000","last_became_peered":"2023-11-27T15:23:31.654847+0000","last_unstale":"2023-11-27T15:23:32.240743+0000","last_undegraded":"2023-11-27T15:23:32.240743+0000","last_fullsized":"2023-11-27T15:23:32.240743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:39:48.554692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.444225+0000","last_change":"2023-11-27T15:23:26.598669+0000","last_active":"2023-11-27T15:23:57.444225+0000","last_peered":"2023-11-27T15:23:57.444225+0000","last_clean":"2023-11-27T15:23:57.444225+0000","last_became_active":"2023-11-27T15:23:26.598224+0000","last_became_peered":"2023-11-27T15:23:26.598224+0000","last_unstale":"2023-11-27T15:23:57.444225+0000","last_undegraded":"2023-11-27T15:23:57.444225+0000","last_fullsized":"2023-11-27T15:23:57.444225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:54:30.744998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805760+0000","last_change":"2023-11-27T15:23:27.182800+0000","last_active":"2023-11-27T15:23:32.805760+0000","last_peered":"2023-11-27T15:23:32.805760+0000","last_clean":"2023-11-27T15:23:32.805760+0000","last_became_active":"2023-11-27T15:23:27.182572+0000","last_became_peered":"2023-11-27T15:23:27.182572+0000","last_unstale":"2023-11-27T15:23:32.805760+0000","last_undegraded":"2023-11-27T15:23:32.805760+0000","last_fullsized":"2023-11-27T15:23:32.805760+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:32:57.327675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150116+0000","last_change":"2023-11-27T15:23:31.645586+0000","last_active":"2023-11-27T15:23:32.150116+0000","last_peered":"2023-11-27T15:23:32.150116+0000","last_clean":"2023-11-27T15:23:32.150116+0000","last_became_active":"2023-11-27T15:23:31.644897+0000","last_became_peered":"2023-11-27T15:23:31.644897+0000","last_unstale":"2023-11-27T15:23:32.150116+0000","last_undegraded":"2023-11-27T15:23:32.150116+0000","last_fullsized":"2023-11-27T15:23:32.150116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:44:50.702316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'19","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:59.767999+0000","last_change":"2023-11-27T15:23:26.601652+0000","last_active":"2023-11-27T15:23:59.767999+0000","last_peered":"2023-11-27T15:23:59.767999+0000","last_clean":"2023-11-27T15:23:59.767999+0000","last_became_active":"2023-11-27T15:23:26.601306+0000","last_became_peered":"2023-11-27T15:23:26.601306+0000","last_unstale":"2023-11-27T15:23:59.767999+0000","last_undegraded":"2023-11-27T15:23:59.767999+0000","last_fullsized":"2023-11-27T15:23:59.767999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:02:11.299102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21175,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":6,"num_read_kb":7,"num_write":20,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805854+0000","last_change":"2023-11-27T15:23:27.186709+0000","last_active":"2023-11-27T15:23:32.805854+0000","last_peered":"2023-11-27T15:23:32.805854+0000","last_clean":"2023-11-27T15:23:32.805854+0000","last_became_active":"2023-11-27T15:23:27.186390+0000","last_became_peered":"2023-11-27T15:23:27.186390+0000","last_unstale":"2023-11-27T15:23:32.805854+0000","last_undegraded":"2023-11-27T15:23:32.805854+0000","last_fullsized":"2023-11-27T15:23:32.805854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:42:34.421537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807393+0000","last_change":"2023-11-27T15:23:31.651851+0000","last_active":"2023-11-27T15:23:32.807393+0000","last_peered":"2023-11-27T15:23:32.807393+0000","last_clean":"2023-11-27T15:23:32.807393+0000","last_became_active":"2023-11-27T15:23:31.651409+0000","last_became_peered":"2023-11-27T15:23:31.651409+0000","last_unstale":"2023-11-27T15:23:32.807393+0000","last_undegraded":"2023-11-27T15:23:32.807393+0000","last_fullsized":"2023-11-27T15:23:32.807393+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:39:45.865051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.248472+0000","last_change":"2023-11-27T15:23:26.602719+0000","last_active":"2023-11-27T15:23:58.248472+0000","last_peered":"2023-11-27T15:23:58.248472+0000","last_clean":"2023-11-27T15:23:58.248472+0000","last_became_active":"2023-11-27T15:23:26.602372+0000","last_became_peered":"2023-11-27T15:23:26.602372+0000","last_unstale":"2023-11-27T15:23:58.248472+0000","last_undegraded":"2023-11-27T15:23:58.248472+0000","last_fullsized":"2023-11-27T15:23:58.248472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:50:18.169759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806555+0000","last_change":"2023-11-27T15:23:27.200479+0000","last_active":"2023-11-27T15:23:32.806555+0000","last_peered":"2023-11-27T15:23:32.806555+0000","last_clean":"2023-11-27T15:23:32.806555+0000","last_became_active":"2023-11-27T15:23:27.200155+0000","last_became_peered":"2023-11-27T15:23:27.200155+0000","last_unstale":"2023-11-27T15:23:32.806555+0000","last_undegraded":"2023-11-27T15:23:32.806555+0000","last_fullsized":"2023-11-27T15:23:32.806555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:00:32.990163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151529+0000","last_change":"2023-11-27T15:23:31.655255+0000","last_active":"2023-11-27T15:23:32.151529+0000","last_peered":"2023-11-27T15:23:32.151529+0000","last_clean":"2023-11-27T15:23:32.151529+0000","last_became_active":"2023-11-27T15:23:31.654789+0000","last_became_peered":"2023-11-27T15:23:31.654789+0000","last_unstale":"2023-11-27T15:23:32.151529+0000","last_undegraded":"2023-11-27T15:23:32.151529+0000","last_fullsized":"2023-11-27T15:23:32.151529+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:06:57.555131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.443790+0000","last_change":"2023-11-27T15:23:26.605927+0000","last_active":"2023-11-27T15:23:57.443790+0000","last_peered":"2023-11-27T15:23:57.443790+0000","last_clean":"2023-11-27T15:23:57.443790+0000","last_became_active":"2023-11-27T15:23:26.605515+0000","last_became_peered":"2023-11-27T15:23:26.605515+0000","last_unstale":"2023-11-27T15:23:57.443790+0000","last_undegraded":"2023-11-27T15:23:57.443790+0000","last_fullsized":"2023-11-27T15:23:57.443790+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:39:06.131896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387649+0000","last_change":"2023-11-27T15:23:27.200237+0000","last_active":"2023-11-27T15:23:33.387649+0000","last_peered":"2023-11-27T15:23:33.387649+0000","last_clean":"2023-11-27T15:23:33.387649+0000","last_became_active":"2023-11-27T15:23:27.199952+0000","last_became_peered":"2023-11-27T15:23:27.199952+0000","last_unstale":"2023-11-27T15:23:33.387649+0000","last_undegraded":"2023-11-27T15:23:33.387649+0000","last_fullsized":"2023-11-27T15:23:33.387649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:25:38.280448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150647+0000","last_change":"2023-11-27T15:23:31.650718+0000","last_active":"2023-11-27T15:23:32.150647+0000","last_peered":"2023-11-27T15:23:32.150647+0000","last_clean":"2023-11-27T15:23:32.150647+0000","last_became_active":"2023-11-27T15:23:31.650144+0000","last_became_peered":"2023-11-27T15:23:31.650144+0000","last_unstale":"2023-11-27T15:23:32.150647+0000","last_undegraded":"2023-11-27T15:23:32.150647+0000","last_fullsized":"2023-11-27T15:23:32.150647+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:06:51.387716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:24:00.252935+0000","last_change":"2023-11-27T15:23:26.600962+0000","last_active":"2023-11-27T15:24:00.252935+0000","last_peered":"2023-11-27T15:24:00.252935+0000","last_clean":"2023-11-27T15:24:00.252935+0000","last_became_active":"2023-11-27T15:23:26.600628+0000","last_became_peered":"2023-11-27T15:23:26.600628+0000","last_unstale":"2023-11-27T15:24:00.252935+0000","last_undegraded":"2023-11-27T15:24:00.252935+0000","last_fullsized":"2023-11-27T15:24:00.252935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:13:05.116836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387225+0000","last_change":"2023-11-27T15:23:27.191939+0000","last_active":"2023-11-27T15:23:33.387225+0000","last_peered":"2023-11-27T15:23:33.387225+0000","last_clean":"2023-11-27T15:23:33.387225+0000","last_became_active":"2023-11-27T15:23:27.191634+0000","last_became_peered":"2023-11-27T15:23:27.191634+0000","last_unstale":"2023-11-27T15:23:33.387225+0000","last_undegraded":"2023-11-27T15:23:33.387225+0000","last_fullsized":"2023-11-27T15:23:33.387225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:09.985147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805930+0000","last_change":"2023-11-27T15:23:31.650733+0000","last_active":"2023-11-27T15:23:32.805930+0000","last_peered":"2023-11-27T15:23:32.805930+0000","last_clean":"2023-11-27T15:23:32.805930+0000","last_became_active":"2023-11-27T15:23:31.650280+0000","last_became_peered":"2023-11-27T15:23:31.650280+0000","last_unstale":"2023-11-27T15:23:32.805930+0000","last_undegraded":"2023-11-27T15:23:32.805930+0000","last_fullsized":"2023-11-27T15:23:32.805930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:34:35.055557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.249316+0000","last_change":"2023-11-27T15:23:26.615536+0000","last_active":"2023-11-27T15:23:58.249316+0000","last_peered":"2023-11-27T15:23:58.249316+0000","last_clean":"2023-11-27T15:23:58.249316+0000","last_became_active":"2023-11-27T15:23:26.615180+0000","last_became_peered":"2023-11-27T15:23:26.615180+0000","last_unstale":"2023-11-27T15:23:58.249316+0000","last_undegraded":"2023-11-27T15:23:58.249316+0000","last_fullsized":"2023-11-27T15:23:58.249316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:05:54.739431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806183+0000","last_change":"2023-11-27T15:23:27.189011+0000","last_active":"2023-11-27T15:23:32.806183+0000","last_peered":"2023-11-27T15:23:32.806183+0000","last_clean":"2023-11-27T15:23:32.806183+0000","last_became_active":"2023-11-27T15:23:27.188673+0000","last_became_peered":"2023-11-27T15:23:27.188673+0000","last_unstale":"2023-11-27T15:23:32.806183+0000","last_undegraded":"2023-11-27T15:23:32.806183+0000","last_fullsized":"2023-11-27T15:23:32.806183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:42:38.911696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806049+0000","last_change":"2023-11-27T15:23:31.644073+0000","last_active":"2023-11-27T15:23:32.806049+0000","last_peered":"2023-11-27T15:23:32.806049+0000","last_clean":"2023-11-27T15:23:32.806049+0000","last_became_active":"2023-11-27T15:23:31.643689+0000","last_became_peered":"2023-11-27T15:23:31.643689+0000","last_unstale":"2023-11-27T15:23:32.806049+0000","last_undegraded":"2023-11-27T15:23:32.806049+0000","last_fullsized":"2023-11-27T15:23:32.806049+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:16:46.525106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806706+0000","last_change":"2023-11-27T15:23:26.604034+0000","last_active":"2023-11-27T15:23:32.806706+0000","last_peered":"2023-11-27T15:23:32.806706+0000","last_clean":"2023-11-27T15:23:32.806706+0000","last_became_active":"2023-11-27T15:23:26.603689+0000","last_became_peered":"2023-11-27T15:23:26.603689+0000","last_unstale":"2023-11-27T15:23:32.806706+0000","last_undegraded":"2023-11-27T15:23:32.806706+0000","last_fullsized":"2023-11-27T15:23:32.806706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:40:44.325758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151781+0000","last_change":"2023-11-27T15:23:27.192723+0000","last_active":"2023-11-27T15:23:32.151781+0000","last_peered":"2023-11-27T15:23:32.151781+0000","last_clean":"2023-11-27T15:23:32.151781+0000","last_became_active":"2023-11-27T15:23:27.192034+0000","last_became_peered":"2023-11-27T15:23:27.192034+0000","last_unstale":"2023-11-27T15:23:32.151781+0000","last_undegraded":"2023-11-27T15:23:32.151781+0000","last_fullsized":"2023-11-27T15:23:32.151781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:40:19.176841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807181+0000","last_change":"2023-11-27T15:23:31.652756+0000","last_active":"2023-11-27T15:23:32.807181+0000","last_peered":"2023-11-27T15:23:32.807181+0000","last_clean":"2023-11-27T15:23:32.807181+0000","last_became_active":"2023-11-27T15:23:31.652280+0000","last_became_peered":"2023-11-27T15:23:31.652280+0000","last_unstale":"2023-11-27T15:23:32.807181+0000","last_undegraded":"2023-11-27T15:23:32.807181+0000","last_fullsized":"2023-11-27T15:23:32.807181+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:42:39.820208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150390+0000","last_change":"2023-11-27T15:23:26.609927+0000","last_active":"2023-11-27T15:23:32.150390+0000","last_peered":"2023-11-27T15:23:32.150390+0000","last_clean":"2023-11-27T15:23:32.150390+0000","last_became_active":"2023-11-27T15:23:26.608928+0000","last_became_peered":"2023-11-27T15:23:26.608928+0000","last_unstale":"2023-11-27T15:23:32.150390+0000","last_undegraded":"2023-11-27T15:23:32.150390+0000","last_fullsized":"2023-11-27T15:23:32.150390+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:22:11.858889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151949+0000","last_change":"2023-11-27T15:23:27.204619+0000","last_active":"2023-11-27T15:23:32.151949+0000","last_peered":"2023-11-27T15:23:32.151949+0000","last_clean":"2023-11-27T15:23:32.151949+0000","last_became_active":"2023-11-27T15:23:27.204298+0000","last_became_peered":"2023-11-27T15:23:27.204298+0000","last_unstale":"2023-11-27T15:23:32.151949+0000","last_undegraded":"2023-11-27T15:23:32.151949+0000","last_fullsized":"2023-11-27T15:23:32.151949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:04:25.075907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807083+0000","last_change":"2023-11-27T15:23:31.644763+0000","last_active":"2023-11-27T15:23:32.807083+0000","last_peered":"2023-11-27T15:23:32.807083+0000","last_clean":"2023-11-27T15:23:32.807083+0000","last_became_active":"2023-11-27T15:23:31.644106+0000","last_became_peered":"2023-11-27T15:23:31.644106+0000","last_unstale":"2023-11-27T15:23:32.807083+0000","last_undegraded":"2023-11-27T15:23:32.807083+0000","last_fullsized":"2023-11-27T15:23:32.807083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:04:07.977842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387293+0000","last_change":"2023-11-27T15:23:26.614703+0000","last_active":"2023-11-27T15:23:33.387293+0000","last_peered":"2023-11-27T15:23:33.387293+0000","last_clean":"2023-11-27T15:23:33.387293+0000","last_became_active":"2023-11-27T15:23:26.614324+0000","last_became_peered":"2023-11-27T15:23:26.614324+0000","last_unstale":"2023-11-27T15:23:33.387293+0000","last_undegraded":"2023-11-27T15:23:33.387293+0000","last_fullsized":"2023-11-27T15:23:33.387293+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:49:23.116644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151572+0000","last_change":"2023-11-27T15:23:27.197288+0000","last_active":"2023-11-27T15:23:32.151572+0000","last_peered":"2023-11-27T15:23:32.151572+0000","last_clean":"2023-11-27T15:23:32.151572+0000","last_became_active":"2023-11-27T15:23:27.196925+0000","last_became_peered":"2023-11-27T15:23:27.196925+0000","last_unstale":"2023-11-27T15:23:32.151572+0000","last_undegraded":"2023-11-27T15:23:32.151572+0000","last_fullsized":"2023-11-27T15:23:32.151572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:43:50.170855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806017+0000","last_change":"2023-11-27T15:23:31.652701+0000","last_active":"2023-11-27T15:23:32.806017+0000","last_peered":"2023-11-27T15:23:32.806017+0000","last_clean":"2023-11-27T15:23:32.806017+0000","last_became_active":"2023-11-27T15:23:31.652259+0000","last_became_peered":"2023-11-27T15:23:31.652259+0000","last_unstale":"2023-11-27T15:23:32.806017+0000","last_undegraded":"2023-11-27T15:23:32.806017+0000","last_fullsized":"2023-11-27T15:23:32.806017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:51:09.415292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806638+0000","last_change":"2023-11-27T15:23:27.190103+0000","last_active":"2023-11-27T15:23:32.806638+0000","last_peered":"2023-11-27T15:23:32.806638+0000","last_clean":"2023-11-27T15:23:32.806638+0000","last_became_active":"2023-11-27T15:23:27.189803+0000","last_became_peered":"2023-11-27T15:23:27.189803+0000","last_unstale":"2023-11-27T15:23:32.806638+0000","last_undegraded":"2023-11-27T15:23:32.806638+0000","last_fullsized":"2023-11-27T15:23:32.806638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:11:38.276085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.250788+0000","last_change":"2023-11-27T15:23:26.606338+0000","last_active":"2023-11-27T15:23:58.250788+0000","last_peered":"2023-11-27T15:23:58.250788+0000","last_clean":"2023-11-27T15:23:58.250788+0000","last_became_active":"2023-11-27T15:23:26.601152+0000","last_became_peered":"2023-11-27T15:23:26.601152+0000","last_unstale":"2023-11-27T15:23:58.250788+0000","last_undegraded":"2023-11-27T15:23:58.250788+0000","last_fullsized":"2023-11-27T15:23:58.250788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:25:51.376468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387602+0000","last_change":"2023-11-27T15:23:31.646457+0000","last_active":"2023-11-27T15:23:33.387602+0000","last_peered":"2023-11-27T15:23:33.387602+0000","last_clean":"2023-11-27T15:23:33.387602+0000","last_became_active":"2023-11-27T15:23:31.646071+0000","last_became_peered":"2023-11-27T15:23:31.646071+0000","last_unstale":"2023-11-27T15:23:33.387602+0000","last_undegraded":"2023-11-27T15:23:33.387602+0000","last_fullsized":"2023-11-27T15:23:33.387602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:50:17.518444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445545+0000","last_change":"2023-11-27T15:23:27.201405+0000","last_active":"2023-11-27T15:23:32.445545+0000","last_peered":"2023-11-27T15:23:32.445545+0000","last_clean":"2023-11-27T15:23:32.445545+0000","last_became_active":"2023-11-27T15:23:27.200951+0000","last_became_peered":"2023-11-27T15:23:27.200951+0000","last_unstale":"2023-11-27T15:23:32.445545+0000","last_undegraded":"2023-11-27T15:23:32.445545+0000","last_fullsized":"2023-11-27T15:23:32.445545+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:43:06.543689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.250588+0000","last_change":"2023-11-27T15:23:26.604328+0000","last_active":"2023-11-27T15:23:58.250588+0000","last_peered":"2023-11-27T15:23:58.250588+0000","last_clean":"2023-11-27T15:23:58.250588+0000","last_became_active":"2023-11-27T15:23:26.603937+0000","last_became_peered":"2023-11-27T15:23:26.603937+0000","last_unstale":"2023-11-27T15:23:58.250588+0000","last_undegraded":"2023-11-27T15:23:58.250588+0000","last_fullsized":"2023-11-27T15:23:58.250588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:13:28.104497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":343,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.386998+0000","last_change":"2023-11-27T15:22:17.949235+0000","last_active":"2023-11-27T15:23:33.386998+0000","last_peered":"2023-11-27T15:23:33.386998+0000","last_clean":"2023-11-27T15:23:33.386998+0000","last_became_active":"2023-11-27T15:22:17.948822+0000","last_became_peered":"2023-11-27T15:22:17.948822+0000","last_unstale":"2023-11-27T15:23:33.386998+0000","last_undegraded":"2023-11-27T15:23:33.386998+0000","last_fullsized":"2023-11-27T15:23:33.386998+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:19:02.909930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:19:02.909930+0000","last_clean_scrub_stamp":"2023-11-27T15:19:02.909930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:35:54.127781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807234+0000","last_change":"2023-11-27T15:23:31.654325+0000","last_active":"2023-11-27T15:23:32.807234+0000","last_peered":"2023-11-27T15:23:32.807234+0000","last_clean":"2023-11-27T15:23:32.807234+0000","last_became_active":"2023-11-27T15:23:31.653857+0000","last_became_peered":"2023-11-27T15:23:31.653857+0000","last_unstale":"2023-11-27T15:23:32.807234+0000","last_undegraded":"2023-11-27T15:23:32.807234+0000","last_fullsized":"2023-11-27T15:23:32.807234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:50:07.042580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805768+0000","last_change":"2023-11-27T15:23:27.192623+0000","last_active":"2023-11-27T15:23:32.805768+0000","last_peered":"2023-11-27T15:23:32.805768+0000","last_clean":"2023-11-27T15:23:32.805768+0000","last_became_active":"2023-11-27T15:23:27.192266+0000","last_became_peered":"2023-11-27T15:23:27.192266+0000","last_unstale":"2023-11-27T15:23:32.805768+0000","last_undegraded":"2023-11-27T15:23:32.805768+0000","last_fullsized":"2023-11-27T15:23:32.805768+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:19:04.605494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.248186+0000","last_change":"2023-11-27T15:23:26.603078+0000","last_active":"2023-11-27T15:23:58.248186+0000","last_peered":"2023-11-27T15:23:58.248186+0000","last_clean":"2023-11-27T15:23:58.248186+0000","last_became_active":"2023-11-27T15:23:26.602758+0000","last_became_peered":"2023-11-27T15:23:26.602758+0000","last_unstale":"2023-11-27T15:23:58.248186+0000","last_undegraded":"2023-11-27T15:23:58.248186+0000","last_fullsized":"2023-11-27T15:23:58.248186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:53:27.360807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151477+0000","last_change":"2023-11-27T15:23:31.648266+0000","last_active":"2023-11-27T15:23:32.151477+0000","last_peered":"2023-11-27T15:23:32.151477+0000","last_clean":"2023-11-27T15:23:32.151477+0000","last_became_active":"2023-11-27T15:23:31.647804+0000","last_became_peered":"2023-11-27T15:23:31.647804+0000","last_unstale":"2023-11-27T15:23:32.151477+0000","last_undegraded":"2023-11-27T15:23:32.151477+0000","last_fullsized":"2023-11-27T15:23:32.151477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:14:46.320009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.440496+0000","last_change":"2023-11-27T15:23:26.615577+0000","last_active":"2023-11-27T15:23:57.440496+0000","last_peered":"2023-11-27T15:23:57.440496+0000","last_clean":"2023-11-27T15:23:57.440496+0000","last_became_active":"2023-11-27T15:23:26.615287+0000","last_became_peered":"2023-11-27T15:23:26.615287+0000","last_unstale":"2023-11-27T15:23:57.440496+0000","last_undegraded":"2023-11-27T15:23:57.440496+0000","last_fullsized":"2023-11-27T15:23:57.440496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:17:38.255280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807155+0000","last_change":"2023-11-27T15:23:27.203162+0000","last_active":"2023-11-27T15:23:32.807155+0000","last_peered":"2023-11-27T15:23:32.807155+0000","last_clean":"2023-11-27T15:23:32.807155+0000","last_became_active":"2023-11-27T15:23:27.201725+0000","last_became_peered":"2023-11-27T15:23:27.201725+0000","last_unstale":"2023-11-27T15:23:32.807155+0000","last_undegraded":"2023-11-27T15:23:32.807155+0000","last_fullsized":"2023-11-27T15:23:32.807155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:26:49.344996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445343+0000","last_change":"2023-11-27T15:23:31.653253+0000","last_active":"2023-11-27T15:23:32.445343+0000","last_peered":"2023-11-27T15:23:32.445343+0000","last_clean":"2023-11-27T15:23:32.445343+0000","last_became_active":"2023-11-27T15:23:31.652777+0000","last_became_peered":"2023-11-27T15:23:31.652777+0000","last_unstale":"2023-11-27T15:23:32.445343+0000","last_undegraded":"2023-11-27T15:23:32.445343+0000","last_fullsized":"2023-11-27T15:23:32.445343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:42:04.663789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.249981+0000","last_change":"2023-11-27T15:23:26.610599+0000","last_active":"2023-11-27T15:23:58.249981+0000","last_peered":"2023-11-27T15:23:58.249981+0000","last_clean":"2023-11-27T15:23:58.249981+0000","last_became_active":"2023-11-27T15:23:26.610205+0000","last_became_peered":"2023-11-27T15:23:26.610205+0000","last_unstale":"2023-11-27T15:23:58.249981+0000","last_undegraded":"2023-11-27T15:23:58.249981+0000","last_fullsized":"2023-11-27T15:23:58.249981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:14:37.484874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445743+0000","last_change":"2023-11-27T15:23:27.203367+0000","last_active":"2023-11-27T15:23:32.445743+0000","last_peered":"2023-11-27T15:23:32.445743+0000","last_clean":"2023-11-27T15:23:32.445743+0000","last_became_active":"2023-11-27T15:23:27.203054+0000","last_became_peered":"2023-11-27T15:23:27.203054+0000","last_unstale":"2023-11-27T15:23:32.445743+0000","last_undegraded":"2023-11-27T15:23:32.445743+0000","last_fullsized":"2023-11-27T15:23:32.445743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:36:56.852033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805939+0000","last_change":"2023-11-27T15:23:31.638372+0000","last_active":"2023-11-27T15:23:32.805939+0000","last_peered":"2023-11-27T15:23:32.805939+0000","last_clean":"2023-11-27T15:23:32.805939+0000","last_became_active":"2023-11-27T15:23:31.637973+0000","last_became_peered":"2023-11-27T15:23:31.637973+0000","last_unstale":"2023-11-27T15:23:32.805939+0000","last_undegraded":"2023-11-27T15:23:32.805939+0000","last_fullsized":"2023-11-27T15:23:32.805939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:01:01.382773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.441204+0000","last_change":"2023-11-27T15:23:26.594218+0000","last_active":"2023-11-27T15:23:57.441204+0000","last_peered":"2023-11-27T15:23:57.441204+0000","last_clean":"2023-11-27T15:23:57.441204+0000","last_became_active":"2023-11-27T15:23:26.593842+0000","last_became_peered":"2023-11-27T15:23:26.593842+0000","last_unstale":"2023-11-27T15:23:57.441204+0000","last_undegraded":"2023-11-27T15:23:57.441204+0000","last_fullsized":"2023-11-27T15:23:57.441204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:10:10.686777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807263+0000","last_change":"2023-11-27T15:23:27.211258+0000","last_active":"2023-11-27T15:23:32.807263+0000","last_peered":"2023-11-27T15:23:32.807263+0000","last_clean":"2023-11-27T15:23:32.807263+0000","last_became_active":"2023-11-27T15:23:27.210963+0000","last_became_peered":"2023-11-27T15:23:27.210963+0000","last_unstale":"2023-11-27T15:23:32.807263+0000","last_undegraded":"2023-11-27T15:23:32.807263+0000","last_fullsized":"2023-11-27T15:23:32.807263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:29:21.488753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240782+0000","last_change":"2023-11-27T15:23:31.652938+0000","last_active":"2023-11-27T15:23:32.240782+0000","last_peered":"2023-11-27T15:23:32.240782+0000","last_clean":"2023-11-27T15:23:32.240782+0000","last_became_active":"2023-11-27T15:23:31.652504+0000","last_became_peered":"2023-11-27T15:23:31.652504+0000","last_unstale":"2023-11-27T15:23:32.240782+0000","last_undegraded":"2023-11-27T15:23:32.240782+0000","last_fullsized":"2023-11-27T15:23:32.240782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:59:59.372384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.443570+0000","last_change":"2023-11-27T15:23:26.602328+0000","last_active":"2023-11-27T15:23:57.443570+0000","last_peered":"2023-11-27T15:23:57.443570+0000","last_clean":"2023-11-27T15:23:57.443570+0000","last_became_active":"2023-11-27T15:23:26.601974+0000","last_became_peered":"2023-11-27T15:23:26.601974+0000","last_unstale":"2023-11-27T15:23:57.443570+0000","last_undegraded":"2023-11-27T15:23:57.443570+0000","last_fullsized":"2023-11-27T15:23:57.443570+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:46:03.610233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805397+0000","last_change":"2023-11-27T15:23:27.194577+0000","last_active":"2023-11-27T15:23:32.805397+0000","last_peered":"2023-11-27T15:23:32.805397+0000","last_clean":"2023-11-27T15:23:32.805397+0000","last_became_active":"2023-11-27T15:23:27.194186+0000","last_became_peered":"2023-11-27T15:23:27.194186+0000","last_unstale":"2023-11-27T15:23:32.805397+0000","last_undegraded":"2023-11-27T15:23:32.805397+0000","last_fullsized":"2023-11-27T15:23:32.805397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:01:42.662103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.150738+0000","last_change":"2023-11-27T15:23:31.649019+0000","last_active":"2023-11-27T15:23:32.150738+0000","last_peered":"2023-11-27T15:23:32.150738+0000","last_clean":"2023-11-27T15:23:32.150738+0000","last_became_active":"2023-11-27T15:23:31.647182+0000","last_became_peered":"2023-11-27T15:23:31.647182+0000","last_unstale":"2023-11-27T15:23:32.150738+0000","last_undegraded":"2023-11-27T15:23:32.150738+0000","last_fullsized":"2023-11-27T15:23:32.150738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:13:57.191561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.444377+0000","last_change":"2023-11-27T15:23:26.611336+0000","last_active":"2023-11-27T15:23:57.444377+0000","last_peered":"2023-11-27T15:23:57.444377+0000","last_clean":"2023-11-27T15:23:57.444377+0000","last_became_active":"2023-11-27T15:23:26.610819+0000","last_became_peered":"2023-11-27T15:23:26.610819+0000","last_unstale":"2023-11-27T15:23:57.444377+0000","last_undegraded":"2023-11-27T15:23:57.444377+0000","last_fullsized":"2023-11-27T15:23:57.444377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:35:57.668215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151229+0000","last_change":"2023-11-27T15:23:27.198336+0000","last_active":"2023-11-27T15:23:32.151229+0000","last_peered":"2023-11-27T15:23:32.151229+0000","last_clean":"2023-11-27T15:23:32.151229+0000","last_became_active":"2023-11-27T15:23:27.198062+0000","last_became_peered":"2023-11-27T15:23:27.198062+0000","last_unstale":"2023-11-27T15:23:32.151229+0000","last_undegraded":"2023-11-27T15:23:32.151229+0000","last_fullsized":"2023-11-27T15:23:32.151229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:10:31.709614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240780+0000","last_change":"2023-11-27T15:23:31.655526+0000","last_active":"2023-11-27T15:23:32.240780+0000","last_peered":"2023-11-27T15:23:32.240780+0000","last_clean":"2023-11-27T15:23:32.240780+0000","last_became_active":"2023-11-27T15:23:31.655011+0000","last_became_peered":"2023-11-27T15:23:31.655011+0000","last_unstale":"2023-11-27T15:23:32.240780+0000","last_undegraded":"2023-11-27T15:23:32.240780+0000","last_fullsized":"2023-11-27T15:23:32.240780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:37:13.232844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806016+0000","last_change":"2023-11-27T15:23:26.604536+0000","last_active":"2023-11-27T15:23:32.806016+0000","last_peered":"2023-11-27T15:23:32.806016+0000","last_clean":"2023-11-27T15:23:32.806016+0000","last_became_active":"2023-11-27T15:23:26.604239+0000","last_became_peered":"2023-11-27T15:23:26.604239+0000","last_unstale":"2023-11-27T15:23:32.806016+0000","last_undegraded":"2023-11-27T15:23:32.806016+0000","last_fullsized":"2023-11-27T15:23:32.806016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:48:36.835527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445522+0000","last_change":"2023-11-27T15:23:27.201159+0000","last_active":"2023-11-27T15:23:32.445522+0000","last_peered":"2023-11-27T15:23:32.445522+0000","last_clean":"2023-11-27T15:23:32.445522+0000","last_became_active":"2023-11-27T15:23:27.200800+0000","last_became_peered":"2023-11-27T15:23:27.200800+0000","last_unstale":"2023-11-27T15:23:32.445522+0000","last_undegraded":"2023-11-27T15:23:32.445522+0000","last_fullsized":"2023-11-27T15:23:32.445522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:05:44.391748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.240746+0000","last_change":"2023-11-27T15:23:31.652835+0000","last_active":"2023-11-27T15:23:32.240746+0000","last_peered":"2023-11-27T15:23:32.240746+0000","last_clean":"2023-11-27T15:23:32.240746+0000","last_became_active":"2023-11-27T15:23:31.652425+0000","last_became_peered":"2023-11-27T15:23:31.652425+0000","last_unstale":"2023-11-27T15:23:32.240746+0000","last_undegraded":"2023-11-27T15:23:32.240746+0000","last_fullsized":"2023-11-27T15:23:32.240746+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:32:00.109457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:57.442980+0000","last_change":"2023-11-27T15:23:26.606303+0000","last_active":"2023-11-27T15:23:57.442980+0000","last_peered":"2023-11-27T15:23:57.442980+0000","last_clean":"2023-11-27T15:23:57.442980+0000","last_became_active":"2023-11-27T15:23:26.605827+0000","last_became_peered":"2023-11-27T15:23:26.605827+0000","last_unstale":"2023-11-27T15:23:57.442980+0000","last_undegraded":"2023-11-27T15:23:57.442980+0000","last_fullsized":"2023-11-27T15:23:57.442980+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:01:59.336718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387434+0000","last_change":"2023-11-27T15:23:27.198422+0000","last_active":"2023-11-27T15:23:33.387434+0000","last_peered":"2023-11-27T15:23:33.387434+0000","last_clean":"2023-11-27T15:23:33.387434+0000","last_became_active":"2023-11-27T15:23:27.198116+0000","last_became_peered":"2023-11-27T15:23:27.198116+0000","last_unstale":"2023-11-27T15:23:33.387434+0000","last_undegraded":"2023-11-27T15:23:33.387434+0000","last_fullsized":"2023-11-27T15:23:33.387434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:06:48.770689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805383+0000","last_change":"2023-11-27T15:23:31.647798+0000","last_active":"2023-11-27T15:23:32.805383+0000","last_peered":"2023-11-27T15:23:32.805383+0000","last_clean":"2023-11-27T15:23:32.805383+0000","last_became_active":"2023-11-27T15:23:31.647087+0000","last_became_peered":"2023-11-27T15:23:31.647087+0000","last_unstale":"2023-11-27T15:23:32.805383+0000","last_undegraded":"2023-11-27T15:23:32.805383+0000","last_fullsized":"2023-11-27T15:23:32.805383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:15:38.922135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:58.251072+0000","last_change":"2023-11-27T15:23:26.603373+0000","last_active":"2023-11-27T15:23:58.251072+0000","last_peered":"2023-11-27T15:23:58.251072+0000","last_clean":"2023-11-27T15:23:58.251072+0000","last_became_active":"2023-11-27T15:23:26.603049+0000","last_became_peered":"2023-11-27T15:23:26.603049+0000","last_unstale":"2023-11-27T15:23:58.251072+0000","last_undegraded":"2023-11-27T15:23:58.251072+0000","last_fullsized":"2023-11-27T15:23:58.251072+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:06:36.526595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387805+0000","last_change":"2023-11-27T15:23:27.186270+0000","last_active":"2023-11-27T15:23:33.387805+0000","last_peered":"2023-11-27T15:23:33.387805+0000","last_clean":"2023-11-27T15:23:33.387805+0000","last_became_active":"2023-11-27T15:23:27.185952+0000","last_became_peered":"2023-11-27T15:23:27.185952+0000","last_unstale":"2023-11-27T15:23:33.387805+0000","last_undegraded":"2023-11-27T15:23:33.387805+0000","last_fullsized":"2023-11-27T15:23:33.387805+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:25:26.018331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805925+0000","last_change":"2023-11-27T15:23:31.648666+0000","last_active":"2023-11-27T15:23:32.805925+0000","last_peered":"2023-11-27T15:23:32.805925+0000","last_clean":"2023-11-27T15:23:32.805925+0000","last_became_active":"2023-11-27T15:23:31.647245+0000","last_became_peered":"2023-11-27T15:23:31.647245+0000","last_unstale":"2023-11-27T15:23:32.805925+0000","last_undegraded":"2023-11-27T15:23:32.805925+0000","last_fullsized":"2023-11-27T15:23:32.805925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:28:31.010202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.807088+0000","last_change":"2023-11-27T15:23:26.606886+0000","last_active":"2023-11-27T15:23:32.807088+0000","last_peered":"2023-11-27T15:23:32.807088+0000","last_clean":"2023-11-27T15:23:32.807088+0000","last_became_active":"2023-11-27T15:23:26.606556+0000","last_became_peered":"2023-11-27T15:23:26.606556+0000","last_unstale":"2023-11-27T15:23:32.807088+0000","last_undegraded":"2023-11-27T15:23:32.807088+0000","last_fullsized":"2023-11-27T15:23:32.807088+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:44:25.371619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387474+0000","last_change":"2023-11-27T15:23:27.194532+0000","last_active":"2023-11-27T15:23:33.387474+0000","last_peered":"2023-11-27T15:23:33.387474+0000","last_clean":"2023-11-27T15:23:33.387474+0000","last_became_active":"2023-11-27T15:23:27.194202+0000","last_became_peered":"2023-11-27T15:23:27.194202+0000","last_unstale":"2023-11-27T15:23:33.387474+0000","last_undegraded":"2023-11-27T15:23:33.387474+0000","last_fullsized":"2023-11-27T15:23:33.387474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:48:31.124655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806037+0000","last_change":"2023-11-27T15:23:31.638466+0000","last_active":"2023-11-27T15:23:32.806037+0000","last_peered":"2023-11-27T15:23:32.806037+0000","last_clean":"2023-11-27T15:23:32.806037+0000","last_became_active":"2023-11-27T15:23:31.638103+0000","last_became_peered":"2023-11-27T15:23:31.638103+0000","last_unstale":"2023-11-27T15:23:32.806037+0000","last_undegraded":"2023-11-27T15:23:32.806037+0000","last_fullsized":"2023-11-27T15:23:32.806037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:46:01.956128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805894+0000","last_change":"2023-11-27T15:23:26.613616+0000","last_active":"2023-11-27T15:23:32.805894+0000","last_peered":"2023-11-27T15:23:32.805894+0000","last_clean":"2023-11-27T15:23:32.805894+0000","last_became_active":"2023-11-27T15:23:26.608240+0000","last_became_peered":"2023-11-27T15:23:26.608240+0000","last_unstale":"2023-11-27T15:23:32.805894+0000","last_undegraded":"2023-11-27T15:23:32.805894+0000","last_fullsized":"2023-11-27T15:23:32.805894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:15:18.962601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:33.387750+0000","last_change":"2023-11-27T15:23:27.185677+0000","last_active":"2023-11-27T15:23:33.387750+0000","last_peered":"2023-11-27T15:23:33.387750+0000","last_clean":"2023-11-27T15:23:33.387750+0000","last_became_active":"2023-11-27T15:23:27.185321+0000","last_became_peered":"2023-11-27T15:23:27.185321+0000","last_unstale":"2023-11-27T15:23:33.387750+0000","last_undegraded":"2023-11-27T15:23:33.387750+0000","last_fullsized":"2023-11-27T15:23:33.387750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:41:04.564578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151397+0000","last_change":"2023-11-27T15:23:27.195847+0000","last_active":"2023-11-27T15:23:32.151397+0000","last_peered":"2023-11-27T15:23:32.151397+0000","last_clean":"2023-11-27T15:23:32.151397+0000","last_became_active":"2023-11-27T15:23:27.195421+0000","last_became_peered":"2023-11-27T15:23:27.195421+0000","last_unstale":"2023-11-27T15:23:32.151397+0000","last_undegraded":"2023-11-27T15:23:32.151397+0000","last_fullsized":"2023-11-27T15:23:32.151397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:43:55.522227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806546+0000","last_change":"2023-11-27T15:23:27.181352+0000","last_active":"2023-11-27T15:23:32.806546+0000","last_peered":"2023-11-27T15:23:32.806546+0000","last_clean":"2023-11-27T15:23:32.806546+0000","last_became_active":"2023-11-27T15:23:27.181042+0000","last_became_peered":"2023-11-27T15:23:27.181042+0000","last_unstale":"2023-11-27T15:23:32.806546+0000","last_undegraded":"2023-11-27T15:23:32.806546+0000","last_fullsized":"2023-11-27T15:23:32.806546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:59:34.485652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805780+0000","last_change":"2023-11-27T15:23:27.199915+0000","last_active":"2023-11-27T15:23:32.805780+0000","last_peered":"2023-11-27T15:23:32.805780+0000","last_clean":"2023-11-27T15:23:32.805780+0000","last_became_active":"2023-11-27T15:23:27.199595+0000","last_became_peered":"2023-11-27T15:23:27.199595+0000","last_unstale":"2023-11-27T15:23:32.805780+0000","last_undegraded":"2023-11-27T15:23:32.805780+0000","last_fullsized":"2023-11-27T15:23:32.805780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:38:25.307144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806283+0000","last_change":"2023-11-27T15:23:27.210258+0000","last_active":"2023-11-27T15:23:32.806283+0000","last_peered":"2023-11-27T15:23:32.806283+0000","last_clean":"2023-11-27T15:23:32.806283+0000","last_became_active":"2023-11-27T15:23:27.208028+0000","last_became_peered":"2023-11-27T15:23:27.208028+0000","last_unstale":"2023-11-27T15:23:32.806283+0000","last_undegraded":"2023-11-27T15:23:32.806283+0000","last_fullsized":"2023-11-27T15:23:32.806283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:23:01.035064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806936+0000","last_change":"2023-11-27T15:23:27.211759+0000","last_active":"2023-11-27T15:23:32.806936+0000","last_peered":"2023-11-27T15:23:32.806936+0000","last_clean":"2023-11-27T15:23:32.806936+0000","last_became_active":"2023-11-27T15:23:27.211445+0000","last_became_peered":"2023-11-27T15:23:27.211445+0000","last_unstale":"2023-11-27T15:23:32.806936+0000","last_undegraded":"2023-11-27T15:23:32.806936+0000","last_fullsized":"2023-11-27T15:23:32.806936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:42:20.173098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151808+0000","last_change":"2023-11-27T15:23:27.185855+0000","last_active":"2023-11-27T15:23:32.151808+0000","last_peered":"2023-11-27T15:23:32.151808+0000","last_clean":"2023-11-27T15:23:32.151808+0000","last_became_active":"2023-11-27T15:23:27.185535+0000","last_became_peered":"2023-11-27T15:23:27.185535+0000","last_unstale":"2023-11-27T15:23:32.151808+0000","last_undegraded":"2023-11-27T15:23:32.151808+0000","last_fullsized":"2023-11-27T15:23:32.151808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:46:29.055191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806865+0000","last_change":"2023-11-27T15:23:27.190163+0000","last_active":"2023-11-27T15:23:32.806865+0000","last_peered":"2023-11-27T15:23:32.806865+0000","last_clean":"2023-11-27T15:23:32.806865+0000","last_became_active":"2023-11-27T15:23:27.188729+0000","last_became_peered":"2023-11-27T15:23:27.188729+0000","last_unstale":"2023-11-27T15:23:32.806865+0000","last_undegraded":"2023-11-27T15:23:32.806865+0000","last_fullsized":"2023-11-27T15:23:32.806865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:30:20.263241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445896+0000","last_change":"2023-11-27T15:23:27.203904+0000","last_active":"2023-11-27T15:23:32.445896+0000","last_peered":"2023-11-27T15:23:32.445896+0000","last_clean":"2023-11-27T15:23:32.445896+0000","last_became_active":"2023-11-27T15:23:27.203578+0000","last_became_peered":"2023-11-27T15:23:27.203578+0000","last_unstale":"2023-11-27T15:23:32.445896+0000","last_undegraded":"2023-11-27T15:23:32.445896+0000","last_fullsized":"2023-11-27T15:23:32.445896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:49:47.755682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806592+0000","last_change":"2023-11-27T15:23:27.206791+0000","last_active":"2023-11-27T15:23:32.806592+0000","last_peered":"2023-11-27T15:23:32.806592+0000","last_clean":"2023-11-27T15:23:32.806592+0000","last_became_active":"2023-11-27T15:23:27.206529+0000","last_became_peered":"2023-11-27T15:23:27.206529+0000","last_unstale":"2023-11-27T15:23:32.806592+0000","last_undegraded":"2023-11-27T15:23:32.806592+0000","last_fullsized":"2023-11-27T15:23:32.806592+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:39:29.543511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.445442+0000","last_change":"2023-11-27T15:23:27.197562+0000","last_active":"2023-11-27T15:23:32.445442+0000","last_peered":"2023-11-27T15:23:32.445442+0000","last_clean":"2023-11-27T15:23:32.445442+0000","last_became_active":"2023-11-27T15:23:27.197233+0000","last_became_peered":"2023-11-27T15:23:27.197233+0000","last_unstale":"2023-11-27T15:23:32.445442+0000","last_undegraded":"2023-11-27T15:23:32.445442+0000","last_fullsized":"2023-11-27T15:23:32.445442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:21:31.977127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.805566+0000","last_change":"2023-11-27T15:23:27.191281+0000","last_active":"2023-11-27T15:23:32.805566+0000","last_peered":"2023-11-27T15:23:32.805566+0000","last_clean":"2023-11-27T15:23:32.805566+0000","last_became_active":"2023-11-27T15:23:27.190973+0000","last_became_peered":"2023-11-27T15:23:27.190973+0000","last_unstale":"2023-11-27T15:23:32.805566+0000","last_undegraded":"2023-11-27T15:23:32.805566+0000","last_fullsized":"2023-11-27T15:23:32.805566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:10:22.602292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.806751+0000","last_change":"2023-11-27T15:23:27.199251+0000","last_active":"2023-11-27T15:23:32.806751+0000","last_peered":"2023-11-27T15:23:32.806751+0000","last_clean":"2023-11-27T15:23:32.806751+0000","last_became_active":"2023-11-27T15:23:27.198926+0000","last_became_peered":"2023-11-27T15:23:27.198926+0000","last_unstale":"2023-11-27T15:23:32.806751+0000","last_undegraded":"2023-11-27T15:23:32.806751+0000","last_fullsized":"2023-11-27T15:23:32.806751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:18:22.223705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-27T15:23:32.151530+0000","last_change":"2023-11-27T15:23:27.192227+0000","last_active":"2023-11-27T15:23:32.151530+0000","last_peered":"2023-11-27T15:23:32.151530+0000","last_clean":"2023-11-27T15:23:32.151530+0000","last_became_active":"2023-11-27T15:23:27.191852+0000","last_became_peered":"2023-11-27T15:23:27.191852+0000","last_unstale":"2023-11-27T15:23:32.151530+0000","last_undegraded":"2023-11-27T15:23:32.151530+0000","last_fullsized":"2023-11-27T15:23:32.151530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:01:52.455108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":28672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":25469,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":137,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":76276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6072,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739080,"statfs":{"total":95995035648,"available":95988817920,"internally_reserved":0,"allocated":966656,"data_stored":467705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49399999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.52700000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.629}]},{"osd":5,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.66400000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.73599999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.7729999999999999,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.82899999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:23:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.85099999999999998}]}]},{"osd":10,"up_from":72,"seq":309237645336,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6964,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738188,"statfs":{"total":95995035648,"available":95987904512,"internally_reserved":0,"allocated":1617920,"data_stored":1080255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.63800000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.73099999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.93400000000000005}]},{"osd":4,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68999999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1,"5min":1,"15min":1},"last":0.96599999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.1259999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65500000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.95099999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.0069999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:23:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.175}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6284,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738868,"statfs":{"total":95995035648,"available":95988600832,"internally_reserved":0,"allocated":987136,"data_stored":472037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.56100000000000005,"15min":0.57599999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64900000000000002,"5min":1.054,"15min":1.054},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45100000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68000000000000005,"15min":0.69099999999999995},"min":{"1min":0.44400000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.0029999999999999,"5min":1.046,"15min":1.046},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.044,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.751}]},{"osd":2,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.079,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.505},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.53100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97199999999999998,"5min":1.294,"15min":1.294},"last":0.52300000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.73699999999999999,"15min":0.755},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.3520000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.58999999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.82399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.55800000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.88800000000000001,"15min":0.90400000000000003},"min":{"1min":0.64800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3129999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.85899999999999999},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Mon Nov 27 15:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.84299999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.94799999999999995,"15min":0.97199999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.429,"5min":2.069,"15min":2.069},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.95599999999999996,"15min":0.98399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.702,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.72499999999999998}]},{"osd":8,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.63900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1990000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.872,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.478,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.629}]},{"osd":10,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.73899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6404,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738748,"statfs":{"total":95995035648,"available":95988477952,"internally_reserved":0,"allocated":978944,"data_stored":467530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80100000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56000000000000005}]},{"osd":1,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.623,"15min":0.623},"min":{"1min":0.34799999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.873,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.85999999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48499999999999999,"15min":0.48599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70499999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48299999999999998,"15min":0.48699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.59199999999999997}]},{"osd":3,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95699999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.68700000000000006,"15min":0.67100000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":3.3940000000000001,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.81599999999999995}]},{"osd":4,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.42599999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.40799999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.91700000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.38100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88600000000000001,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73399999999999999,"15min":0.73699999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81999999999999995,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.88200000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.4020000000000001,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.996},{"interface":"front","average":{"1min":0.752,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.0429999999999999,"5min":1.069,"15min":1.069},"last":0.93799999999999994}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.73699999999999999,"15min":0.72199999999999998},"min":{"1min":0.63200000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.59399999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.117,"5min":1.119,"15min":1.119},"last":0.83399999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.90800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7120,"kb_used_data":1544,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738032,"statfs":{"total":95995035648,"available":95987744768,"internally_reserved":0,"allocated":1581056,"data_stored":1068159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.35799999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46700000000000003,"15min":0.46300000000000002},"min":{"1min":0.26400000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.48099999999999998,"15min":0.47799999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.50700000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.66600000000000004,"15min":0.63600000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89900000000000002,"5min":1.03,"15min":1.03},"last":0.68899999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.59899999999999998,"15min":0.58799999999999997},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.64200000000000002,"15min":0.63400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.378,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84899999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53700000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68100000000000005,"15min":0.67900000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.93100000000000005,"5min":1.018,"15min":1.018},"last":0.72299999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.74199999999999999,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.81799999999999995,"15min":0.78300000000000003},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":6.1349999999999998,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":0.65300000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:23:23 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.79500000000000004,"15min":0.76400000000000001},"min":{"1min":0.505,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":6.1050000000000004,"5min":6.1050000000000004,"15min":6.1050000000000004},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.81399999999999995,"15min":0.78400000000000003},"min":{"1min":0.53800000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":6.0330000000000004,"5min":6.0330000000000004,"15min":6.0330000000000004},"last":0.82999999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.54600000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98299999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.82299999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6392,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738760,"statfs":{"total":95995035648,"available":95988490240,"internally_reserved":0,"allocated":966656,"data_stored":467213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.504,"15min":0.49199999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54000000000000004,"15min":0.54200000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.57699999999999996}]},{"osd":1,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73999999999999999,"15min":0.72999999999999998},"min":{"1min":0.54900000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.752},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65100000000000002,"15min":0.63700000000000001},"min":{"1min":0.42699999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80200000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69899999999999995,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0149999999999999,"5min":1.101,"15min":1.101},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.77700000000000002,"15min":0.75700000000000001},"min":{"1min":0.59399999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.335,"5min":2.335,"15min":2.335},"last":0.69999999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65800000000000003,"15min":0.65200000000000002},"min":{"1min":0.39500000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.62},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.51100000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81599999999999995,"15min":0.80500000000000005},"min":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75900000000000001,"15min":0.748},"min":{"1min":0.53200000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.66400000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78500000000000003,"15min":0.78200000000000003},"min":{"1min":0.45600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.093,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.80600000000000005,"15min":0.80200000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.1080000000000001,"5min":1.581,"15min":1.581},"last":0.874}]},{"osd":7,"last update":"Mon Nov 27 15:23:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82199999999999995,"15min":0.81699999999999995},"min":{"1min":0.51100000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.159,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.82899999999999996,"15min":0.82399999999999995},"min":{"1min":0.53000000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.107,"5min":1.175,"15min":1.175},"last":0.83599999999999997}]},{"osd":8,"last update":"Mon Nov 27 15:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85499999999999998,"15min":0.85899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.874,"15min":0.88},"min":{"1min":0.63600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1299999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.91400000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.57499999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":0.93799999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.85699999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.90000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7360,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737792,"statfs":{"total":95995035648,"available":95987499008,"internally_reserved":0,"allocated":974848,"data_stored":470917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.32700000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.94799999999999995,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.54600000000000004},"min":{"1min":0.41099999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.89400000000000002,"5min":6.2839999999999998,"15min":6.2839999999999998},"last":0.63400000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.54700000000000004,"15min":0.51800000000000002},"min":{"1min":0.38700000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85999999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.56100000000000005,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57899999999999996,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72199999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.441},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71699999999999997,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.8100000000000001,"5min":6.3179999999999996,"15min":6.3179999999999996},"last":0.72299999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46899999999999997,"15min":0.42099999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.5,"15min":0.45800000000000002},"min":{"1min":0.308,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.41999999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:23:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.65300000000000002,"15min":0.623},"min":{"1min":0.46400000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93000000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.66000000000000003,"15min":0.625},"min":{"1min":0.52000000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.0029999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.80400000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72999999999999998,"15min":0.72799999999999998},"min":{"1min":0.498,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0289999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.53100000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.91400000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.68700000000000006}]},{"osd":7,"last update":"Mon Nov 27 15:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.377,"5min":0.316,"15min":0.316},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.70399999999999996,"15min":0.68500000000000005},"min":{"1min":0.42499999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.82699999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.51900000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.59999999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.86699999999999999,"15min":0.879},"min":{"1min":0.51600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1020000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.91500000000000004,"15min":0.92500000000000004},"min":{"1min":0.59599999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.876}]},{"osd":10,"last update":"Mon Nov 27 15:23:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.86699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7372,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737780,"statfs":{"total":95995035648,"available":95987486720,"internally_reserved":0,"allocated":987136,"data_stored":468270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.59499999999999997,"15min":0.54500000000000004},"min":{"1min":0.439,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93400000000000005,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.504},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.59699999999999998,"15min":0.55300000000000005},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.95699999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53700000000000003}]},{"osd":2,"last update":"Mon Nov 27 15:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56599999999999995,"15min":0.54100000000000004},"min":{"1min":0.503,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.59899999999999998,"15min":0.57599999999999996},"min":{"1min":0.41599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66600000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.379,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.59299999999999997,"15min":0.54400000000000004},"min":{"1min":0.46600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.76400000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67400000000000004,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92200000000000004,"5min":0.998,"15min":0.998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.98299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.89300000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.45900000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93999999999999995,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":1.115,"15min":1.115},"last":0.82799999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.76600000000000001,"15min":0.751},"min":{"1min":0.59299999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0720000000000001,"5min":1.147,"15min":1.147},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.76900000000000002,"15min":0.753},"min":{"1min":0.61299999999999999,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.107,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.85999999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:23:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77800000000000002},"min":{"1min":0.57099999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.874},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.079,"5min":1.22,"15min":1.22},"last":0.93500000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.89600000000000002,"15min":0.91100000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":0.874},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.032,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.92400000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.98399999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:23:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":1.0229999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7480,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737672,"statfs":{"total":95995035648,"available":95987376128,"internally_reserved":0,"allocated":1032192,"data_stored":507057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.85399999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.51500000000000001,"15min":0.50700000000000001},"min":{"1min":0.34999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79500000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58699999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:23:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.54800000000000004,"15min":0.502},"min":{"1min":0.439,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1479999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.55300000000000005,"15min":0.501},"min":{"1min":0.36899999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0700000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.74399999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.72299999999999998,"15min":0.82899999999999996},"min":{"1min":0.38600000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.874,"5min":7.1189999999999998,"15min":7.1189999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58899999999999997,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.91300000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:22:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.36699999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.55800000000000005},"min":{"1min":0.42599999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77500000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.45500000000000002,"15min":0.44800000000000001},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.62},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.47999999999999998},"min":{"1min":0.30499999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.754,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72099999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:23:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65600000000000003,"15min":0.63200000000000001},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.96199999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.71999999999999997,"15min":0.70899999999999996},"min":{"1min":0.53500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.90000000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":1.044},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.61899999999999999,"15min":0.61399999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.82299999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":1.0189999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6652,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738500,"statfs":{"total":95995035648,"available":95988224000,"internally_reserved":0,"allocated":970752,"data_stored":467227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.41799999999999998},"min":{"1min":0.313,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.67500000000000004,"5min":1.159,"15min":1.159},"last":0.437},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.34000000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67600000000000005,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.40600000000000003}]},{"osd":1,"last update":"Mon Nov 27 15:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.60299999999999998,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84799999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64200000000000002,"15min":0.60599999999999998},"min":{"1min":0.435,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81499999999999995,"5min":1.115,"15min":1.115},"last":0.77300000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89200000000000002,"5min":1.143,"15min":1.143},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.65500000000000003,"15min":0.63500000000000001},"min":{"1min":0.46700000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85499999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.48899999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:22:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.63400000000000001,"15min":0.60799999999999998},"min":{"1min":0.54700000000000004,"5min":0.443,"15min":0.443},"max":{"1min":1.032,"5min":1.034,"15min":1.034},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.47099999999999997,"5min":0.443,"15min":0.443},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.74399999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:23:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67100000000000004,"15min":0.64500000000000002},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.69299999999999995,"15min":0.67500000000000004},"min":{"1min":0.53600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0720000000000001,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.52600000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97099999999999997,"5min":1.306,"15min":1.306},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88,"5min":1.284,"15min":1.284},"last":0.998}]},{"osd":7,"last update":"Mon Nov 27 15:23:08 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.81899999999999995,"15min":0.80700000000000005},"min":{"1min":0.622,"5min":0.442,"15min":0.442},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.80500000000000005},"min":{"1min":0.64800000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":1.0209999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:23:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.81399999999999995},"min":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.84699999999999998,"15min":0.84999999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.80500000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:22:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.86499999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:23:14 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.92900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":970752,"data_stored":467778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.58399999999999996}]},{"osd":1,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.375},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.70899999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.498,"5min":0.436,"15min":0.39600000000000002},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72199999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61099999999999999,"15min":0.59599999999999997},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.62,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":0.85499999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.58699999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":1.329,"15min":1.329},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63200000000000001,"15min":0.59999999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92700000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.64800000000000002,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.93899999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.46800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97599999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.84699999999999998,"5min":1.288,"15min":1.288},"last":0.83599999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:23:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69299999999999995,"15min":0.69599999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.88600000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70099999999999996,"15min":0.70699999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97799999999999998,"5min":1.038,"15min":1.038},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.79900000000000004,"15min":0.80700000000000005},"min":{"1min":0.48399999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.05,"5min":1.107,"15min":1.107},"last":0.91300000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1599999999999999,"5min":1.222,"15min":1.222},"last":0.998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.84099999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.96899999999999997,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.87}]},{"osd":10,"last update":"Mon Nov 27 15:23:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.92700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724591,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7256,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737896,"statfs":{"total":95995035648,"available":95987605504,"internally_reserved":0,"allocated":1589248,"data_stored":1063405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.60099999999999998,"15min":0.61699999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.84799999999999998,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51700000000000002,"15min":0.505},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85799999999999998,"5min":0.873,"15min":0.873},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.42999999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84599999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.52200000000000002},"min":{"1min":0.44700000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.75900000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.41399999999999998,"15min":0.39800000000000002},"min":{"1min":0.307,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.437,"15min":0.42399999999999999},"min":{"1min":0.33800000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.35799999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.41099999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.875,"15min":0.875},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.82699999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63,"15min":0.61699999999999999},"min":{"1min":0.40799999999999997,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89600000000000002,"5min":2.097,"15min":2.097},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58899999999999997,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97099999999999997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.68300000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90100000000000002,"5min":1.004,"15min":1.004},"last":0.62},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91500000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.66000000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.67000000000000004},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.68400000000000005,"15min":0.67900000000000005},"min":{"1min":0.48499999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71399999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:22:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.69799999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.72599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":40207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":23095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:24:03.211 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-27T15:24:03.212 INFO:tasks.cephadm:Setting up client nodes... 2023-11-27T15:24:03.212 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-11-27T15:24:03.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:03 smithi008 ceph-mon[128788]: pgmap v300: 129 pgs: 129 active+clean; 602 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2023-11-27T15:24:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:03 smithi060 ceph-mon[131518]: pgmap v300: 129 pgs: 129 active+clean; 602 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2023-11-27T15:24:03.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:03 smithi029 ceph-mon[131166]: pgmap v300: 129 pgs: 129 active+clean; 602 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2023-11-27T15:24:04.105 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:04.723 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:04 smithi008 ceph-mon[128788]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:04.724 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:04 smithi008 ceph-mon[128788]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:04 smithi060 ceph-mon[131518]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:04 smithi060 ceph-mon[131518]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:04 smithi029 ceph-mon[131166]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:04 smithi029 ceph-mon[131166]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:05.608 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:05 smithi008 ceph-mon[128788]: pgmap v301: 129 pgs: 129 active+clean; 602 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2023-11-27T15:24:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:05 smithi060 ceph-mon[131518]: pgmap v301: 129 pgs: 129 active+clean; 602 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2023-11-27T15:24:05.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.878+0000 7f7a1bbca700 1 -- 172.21.15.8:0/2974927363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14103690 msgr2=0x7f7a14105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:05.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.878+0000 7f7a1bbca700 1 --2- 172.21.15.8:0/2974927363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14103690 0x7f7a14105a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009990 tx=0x7f7a04009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:05.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.878+0000 7f7a1bbca700 1 -- 172.21.15.8:0/2974927363 shutdown_connections 2023-11-27T15:24:05.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.878+0000 7f7a1bbca700 1 --2- 172.21.15.8:0/2974927363 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a14100d70 0x7f7a14103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:05.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.878+0000 7f7a1bbca700 1 --2- 172.21.15.8:0/2974927363 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a14105fb0 0x7f7a14108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:05.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.878+0000 7f7a1bbca700 1 --2- 172.21.15.8:0/2974927363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14103690 0x7f7a14105a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:05.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.878+0000 7f7a1bbca700 1 -- 172.21.15.8:0/2974927363 >> 172.21.15.8:0/2974927363 conn(0x7f7a140fa7f0 msgr2=0x7f7a140fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:05.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.878+0000 7f7a1bbca700 1 -- 172.21.15.8:0/2974927363 shutdown_connections 2023-11-27T15:24:05.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.878+0000 7f7a1bbca700 1 -- 172.21.15.8:0/2974927363 wait complete. 2023-11-27T15:24:05.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.879+0000 7f7a1bbca700 1 Processor -- start 2023-11-27T15:24:05.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.879+0000 7f7a1bbca700 1 -- start start 2023-11-27T15:24:05.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.879+0000 7f7a1bbca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14100d70 0x7f7a141006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:05.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.879+0000 7f7a1bbca700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a14103690 0x7f7a140fed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:05.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.879+0000 7f7a1bbca700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a14105fb0 0x7f7a140ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:05.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.879+0000 7f7a1bbca700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140ff9e0 con 0x7f7a14100d70 2023-11-27T15:24:05.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.879+0000 7f7a1bbca700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140ffb50 con 0x7f7a14103690 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.879+0000 7f7a1bbca700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a141a0150 con 0x7f7a14105fb0 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a19966700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14100d70 0x7f7a141006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a19966700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14100d70 0x7f7a141006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53622/0 (socket says 172.21.15.8:53622) 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a19966700 1 -- 172.21.15.8:0/1200812101 learned_addr learned my addr 172.21.15.8:0/1200812101 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a1a167700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a14105fb0 0x7f7a140ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a19165700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a14103690 0x7f7a140fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a19966700 1 -- 172.21.15.8:0/1200812101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a14105fb0 msgr2=0x7f7a140ff260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a19966700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a14105fb0 0x7f7a140ff260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a19966700 1 -- 172.21.15.8:0/1200812101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a14103690 msgr2=0x7f7a140fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a19966700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a14103690 0x7f7a140fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:05.885 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a19966700 1 -- 172.21.15.8:0/1200812101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a040095f0 con 0x7f7a14100d70 2023-11-27T15:24:05.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.880+0000 7f7a19966700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14100d70 0x7f7a141006d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d950 tx=0x7f7a1000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:05.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.881+0000 7f7a0affd700 1 -- 172.21.15.8:0/1200812101 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000f890 con 0x7f7a14100d70 2023-11-27T15:24:05.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.881+0000 7f7a0affd700 1 -- 172.21.15.8:0/1200812101 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a10010460 con 0x7f7a14100d70 2023-11-27T15:24:05.886 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.881+0000 7f7a1bbca700 1 -- 172.21.15.8:0/1200812101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1410bcd0 con 0x7f7a14100d70 2023-11-27T15:24:05.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.881+0000 7f7a0affd700 1 -- 172.21.15.8:0/1200812101 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000e610 con 0x7f7a14100d70 2023-11-27T15:24:05.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.881+0000 7f7a1bbca700 1 -- 172.21.15.8:0/1200812101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1410bb40 con 0x7f7a14100d70 2023-11-27T15:24:05.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.882+0000 7f7a1bbca700 1 -- 172.21.15.8:0/1200812101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a140fc360 con 0x7f7a14100d70 2023-11-27T15:24:05.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.884+0000 7f7a0affd700 1 -- 172.21.15.8:0/1200812101 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f7a100108d0 con 0x7f7a14100d70 2023-11-27T15:24:05.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.885+0000 7f7a0affd700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7a000a11e0 0x7f7a000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:05.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.886+0000 7f7a0affd700 1 -- 172.21.15.8:0/1200812101 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7a100ccad0 con 0x7f7a14100d70 2023-11-27T15:24:05.889 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.886+0000 7f7a19165700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7a000a11e0 0x7f7a000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:05.890 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.886+0000 7f7a19165700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7a000a11e0 0x7f7a000a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400b5c0 tx=0x7f7a04005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:05.893 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:05.890+0000 7f7a0affd700 1 -- 172.21.15.8:0/1200812101 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7a10095e70 con 0x7f7a14100d70 2023-11-27T15:24:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:05 smithi029 ceph-mon[131166]: pgmap v301: 129 pgs: 129 active+clean; 602 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2023-11-27T15:24:06.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.179+0000 7f7a1bbca700 1 -- 172.21.15.8:0/1200812101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f7a14061730 con 0x7f7a14100d70 2023-11-27T15:24:06.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.184+0000 7f7a0affd700 1 -- 172.21.15.8:0/1200812101 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f7a100160f0 con 0x7f7a14100d70 2023-11-27T15:24:06.188 INFO:teuthology.orchestra.run.smithi008.stdout:[client.0] 2023-11-27T15:24:06.188 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQCWtGRlpV/FChAAS9KBuwZmJAv/cmJxQ2eXZg== 2023-11-27T15:24:06.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 -- 172.21.15.8:0/1200812101 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7a000a11e0 msgr2=0x7f7a000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:06.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7a000a11e0 0x7f7a000a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400b5c0 tx=0x7f7a04005fb0 comp rx=0 tx=0).stop 2023-11-27T15:24:06.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 -- 172.21.15.8:0/1200812101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14100d70 msgr2=0x7f7a141006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:06.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14100d70 0x7f7a141006d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d950 tx=0x7f7a1000dc60 comp rx=0 tx=0).stop 2023-11-27T15:24:06.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 -- 172.21.15.8:0/1200812101 shutdown_connections 2023-11-27T15:24:06.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7a000a11e0 0x7f7a000a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:06.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7a14105fb0 0x7f7a140ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:06.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a14100d70 0x7f7a141006d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:06.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 --2- 172.21.15.8:0/1200812101 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7a14103690 0x7f7a140fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:06.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 -- 172.21.15.8:0/1200812101 >> 172.21.15.8:0/1200812101 conn(0x7f7a140fa7f0 msgr2=0x7f7a141019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:06.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 -- 172.21.15.8:0/1200812101 shutdown_connections 2023-11-27T15:24:06.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:06.186+0000 7f7a1bbca700 1 -- 172.21.15.8:0/1200812101 wait complete. 2023-11-27T15:24:06.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:06 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1200812101' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-27T15:24:06.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:06 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1200812101' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-27T15:24:06.711 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:24:06.711 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-11-27T15:24:06.711 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-27T15:24:06.813 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-27T15:24:06.813 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-11-27T15:24:06.813 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph mgr dump --format=json 2023-11-27T15:24:06.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:06 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1200812101' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-27T15:24:06.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:06 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1200812101' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-27T15:24:06.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:06 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1200812101' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-27T15:24:06.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:06 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1200812101' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-27T15:24:07.714 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:07 smithi060 ceph-mon[131518]: pgmap v302: 129 pgs: 129 active+clean; 624 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2023-11-27T15:24:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:07 smithi008 ceph-mon[128788]: pgmap v302: 129 pgs: 129 active+clean; 624 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2023-11-27T15:24:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:07 smithi029 ceph-mon[131166]: pgmap v302: 129 pgs: 129 active+clean; 624 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.0 KiB/s wr, 13 op/s 2023-11-27T15:24:09.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.544+0000 7f4b7371c700 1 -- 172.21.15.8:0/586003156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b6c100550 msgr2=0x7f4b6c100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:09.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.544+0000 7f4b7371c700 1 --2- 172.21.15.8:0/586003156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b6c100550 0x7f4b6c100920 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b680099c0 tx=0x7f4b68009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.544+0000 7f4b7371c700 1 -- 172.21.15.8:0/586003156 shutdown_connections 2023-11-27T15:24:09.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.544+0000 7f4b7371c700 1 --2- 172.21.15.8:0/586003156 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b6c1055f0 0x7f4b6c10d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.544+0000 7f4b7371c700 1 --2- 172.21.15.8:0/586003156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b6c100550 0x7f4b6c100920 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.544+0000 7f4b7371c700 1 --2- 172.21.15.8:0/586003156 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c100ef0 0x7f4b6c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.544+0000 7f4b7371c700 1 -- 172.21.15.8:0/586003156 >> 172.21.15.8:0/586003156 conn(0x7f4b6c0fbdf0 msgr2=0x7f4b6c0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:09.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.544+0000 7f4b7371c700 1 -- 172.21.15.8:0/586003156 shutdown_connections 2023-11-27T15:24:09.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.544+0000 7f4b7371c700 1 -- 172.21.15.8:0/586003156 wait complete. 2023-11-27T15:24:09.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.545+0000 7f4b7371c700 1 Processor -- start 2023-11-27T15:24:09.550 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.545+0000 7f4b7371c700 1 -- start start 2023-11-27T15:24:09.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.545+0000 7f4b7371c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b6c100550 0x7f4b6c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:09.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.545+0000 7f4b7371c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c100ef0 0x7f4b6c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:09.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.545+0000 7f4b7371c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b6c1055f0 0x7f4b6c0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:09.552 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.545+0000 7f4b7371c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c1a48e0 con 0x7f4b6c1055f0 2023-11-27T15:24:09.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.545+0000 7f4b7371c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c1a4a50 con 0x7f4b6c100ef0 2023-11-27T15:24:09.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.545+0000 7f4b7371c700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c1a4c10 con 0x7f4b6c100550 2023-11-27T15:24:09.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.546+0000 7f4b71cb9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b6c1055f0 0x7f4b6c0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:09.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.546+0000 7f4b71cb9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b6c1055f0 0x7f4b6c0738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53648/0 (socket says 172.21.15.8:53648) 2023-11-27T15:24:09.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.546+0000 7f4b71cb9700 1 -- 172.21.15.8:0/4263776545 learned_addr learned my addr 172.21.15.8:0/4263776545 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:09.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.546+0000 7f4b70cb7700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c100ef0 0x7f4b6c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:09.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.546+0000 7f4b714b8700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b6c100550 0x7f4b6c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:09.553 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.546+0000 7f4b71cb9700 1 -- 172.21.15.8:0/4263776545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b6c100550 msgr2=0x7f4b6c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:09.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.546+0000 7f4b71cb9700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b6c100550 0x7f4b6c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.546+0000 7f4b71cb9700 1 -- 172.21.15.8:0/4263776545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c100ef0 msgr2=0x7f4b6c073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:09.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.546+0000 7f4b71cb9700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c100ef0 0x7f4b6c073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.546+0000 7f4b71cb9700 1 -- 172.21.15.8:0/4263776545 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b680095f0 con 0x7f4b6c1055f0 2023-11-27T15:24:09.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.547+0000 7f4b71cb9700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b6c1055f0 0x7f4b6c0738b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400ef30 tx=0x7f4b6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:09.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.547+0000 7f4b627fc700 1 -- 172.21.15.8:0/4263776545 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64018070 con 0x7f4b6c1055f0 2023-11-27T15:24:09.554 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.547+0000 7f4b627fc700 1 -- 172.21.15.8:0/4263776545 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b6400f040 con 0x7f4b6c1055f0 2023-11-27T15:24:09.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.547+0000 7f4b7371c700 1 -- 172.21.15.8:0/4263776545 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c110380 con 0x7f4b6c1055f0 2023-11-27T15:24:09.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.547+0000 7f4b627fc700 1 -- 172.21.15.8:0/4263776545 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6400cd00 con 0x7f4b6c1055f0 2023-11-27T15:24:09.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.547+0000 7f4b7371c700 1 -- 172.21.15.8:0/4263776545 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c1101f0 con 0x7f4b6c1055f0 2023-11-27T15:24:09.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.548+0000 7f4b7371c700 1 -- 172.21.15.8:0/4263776545 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6c110510 con 0x7f4b6c1055f0 2023-11-27T15:24:09.556 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.550+0000 7f4b627fc700 1 -- 172.21.15.8:0/4263776545 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f4b64003a00 con 0x7f4b6c1055f0 2023-11-27T15:24:09.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.551+0000 7f4b627fc700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4b580a1260 0x7f4b580a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:09.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.551+0000 7f4b627fc700 1 -- 172.21.15.8:0/4263776545 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4b640cc730 con 0x7f4b6c1055f0 2023-11-27T15:24:09.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.552+0000 7f4b714b8700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4b580a1260 0x7f4b580a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:09.557 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.552+0000 7f4b714b8700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4b580a1260 0x7f4b580a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009990 tx=0x7f4b68005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:09.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.556+0000 7f4b627fc700 1 -- 172.21.15.8:0/4263776545 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4b64095b60 con 0x7f4b6c1055f0 2023-11-27T15:24:09.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:09 smithi060 ceph-mon[131518]: pgmap v303: 129 pgs: 129 active+clean; 624 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2023-11-27T15:24:09.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:09 smithi008 ceph-mon[128788]: pgmap v303: 129 pgs: 129 active+clean; 624 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2023-11-27T15:24:09.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:09 smithi029 ceph-mon[131166]: pgmap v303: 129 pgs: 129 active+clean; 624 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2023-11-27T15:24:09.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.823+0000 7f4b7371c700 1 -- 172.21.15.8:0/4263776545 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4b6c066ea0 con 0x7f4b6c1055f0 2023-11-27T15:24:09.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.829+0000 7f4b627fc700 1 -- 172.21.15.8:0/4263776545 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260117 (secure 0 0 0) 0x7f4b64099180 con 0x7f4b6c1055f0 2023-11-27T15:24:09.839 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:09.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 -- 172.21.15.8:0/4263776545 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4b580a1260 msgr2=0x7f4b580a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:09.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4b580a1260 0x7f4b580a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009990 tx=0x7f4b68005fb0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 -- 172.21.15.8:0/4263776545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b6c1055f0 msgr2=0x7f4b6c0738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:09.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b6c1055f0 0x7f4b6c0738b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400ef30 tx=0x7f4b6400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 -- 172.21.15.8:0/4263776545 shutdown_connections 2023-11-27T15:24:09.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4b580a1260 0x7f4b580a3710 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4b6c100550 0x7f4b6c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b6c1055f0 0x7f4b6c0738b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 --2- 172.21.15.8:0/4263776545 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4b6c100ef0 0x7f4b6c073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:09.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 -- 172.21.15.8:0/4263776545 >> 172.21.15.8:0/4263776545 conn(0x7f4b6c0fbdf0 msgr2=0x7f4b6c0fc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:09.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 -- 172.21.15.8:0/4263776545 shutdown_connections 2023-11-27T15:24:09.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:09.833+0000 7f4b7371c700 1 -- 172.21.15.8:0/4263776545 wait complete. 2023-11-27T15:24:10.315 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":1756612424},{"type":"v1","addr":"172.21.15.8:6801","nonce":1756612424}]},"active_addr":"172.21.15.8:6801/1756612424","active_change":"2023-11-27T15:16:38.243059+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.8:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":2984393656}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":3264476100}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":3796022999}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":3193049984}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":4103695600}]}]} 2023-11-27T15:24:10.321 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-11-27T15:24:10.321 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-11-27T15:24:10.322 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-11-27T15:24:10.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:10 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/4263776545' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-27T15:24:10.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:10 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4263776545' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-27T15:24:10.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:10 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/4263776545' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-27T15:24:11.182 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:11.500 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:11 smithi008 ceph-mon[128788]: pgmap v304: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2023-11-27T15:24:11.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:11 smithi060 ceph-mon[131518]: pgmap v304: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2023-11-27T15:24:11.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:11 smithi029 ceph-mon[131166]: pgmap v304: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.800+0000 7f811efb2700 1 -- 172.21.15.8:0/817614427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8118100ee0 msgr2=0x7f8118104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.800+0000 7f811efb2700 1 --2- 172.21.15.8:0/817614427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8118100ee0 0x7f8118104eb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f81040099c0 tx=0x7f8104009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.800+0000 7f811efb2700 1 -- 172.21.15.8:0/817614427 shutdown_connections 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.800+0000 7f811efb2700 1 --2- 172.21.15.8:0/817614427 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f81181055e0 0x7f811810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.800+0000 7f811efb2700 1 --2- 172.21.15.8:0/817614427 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118100540 0x7f8118100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.800+0000 7f811efb2700 1 --2- 172.21.15.8:0/817614427 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8118100ee0 0x7f8118104eb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.800+0000 7f811efb2700 1 -- 172.21.15.8:0/817614427 >> 172.21.15.8:0/817614427 conn(0x7f81180fbdc0 msgr2=0x7f81180fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.800+0000 7f811efb2700 1 -- 172.21.15.8:0/817614427 shutdown_connections 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.800+0000 7f811efb2700 1 -- 172.21.15.8:0/817614427 wait complete. 2023-11-27T15:24:12.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.801+0000 7f811efb2700 1 Processor -- start 2023-11-27T15:24:12.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.801+0000 7f811efb2700 1 -- start start 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.801+0000 7f811efb2700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118100540 0x7f81181985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f811efb2700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8118100ee0 0x7f8118198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f811efb2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81181055e0 0x7f811819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f811efb2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811819d580 con 0x7f81181055e0 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f811efb2700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811819d6f0 con 0x7f8118100540 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f811efb2700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811819d8b0 con 0x7f8118100ee0 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f8117fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8118100ee0 0x7f8118198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f811cd4e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118100540 0x7f81181985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f8117fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8118100ee0 0x7f8118198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:43960/0 (socket says 172.21.15.8:43960) 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f8117fff700 1 -- 172.21.15.8:0/1549996334 learned_addr learned my addr 172.21.15.8:0/1549996334 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f811d54f700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81181055e0 0x7f811819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f8117fff700 1 -- 172.21.15.8:0/1549996334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118100540 msgr2=0x7f81181985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f8117fff700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118100540 0x7f81181985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f8117fff700 1 -- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81181055e0 msgr2=0x7f811819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f8117fff700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81181055e0 0x7f811819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f8117fff700 1 -- 172.21.15.8:0/1549996334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81040095f0 con 0x7f8118100ee0 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.802+0000 7f811d54f700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81181055e0 0x7f811819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.803+0000 7f8117fff700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8118100ee0 0x7f8118198b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8104009f50 tx=0x7f8104005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.803+0000 7f8115ffb700 1 -- 172.21.15.8:0/1549996334 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810401d070 con 0x7f8118100ee0 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.803+0000 7f8115ffb700 1 -- 172.21.15.8:0/1549996334 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8104003f90 con 0x7f8118100ee0 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.803+0000 7f811efb2700 1 -- 172.21.15.8:0/1549996334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8118110360 con 0x7f8118100ee0 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.803+0000 7f8115ffb700 1 -- 172.21.15.8:0/1549996334 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104022bd0 con 0x7f8118100ee0 2023-11-27T15:24:12.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.803+0000 7f811efb2700 1 -- 172.21.15.8:0/1549996334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81181101d0 con 0x7f8118100ee0 2023-11-27T15:24:12.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.804+0000 7f811efb2700 1 -- 172.21.15.8:0/1549996334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81181104f0 con 0x7f8118100ee0 2023-11-27T15:24:12.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.806+0000 7f8115ffb700 1 -- 172.21.15.8:0/1549996334 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8104003ad0 con 0x7f8118100ee0 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.808+0000 7f8115ffb700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f81080a1260 0x7f81080a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.808+0000 7f8115ffb700 1 -- 172.21.15.8:0/1549996334 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f81040ce720 con 0x7f8118100ee0 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.808+0000 7f811cd4e700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f81080a1260 0x7f81080a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.808+0000 7f811cd4e700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f81080a1260 0x7f81080a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f810c007be0 tx=0x7f810c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:12.810+0000 7f8115ffb700 1 -- 172.21.15.8:0/1549996334 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8104097b50 con 0x7f8118100ee0 2023-11-27T15:24:13.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.020+0000 7f811efb2700 1 -- 172.21.15.8:0/1549996334 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81181a5640 con 0x7f8118100ee0 2023-11-27T15:24:13.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.022+0000 7f8115ffb700 1 -- 172.21.15.8:0/1549996334 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23817 (secure 0 0 0) 0x7f810409b170 con 0x7f8118100ee0 2023-11-27T15:24:13.026 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:13.026 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":88,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:23:32.146115+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:24:13.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.024+0000 7f811efb2700 1 -- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f81080a1260 msgr2=0x7f81080a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:13.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.024+0000 7f811efb2700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f81080a1260 0x7f81080a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f810c007be0 tx=0x7f810c0073d0 comp rx=0 tx=0).stop 2023-11-27T15:24:13.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.024+0000 7f811efb2700 1 -- 172.21.15.8:0/1549996334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8118100ee0 msgr2=0x7f8118198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:13.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.024+0000 7f811efb2700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8118100ee0 0x7f8118198b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8104009f50 tx=0x7f8104005740 comp rx=0 tx=0).stop 2023-11-27T15:24:13.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.024+0000 7f811efb2700 1 -- 172.21.15.8:0/1549996334 shutdown_connections 2023-11-27T15:24:13.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.024+0000 7f811efb2700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81181055e0 0x7f811819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:13.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.024+0000 7f811efb2700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8118100540 0x7f81181985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:13.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.024+0000 7f811efb2700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f81080a1260 0x7f81080a3710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:13.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.024+0000 7f811efb2700 1 --2- 172.21.15.8:0/1549996334 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8118100ee0 0x7f8118198b00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:13.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.024+0000 7f811efb2700 1 -- 172.21.15.8:0/1549996334 >> 172.21.15.8:0/1549996334 conn(0x7f81180fbdc0 msgr2=0x7f81180fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:13.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.025+0000 7f811efb2700 1 -- 172.21.15.8:0/1549996334 shutdown_connections 2023-11-27T15:24:13.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:13.025+0000 7f811efb2700 1 -- 172.21.15.8:0/1549996334 wait complete. 2023-11-27T15:24:13.485 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-11-27T15:24:13.485 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-11-27T15:24:13.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:13 smithi029 ceph-mon[131166]: pgmap v305: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-11-27T15:24:13.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:13 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1549996334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:13 smithi060 ceph-mon[131518]: pgmap v305: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-11-27T15:24:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:13 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1549996334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:13 smithi008 ceph-mon[128788]: pgmap v305: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2023-11-27T15:24:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:13 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1549996334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:14.346 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:15.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:15 smithi008 ceph-mon[128788]: pgmap v306: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2023-11-27T15:24:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:15 smithi060 ceph-mon[131518]: pgmap v306: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2023-11-27T15:24:15.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.893+0000 7f6104ec8700 1 -- 172.21.15.8:0/1142294311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6100101e50 msgr2=0x7f610010a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:15.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.893+0000 7f6104ec8700 1 --2- 172.21.15.8:0/1142294311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6100101e50 0x7f610010a350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009990 tx=0x7f60f4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.893+0000 7f6104ec8700 1 -- 172.21.15.8:0/1142294311 shutdown_connections 2023-11-27T15:24:15.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.893+0000 7f6104ec8700 1 --2- 172.21.15.8:0/1142294311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f610010a890 0x7f610010ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.893+0000 7f6104ec8700 1 --2- 172.21.15.8:0/1142294311 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6100101540 0x7f6100101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.897 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.893+0000 7f6104ec8700 1 --2- 172.21.15.8:0/1142294311 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6100101e50 0x7f610010a350 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.893+0000 7f6104ec8700 1 -- 172.21.15.8:0/1142294311 >> 172.21.15.8:0/1142294311 conn(0x7f61000770a0 msgr2=0x7f61000774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:15.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.893+0000 7f6104ec8700 1 -- 172.21.15.8:0/1142294311 shutdown_connections 2023-11-27T15:24:15.898 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.893+0000 7f6104ec8700 1 -- 172.21.15.8:0/1142294311 wait complete. 2023-11-27T15:24:15.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:15 smithi029 ceph-mon[131166]: pgmap v306: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.3 KiB/s wr, 6 op/s 2023-11-27T15:24:15.900 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.894+0000 7f6104ec8700 1 Processor -- start 2023-11-27T15:24:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.894+0000 7f6104ec8700 1 -- start start 2023-11-27T15:24:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.894+0000 7f6104ec8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6100101540 0x7f61000ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.894+0000 7f6104ec8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6100101e50 0x7f61000ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.894+0000 7f6104ec8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f610010a890 0x7f61000ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.894+0000 7f6104ec8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61001a48b0 con 0x7f6100101540 2023-11-27T15:24:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.894+0000 7f6104ec8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61001a4a20 con 0x7f6100101e50 2023-11-27T15:24:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.894+0000 7f6104ec8700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61001a4be0 con 0x7f610010a890 2023-11-27T15:24:15.901 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fe59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6100101540 0x7f61000ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fe59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6100101540 0x7f61000ff1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55614/0 (socket says 172.21.15.8:55614) 2023-11-27T15:24:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fe59c700 1 -- 172.21.15.8:0/2848065793 learned_addr learned my addr 172.21.15.8:0/2848065793 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fdd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6100101e50 0x7f61000ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fed9d700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f610010a890 0x7f61000ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fe59c700 1 -- 172.21.15.8:0/2848065793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f610010a890 msgr2=0x7f61000ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fe59c700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f610010a890 0x7f61000ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.902 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fe59c700 1 -- 172.21.15.8:0/2848065793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6100101e50 msgr2=0x7f61000ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fe59c700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6100101e50 0x7f61000ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fe59c700 1 -- 172.21.15.8:0/2848065793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f8009c40 con 0x7f6100101540 2023-11-27T15:24:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.895+0000 7f60fdd9b700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6100101e50 0x7f61000ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.896+0000 7f60fe59c700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6100101540 0x7f61000ff1a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f60f000e9a0 tx=0x7f60f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.896+0000 7f60ef7fe700 1 -- 172.21.15.8:0/2848065793 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0009790 con 0x7f6100101540 2023-11-27T15:24:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.896+0000 7f60ef7fe700 1 -- 172.21.15.8:0/2848065793 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60f000ce80 con 0x7f6100101540 2023-11-27T15:24:15.903 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.896+0000 7f60ef7fe700 1 -- 172.21.15.8:0/2848065793 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0018680 con 0x7f6100101540 2023-11-27T15:24:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.896+0000 7f6104ec8700 1 -- 172.21.15.8:0/2848065793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f40095f0 con 0x7f6100101540 2023-11-27T15:24:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.896+0000 7f6104ec8700 1 -- 172.21.15.8:0/2848065793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6100110360 con 0x7f6100101540 2023-11-27T15:24:15.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.897+0000 7f6104ec8700 1 -- 172.21.15.8:0/2848065793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61001101d0 con 0x7f6100101540 2023-11-27T15:24:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.899+0000 7f60ef7fe700 1 -- 172.21.15.8:0/2848065793 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f60f0010430 con 0x7f6100101540 2023-11-27T15:24:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.900+0000 7f60ef7fe700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60e80a11e0 0x7f60e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.900+0000 7f60ef7fe700 1 -- 172.21.15.8:0/2848065793 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f60f0014070 con 0x7f6100101540 2023-11-27T15:24:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.901+0000 7f60fdd9b700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60e80a11e0 0x7f60e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:15.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.901+0000 7f60fdd9b700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60e80a11e0 0x7f60e80a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6100100990 tx=0x7f60f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:15.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:15.904+0000 7f60ef7fe700 1 -- 172.21.15.8:0/2848065793 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f60f0051730 con 0x7f6100101540 2023-11-27T15:24:16.118 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.114+0000 7f6104ec8700 1 -- 172.21.15.8:0/2848065793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61001104f0 con 0x7f6100101540 2023-11-27T15:24:16.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.116+0000 7f60ef7fe700 1 -- 172.21.15.8:0/2848065793 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23817 (secure 0 0 0) 0x7f60f0099ce0 con 0x7f6100101540 2023-11-27T15:24:16.120 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:16.120 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":88,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:23:32.146115+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:24:16.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.118+0000 7f6104ec8700 1 -- 172.21.15.8:0/2848065793 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60e80a11e0 msgr2=0x7f60e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:16.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.118+0000 7f6104ec8700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60e80a11e0 0x7f60e80a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6100100990 tx=0x7f60f4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.118+0000 7f6104ec8700 1 -- 172.21.15.8:0/2848065793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6100101540 msgr2=0x7f61000ff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:16.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.118+0000 7f6104ec8700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6100101540 0x7f61000ff1a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f60f000e9a0 tx=0x7f60f000ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.118+0000 7f6104ec8700 1 -- 172.21.15.8:0/2848065793 shutdown_connections 2023-11-27T15:24:16.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.118+0000 7f6104ec8700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60e80a11e0 0x7f60e80a3690 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.118+0000 7f6104ec8700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f610010a890 0x7f61000ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.119+0000 7f6104ec8700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6100101540 0x7f61000ff1a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.119+0000 7f6104ec8700 1 --2- 172.21.15.8:0/2848065793 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6100101e50 0x7f61000ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.119+0000 7f6104ec8700 1 -- 172.21.15.8:0/2848065793 >> 172.21.15.8:0/2848065793 conn(0x7f61000770a0 msgr2=0x7f6100102260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:16.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.119+0000 7f6104ec8700 1 -- 172.21.15.8:0/2848065793 shutdown_connections 2023-11-27T15:24:16.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:16.119+0000 7f6104ec8700 1 -- 172.21.15.8:0/2848065793 wait complete. 2023-11-27T15:24:16.551 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-11-27T15:24:16.551 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-11-27T15:24:16.552 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-11-27T15:24:16.552 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-11-27T15:24:16.553 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-11-27T15:24:16.553 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-11-27T15:24:16.554 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-11-27T15:24:16.554 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-11-27T15:24:16.555 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-11-27T15:24:16.555 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-11-27T15:24:16.555 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-11-27T15:24:16.556 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-11-27T15:24:16.685 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:16 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2848065793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:16.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:16 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2848065793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:16.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:16 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2848065793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:17.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:17 smithi060 ceph-mon[131518]: pgmap v307: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2023-11-27T15:24:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:17 smithi008 ceph-mon[128788]: pgmap v307: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2023-11-27T15:24:17.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:17 smithi029 ceph-mon[131166]: pgmap v307: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2023-11-27T15:24:18.924 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:19.135 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:19.149 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:19.330 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:19.377 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:19.435 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:19.435 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:19.572 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:19 smithi008 ceph-mon[128788]: pgmap v308: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2023-11-27T15:24:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:19 smithi060 ceph-mon[131518]: pgmap v308: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2023-11-27T15:24:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:19 smithi029 ceph-mon[131166]: pgmap v308: 129 pgs: 129 active+clean; 636 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2023-11-27T15:24:19.702 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:19.755 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:19.933 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:19.934 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:20.430 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:21.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:21 smithi029 ceph-mon[131166]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:24:21.706 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:21 smithi008 ceph-mon[128788]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:24:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:21 smithi060 ceph-mon[131518]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:24:23.424 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:23 smithi008 ceph-mon[128788]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-27T15:24:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:23 smithi060 ceph-mon[131518]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-27T15:24:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:23 smithi029 ceph-mon[131166]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.121+0000 7faca1ce5700 1 -- 172.21.15.8:0/2758008291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac9c1055e0 msgr2=0x7fac9c10d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.121+0000 7faca1ce5700 1 --2- 172.21.15.8:0/2758008291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac9c1055e0 0x7fac9c10d160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fac9400b0d0 tx=0x7fac9400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.121+0000 7faca1ce5700 1 -- 172.21.15.8:0/2758008291 shutdown_connections 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.121+0000 7faca1ce5700 1 --2- 172.21.15.8:0/2758008291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac9c1055e0 0x7fac9c10d160 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.121+0000 7faca1ce5700 1 --2- 172.21.15.8:0/2758008291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c100540 0x7fac9c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.121+0000 7faca1ce5700 1 --2- 172.21.15.8:0/2758008291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac9c100ee0 0x7fac9c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.121+0000 7faca1ce5700 1 -- 172.21.15.8:0/2758008291 >> 172.21.15.8:0/2758008291 conn(0x7fac9c0fbdc0 msgr2=0x7fac9c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.121+0000 7faca1ce5700 1 -- 172.21.15.8:0/2758008291 shutdown_connections 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.121+0000 7faca1ce5700 1 -- 172.21.15.8:0/2758008291 wait complete. 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.122+0000 7faca1ce5700 1 Processor -- start 2023-11-27T15:24:24.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.122+0000 7faca1ce5700 1 -- start start 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7faca1ce5700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac9c100540 0x7fac9c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7faca1ce5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac9c100ee0 0x7fac9c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7faca1ce5700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c1055e0 0x7fac9c19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7faca1ce5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c19c5c0 con 0x7fac9c100ee0 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7faca1ce5700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c19c730 con 0x7fac9c1055e0 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7faca1ce5700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c19c8f0 con 0x7fac9c100540 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7fac9b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac9c100540 0x7fac9c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7fac9affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac9c100ee0 0x7fac9c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7fac9b7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac9c100540 0x7fac9c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58778/0 (socket says 172.21.15.8:58778) 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7fac9b7fe700 1 -- 172.21.15.8:0/2203252556 learned_addr learned my addr 172.21.15.8:0/2203252556 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7fac9bfff700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c1055e0 0x7fac9c19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7fac9b7fe700 1 -- 172.21.15.8:0/2203252556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c1055e0 msgr2=0x7fac9c19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7fac9b7fe700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fac9c1055e0 0x7fac9c19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7fac9b7fe700 1 -- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac9c100ee0 msgr2=0x7fac9c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.123+0000 7fac9b7fe700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac9c100ee0 0x7fac9c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.124+0000 7fac9b7fe700 1 -- 172.21.15.8:0/2203252556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac94009d00 con 0x7fac9c100540 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.124+0000 7fac9affd700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fac9c100ee0 0x7fac9c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.124+0000 7fac9b7fe700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac9c100540 0x7fac9c198710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00e9a0 tx=0x7fac8c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.124+0000 7fac98ff9700 1 -- 172.21.15.8:0/2203252556 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c009790 con 0x7fac9c100540 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.124+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c110370 con 0x7fac9c100540 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.124+0000 7fac98ff9700 1 -- 172.21.15.8:0/2203252556 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac8c010430 con 0x7fac9c100540 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.124+0000 7fac98ff9700 1 -- 172.21.15.8:0/2203252556 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c018680 con 0x7fac9c100540 2023-11-27T15:24:24.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.124+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c1101e0 con 0x7fac9c100540 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.125+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac9c10c0b0 con 0x7fac9c100540 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.127+0000 7fac98ff9700 1 -- 172.21.15.8:0/2203252556 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fac8c0105a0 con 0x7fac9c100540 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.129+0000 7fac98ff9700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac840a11e0 0x7fac840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.129+0000 7fac98ff9700 1 -- 172.21.15.8:0/2203252556 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fac8c014070 con 0x7fac9c100540 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.129+0000 7fac9affd700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac840a11e0 0x7fac840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.129+0000 7fac98ff9700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] conn(0x7fac840abd50 0x7fac840ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.129+0000 7fac98ff9700 1 -- 172.21.15.8:0/2203252556 --> [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac840ae850 con 0x7fac840abd50 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.129+0000 7fac98ff9700 1 -- 172.21.15.8:0/2203252556 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac8c0cd4f0 con 0x7fac9c100540 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.130+0000 7fac9bfff700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] conn(0x7fac840abd50 0x7fac840ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.130+0000 7fac9affd700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac840a11e0 0x7fac840a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac9c199ce0 tx=0x7fac90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.130+0000 7fac9bfff700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] conn(0x7fac840abd50 0x7fac840ae180 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:24.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.131+0000 7fac98ff9700 1 -- 172.21.15.8:0/2203252556 <== osd.0 v2:172.21.15.8:6802/225604885 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fac840ae850 con 0x7fac840abd50 2023-11-27T15:24:24.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.148+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 --> [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fac9c110500 con 0x7fac840abd50 2023-11-27T15:24:24.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.149+0000 7fac98ff9700 1 -- 172.21.15.8:0/2203252556 <== osd.0 v2:172.21.15.8:6802/225604885 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fac9c110500 con 0x7fac840abd50 2023-11-27T15:24:24.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.149+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] conn(0x7fac840abd50 msgr2=0x7fac840ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:24.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.149+0000 7faca1ce5700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] conn(0x7fac840abd50 0x7fac840ae180 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:24.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.149+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac840a11e0 msgr2=0x7fac840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:24.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.149+0000 7faca1ce5700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac840a11e0 0x7fac840a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fac9c199ce0 tx=0x7fac90009260 comp rx=0 tx=0).stop 2023-11-27T15:24:24.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.149+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac9c100540 msgr2=0x7fac9c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:24.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.149+0000 7faca1ce5700 1 --2- 172.21.15.8:0/2203252556 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fac9c100540 0x7fac9c198710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00e9a0 tx=0x7fac8c00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:24.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.149+0000 7fac9bfff700 1 -- 172.21.15.8:0/2203252556 reap_dead start 2023-11-27T15:24:24.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.150+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 shutdown_connections 2023-11-27T15:24:24.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.150+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 >> 172.21.15.8:0/2203252556 conn(0x7fac9c0fbdc0 msgr2=0x7fac9c10b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:24.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.150+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 shutdown_connections 2023-11-27T15:24:24.649 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.150+0000 7faca1ce5700 1 -- 172.21.15.8:0/2203252556 wait complete. 2023-11-27T15:24:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.366+0000 7f4c2733c700 1 -- 172.21.15.8:0/2630752673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c20111120 msgr2=0x7f4c20113500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.366+0000 7f4c2733c700 1 --2- 172.21.15.8:0/2630752673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c20111120 0x7f4c20113500 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800b0a0 tx=0x7f4c1800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.366+0000 7f4c2733c700 1 -- 172.21.15.8:0/2630752673 shutdown_connections 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.366+0000 7f4c2733c700 1 --2- 172.21.15.8:0/2630752673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c20111120 0x7f4c20113500 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.366+0000 7f4c2733c700 1 --2- 172.21.15.8:0/2630752673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c20072e90 0x7f4c20073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.366+0000 7f4c2733c700 1 --2- 172.21.15.8:0/2630752673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c200737a0 0x7f4c20110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.366+0000 7f4c2733c700 1 -- 172.21.15.8:0/2630752673 >> 172.21.15.8:0/2630752673 conn(0x7f4c200fbbd0 msgr2=0x7f4c200fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.367+0000 7f4c2733c700 1 -- 172.21.15.8:0/2630752673 shutdown_connections 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.367+0000 7f4c2733c700 1 -- 172.21.15.8:0/2630752673 wait complete. 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.368+0000 7f4c2733c700 1 Processor -- start 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.368+0000 7f4c2733c700 1 -- start start 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.368+0000 7f4c2733c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c20072e90 0x7f4c201a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.368+0000 7f4c2733c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c200737a0 0x7f4c201a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.368+0000 7f4c2733c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c20111120 0x7f4c2019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.368+0000 7f4c2733c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2019d2b0 con 0x7f4c200737a0 2023-11-27T15:24:25.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.368+0000 7f4c2733c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2019d420 con 0x7f4c20111120 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.368+0000 7f4c2733c700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2019d5e0 con 0x7f4c20072e90 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c25b39700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c200737a0 0x7f4c201a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c2633a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c20072e90 0x7f4c201a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c25b39700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c200737a0 0x7f4c201a2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60062/0 (socket says 172.21.15.8:60062) 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c25b39700 1 -- 172.21.15.8:0/1784116371 learned_addr learned my addr 172.21.15.8:0/1784116371 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c26b3b700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c20111120 0x7f4c2019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c2633a700 1 -- 172.21.15.8:0/1784116371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c20111120 msgr2=0x7f4c2019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c2633a700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4c20111120 0x7f4c2019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c2633a700 1 -- 172.21.15.8:0/1784116371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c200737a0 msgr2=0x7f4c201a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c2633a700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c200737a0 0x7f4c201a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c2633a700 1 -- 172.21.15.8:0/1784116371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c18009d00 con 0x7f4c20072e90 2023-11-27T15:24:25.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c25b39700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c200737a0 0x7f4c201a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.369+0000 7f4c2633a700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c20072e90 0x7f4c201a2a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00e9f0 tx=0x7f4c1c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.370+0000 7f4c177fe700 1 -- 172.21.15.8:0/1784116371 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c00c9c0 con 0x7f4c20072e90 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.370+0000 7f4c177fe700 1 -- 172.21.15.8:0/1784116371 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c1c010430 con 0x7f4c20072e90 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.370+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c201167e0 con 0x7f4c20072e90 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.370+0000 7f4c177fe700 1 -- 172.21.15.8:0/1784116371 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c018680 con 0x7f4c20072e90 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.370+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c20116650 con 0x7f4c20072e90 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.370+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c200611b0 con 0x7f4c20072e90 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.373+0000 7f4c177fe700 1 -- 172.21.15.8:0/1784116371 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f4c1c010860 con 0x7f4c20072e90 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.374+0000 7f4c177fe700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4c0c0a11e0 0x7f4c0c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.374+0000 7f4c177fe700 1 -- 172.21.15.8:0/1784116371 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4c1c014070 con 0x7f4c20072e90 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.374+0000 7f4c25b39700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4c0c0a11e0 0x7f4c0c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.375+0000 7f4c177fe700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] conn(0x7f4c0c0abd50 0x7f4c0c0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.171 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.375+0000 7f4c177fe700 1 -- 172.21.15.8:0/1784116371 --> [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c0c0ae850 con 0x7f4c0c0abd50 2023-11-27T15:24:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.375+0000 7f4c177fe700 1 -- 172.21.15.8:0/1784116371 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c1c0cd430 con 0x7f4c20072e90 2023-11-27T15:24:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.375+0000 7f4c26b3b700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] conn(0x7f4c0c0abd50 0x7f4c0c0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.375+0000 7f4c25b39700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4c0c0a11e0 0x7f4c0c0a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4c2019e290 tx=0x7f4c10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.376+0000 7f4c26b3b700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] conn(0x7f4c0c0abd50 0x7f4c0c0ae180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.376+0000 7f4c177fe700 1 -- 172.21.15.8:0/1784116371 <== osd.1 v2:172.21.15.29:6800/3314882267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4c0c0ae850 con 0x7f4c0c0abd50 2023-11-27T15:24:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.393+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 --> [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c20116970 con 0x7f4c0c0abd50 2023-11-27T15:24:25.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c177fe700 1 -- 172.21.15.8:0/1784116371 <== osd.1 v2:172.21.15.29:6800/3314882267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4c20116970 con 0x7f4c0c0abd50 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 >> [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] conn(0x7f4c0c0abd50 msgr2=0x7f4c0c0ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c2733c700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] conn(0x7f4c0c0abd50 0x7f4c0c0ae180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4c0c0a11e0 msgr2=0x7f4c0c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c2733c700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4c0c0a11e0 0x7f4c0c0a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4c2019e290 tx=0x7f4c10009260 comp rx=0 tx=0).stop 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c20072e90 msgr2=0x7f4c201a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c2733c700 1 --2- 172.21.15.8:0/1784116371 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4c20072e90 0x7f4c201a2a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00e9f0 tx=0x7f4c1c00ed00 comp rx=0 tx=0).stop 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c26b3b700 1 -- 172.21.15.8:0/1784116371 reap_dead start 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 shutdown_connections 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 >> 172.21.15.8:0/1784116371 conn(0x7f4c200fbbd0 msgr2=0x7f4c200fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 shutdown_connections 2023-11-27T15:24:25.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:24.394+0000 7f4c2733c700 1 -- 172.21.15.8:0/1784116371 wait complete. 2023-11-27T15:24:25.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:25 smithi008 ceph-mon[128788]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-27T15:24:25.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:25 smithi060 ceph-mon[131518]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-27T15:24:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:25 smithi029 ceph-mon[131166]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-27T15:24:26.342 INFO:teuthology.orchestra.run.smithi008.stdout:55834574919 2023-11-27T15:24:26.343 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-11-27T15:24:27.052 INFO:teuthology.orchestra.run.smithi008.stdout:34359738444 2023-11-27T15:24:27.052 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-11-27T15:24:27.564 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:27 smithi008 ceph-mon[128788]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-27T15:24:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:27 smithi060 ceph-mon[131518]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-27T15:24:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:27 smithi029 ceph-mon[131166]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-27T15:24:28.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.642+0000 7f66b5481700 1 -- 172.21.15.8:0/923646389 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66b01055e0 msgr2=0x7f66b010d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.051 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.642+0000 7f66b5481700 1 --2- 172.21.15.8:0/923646389 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66b01055e0 0x7f66b010d160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66a800b0d0 tx=0x7f66a800b3e0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.642+0000 7f66b5481700 1 -- 172.21.15.8:0/923646389 shutdown_connections 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.642+0000 7f66b5481700 1 --2- 172.21.15.8:0/923646389 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b0100540 0x7f66b0100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.642+0000 7f66b5481700 1 --2- 172.21.15.8:0/923646389 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66b01055e0 0x7f66b010d160 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.642+0000 7f66b5481700 1 --2- 172.21.15.8:0/923646389 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b0100ee0 0x7f66b0104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.642+0000 7f66b5481700 1 -- 172.21.15.8:0/923646389 >> 172.21.15.8:0/923646389 conn(0x7f66b00fbdc0 msgr2=0x7f66b00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.642+0000 7f66b5481700 1 -- 172.21.15.8:0/923646389 shutdown_connections 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.642+0000 7f66b5481700 1 -- 172.21.15.8:0/923646389 wait complete. 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.643+0000 7f66b5481700 1 Processor -- start 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.643+0000 7f66b5481700 1 -- start start 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.643+0000 7f66b5481700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66b0100540 0x7f66b01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.052 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.643+0000 7f66b5481700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b0100ee0 0x7f66b0198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.643+0000 7f66b5481700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b01055e0 0x7f66b019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.643+0000 7f66b5481700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b019c5a0 con 0x7f66b0100540 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.643+0000 7f66b5481700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b019c710 con 0x7f66b01055e0 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.643+0000 7f66b5481700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b019c8d0 con 0x7f66b0100ee0 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66aeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66b0100540 0x7f66b01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66aeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66b0100540 0x7f66b01986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60092/0 (socket says 172.21.15.8:60092) 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66aeffd700 1 -- 172.21.15.8:0/140416958 learned_addr learned my addr 172.21.15.8:0/140416958 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66ae7fc700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b0100ee0 0x7f66b0198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66af7fe700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b01055e0 0x7f66b019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66aeffd700 1 -- 172.21.15.8:0/140416958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b0100ee0 msgr2=0x7f66b0198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66aeffd700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f66b0100ee0 0x7f66b0198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66aeffd700 1 -- 172.21.15.8:0/140416958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b01055e0 msgr2=0x7f66b019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.053 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66aeffd700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b01055e0 0x7f66b019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66aeffd700 1 -- 172.21.15.8:0/140416958 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a8009d00 con 0x7f66b0100540 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66af7fe700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f66b01055e0 0x7f66b019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f66aeffd700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66b0100540 0x7f66b01986a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e9a0 tx=0x7f66a000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.644+0000 7f6697fff700 1 -- 172.21.15.8:0/140416958 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0009790 con 0x7f66b0100540 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.645+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b0110320 con 0x7f66b0100540 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.645+0000 7f6697fff700 1 -- 172.21.15.8:0/140416958 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a000cbd0 con 0x7f66b0100540 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.645+0000 7f6697fff700 1 -- 172.21.15.8:0/140416958 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0018680 con 0x7f66b0100540 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.645+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b0110190 con 0x7f66b0100540 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.646+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f66b010c070 con 0x7f66b0100540 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.648+0000 7f6697fff700 1 -- 172.21.15.8:0/140416958 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f66a000cd40 con 0x7f66b0100540 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.649+0000 7f6697fff700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66980a11e0 0x7f66980a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.054 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.649+0000 7f6697fff700 1 -- 172.21.15.8:0/140416958 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f66a0014070 con 0x7f66b0100540 2023-11-27T15:24:28.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.649+0000 7f66ae7fc700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66980a11e0 0x7f66980a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.649+0000 7f6697fff700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] conn(0x7f66980abd50 0x7f66980ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.649+0000 7f6697fff700 1 -- 172.21.15.8:0/140416958 --> [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66980ae850 con 0x7f66980abd50 2023-11-27T15:24:28.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.649+0000 7f6697fff700 1 -- 172.21.15.8:0/140416958 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66a00cda00 con 0x7f66b0100540 2023-11-27T15:24:28.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.650+0000 7f66af7fe700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] conn(0x7f66980abd50 0x7f66980ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.650+0000 7f66ae7fc700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66980a11e0 0x7f66980a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f66b0199cc0 tx=0x7f66a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.650+0000 7f66af7fe700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] conn(0x7f66980abd50 0x7f66980ae180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.651+0000 7f6697fff700 1 -- 172.21.15.8:0/140416958 <== osd.3 v2:172.21.15.8:6810/709452183 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f66980ae850 con 0x7f66980abd50 2023-11-27T15:24:28.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.668+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 --> [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f66b01104b0 con 0x7f66980abd50 2023-11-27T15:24:28.055 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.668+0000 7f6697fff700 1 -- 172.21.15.8:0/140416958 <== osd.3 v2:172.21.15.8:6810/709452183 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f66b01104b0 con 0x7f66980abd50 2023-11-27T15:24:28.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] conn(0x7f66980abd50 msgr2=0x7f66980ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66b5481700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] conn(0x7f66980abd50 0x7f66980ae180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66980a11e0 msgr2=0x7f66980a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66b5481700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f66980a11e0 0x7f66980a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f66b0199cc0 tx=0x7f66a4009260 comp rx=0 tx=0).stop 2023-11-27T15:24:28.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66b0100540 msgr2=0x7f66b01986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66b5481700 1 --2- 172.21.15.8:0/140416958 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66b0100540 0x7f66b01986a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66a000e9a0 tx=0x7f66a000ed60 comp rx=0 tx=0).stop 2023-11-27T15:24:28.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66af7fe700 1 -- 172.21.15.8:0/140416958 reap_dead start 2023-11-27T15:24:28.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 shutdown_connections 2023-11-27T15:24:28.056 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 >> 172.21.15.8:0/140416958 conn(0x7f66b00fbdc0 msgr2=0x7f66b010b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:28.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 shutdown_connections 2023-11-27T15:24:28.057 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:25.669+0000 7f66b5481700 1 -- 172.21.15.8:0/140416958 wait complete. 2023-11-27T15:24:28.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.517+0000 7f04345f1700 1 -- 172.21.15.8:0/3378899233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042c101540 msgr2=0x7f042c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.517+0000 7f04345f1700 1 --2- 172.21.15.8:0/3378899233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042c101540 0x7f042c101910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b0a0 tx=0x7f041c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.517+0000 7f04345f1700 1 -- 172.21.15.8:0/3378899233 shutdown_connections 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.517+0000 7f04345f1700 1 --2- 172.21.15.8:0/3378899233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042c101540 0x7f042c101910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.517+0000 7f04345f1700 1 --2- 172.21.15.8:0/3378899233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f042c10a890 0x7f042c10ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.517+0000 7f04345f1700 1 --2- 172.21.15.8:0/3378899233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f042c101e50 0x7f042c10a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.517+0000 7f04345f1700 1 -- 172.21.15.8:0/3378899233 >> 172.21.15.8:0/3378899233 conn(0x7f042c0770a0 msgr2=0x7f042c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.517+0000 7f04345f1700 1 -- 172.21.15.8:0/3378899233 shutdown_connections 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.517+0000 7f04345f1700 1 -- 172.21.15.8:0/3378899233 wait complete. 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.518+0000 7f04345f1700 1 Processor -- start 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.518+0000 7f04345f1700 1 -- start start 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f04345f1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f042c101540 0x7f042c19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f04345f1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042c101e50 0x7f042c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f04345f1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f042c10a890 0x7f042c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f04345f1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c198cf0 con 0x7f042c101540 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f04345f1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c198e60 con 0x7f042c10a890 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f04345f1700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c199020 con 0x7f042c101e50 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f043238d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f042c101540 0x7f042c19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f0431b8c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042c101e50 0x7f042c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f0432b8e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f042c10a890 0x7f042c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f0431b8c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042c101e50 0x7f042c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58838/0 (socket says 172.21.15.8:58838) 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f0431b8c700 1 -- 172.21.15.8:0/2896310517 learned_addr learned my addr 172.21.15.8:0/2896310517 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f0431b8c700 1 -- 172.21.15.8:0/2896310517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f042c10a890 msgr2=0x7f042c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.519+0000 7f0431b8c700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f042c10a890 0x7f042c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f0431b8c700 1 -- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f042c101540 msgr2=0x7f042c19e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f0431b8c700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f042c101540 0x7f042c19e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f0431b8c700 1 -- 172.21.15.8:0/2896310517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c009d00 con 0x7f042c101e50 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f0432b8e700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f042c10a890 0x7f042c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f043238d700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f042c101540 0x7f042c19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f0431b8c700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042c101e50 0x7f042c19eb40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f042400e9a0 tx=0x7f042400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f042b7fe700 1 -- 172.21.15.8:0/2896310517 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424009790 con 0x7f042c101e50 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c110360 con 0x7f042c101e50 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f042b7fe700 1 -- 172.21.15.8:0/2896310517 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f042400ce80 con 0x7f042c101e50 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f042b7fe700 1 -- 172.21.15.8:0/2896310517 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424018680 con 0x7f042c101e50 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.520+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c1101d0 con 0x7f042c101e50 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.521+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f042c100b90 con 0x7f042c101e50 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.523+0000 7f042b7fe700 1 -- 172.21.15.8:0/2896310517 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f042400c9c0 con 0x7f042c101e50 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.524+0000 7f042b7fe700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f04140a11e0 0x7f04140a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.525+0000 7f042b7fe700 1 -- 172.21.15.8:0/2896310517 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0424014070 con 0x7f042c101e50 2023-11-27T15:24:28.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.525+0000 7f042b7fe700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] conn(0x7f04140abd50 0x7f04140ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.525+0000 7f042b7fe700 1 -- 172.21.15.8:0/2896310517 --> [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04140ae850 con 0x7f04140abd50 2023-11-27T15:24:28.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.525+0000 7f042b7fe700 1 -- 172.21.15.8:0/2896310517 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04240cf2e0 con 0x7f042c101e50 2023-11-27T15:24:28.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.525+0000 7f043238d700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f04140a11e0 0x7f04140a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.525+0000 7f0432b8e700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] conn(0x7f04140abd50 0x7f04140ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.526+0000 7f0432b8e700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] conn(0x7f04140abd50 0x7f04140ae180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.526+0000 7f043238d700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f04140a11e0 0x7f04140a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f041c009fd0 tx=0x7f041c006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.527+0000 7f042b7fe700 1 -- 172.21.15.8:0/2896310517 <== osd.6 v2:172.21.15.8:6818/1285430534 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f04140ae850 con 0x7f04140abd50 2023-11-27T15:24:28.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.544+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 --> [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f042c1104f0 con 0x7f04140abd50 2023-11-27T15:24:28.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.545+0000 7f042b7fe700 1 -- 172.21.15.8:0/2896310517 <== osd.6 v2:172.21.15.8:6818/1285430534 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f042c1104f0 con 0x7f04140abd50 2023-11-27T15:24:28.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.545+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] conn(0x7f04140abd50 msgr2=0x7f04140ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.545+0000 7f04345f1700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] conn(0x7f04140abd50 0x7f04140ae180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.545+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f04140a11e0 msgr2=0x7f04140a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.545+0000 7f04345f1700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f04140a11e0 0x7f04140a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f041c009fd0 tx=0x7f041c006210 comp rx=0 tx=0).stop 2023-11-27T15:24:28.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.545+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042c101e50 msgr2=0x7f042c19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.545+0000 7f04345f1700 1 --2- 172.21.15.8:0/2896310517 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f042c101e50 0x7f042c19eb40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f042400e9a0 tx=0x7f042400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.546+0000 7f0432b8e700 1 -- 172.21.15.8:0/2896310517 reap_dead start 2023-11-27T15:24:28.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.546+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 shutdown_connections 2023-11-27T15:24:28.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.546+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 >> 172.21.15.8:0/2896310517 conn(0x7f042c0770a0 msgr2=0x7f042c1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:28.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.546+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 shutdown_connections 2023-11-27T15:24:28.257 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:27.546+0000 7f04345f1700 1 -- 172.21.15.8:0/2896310517 wait complete. 2023-11-27T15:24:28.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.121+0000 7fe8f815b700 1 -- 172.21.15.8:0/52077033 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0100540 msgr2=0x7fe8f0100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.121+0000 7fe8f815b700 1 --2- 172.21.15.8:0/52077033 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0100540 0x7fe8f0100910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0099c0 tx=0x7fe8ec009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.121+0000 7fe8f815b700 1 -- 172.21.15.8:0/52077033 shutdown_connections 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.121+0000 7fe8f815b700 1 --2- 172.21.15.8:0/52077033 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f01055e0 0x7fe8f010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.121+0000 7fe8f815b700 1 --2- 172.21.15.8:0/52077033 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0100540 0x7fe8f0100910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.121+0000 7fe8f815b700 1 --2- 172.21.15.8:0/52077033 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0100ee0 0x7fe8f0104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.121+0000 7fe8f815b700 1 -- 172.21.15.8:0/52077033 >> 172.21.15.8:0/52077033 conn(0x7fe8f00fbdc0 msgr2=0x7fe8f00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.122+0000 7fe8f815b700 1 -- 172.21.15.8:0/52077033 shutdown_connections 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.122+0000 7fe8f815b700 1 -- 172.21.15.8:0/52077033 wait complete. 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f815b700 1 Processor -- start 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f815b700 1 -- start start 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f815b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0100540 0x7fe8f0198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f815b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0100ee0 0x7fe8f0198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f815b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f01055e0 0x7fe8f019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f815b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f019d620 con 0x7fe8f0100540 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f815b700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f019d790 con 0x7fe8f01055e0 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f815b700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f019d950 con 0x7fe8f0100ee0 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f5ef7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0100540 0x7fe8f0198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f5ef7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0100540 0x7fe8f0198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60126/0 (socket says 172.21.15.8:60126) 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.123+0000 7fe8f5ef7700 1 -- 172.21.15.8:0/3318487256 learned_addr learned my addr 172.21.15.8:0/3318487256 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8f66f8700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f01055e0 0x7fe8f019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8f56f6700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0100ee0 0x7fe8f0198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8f5ef7700 1 -- 172.21.15.8:0/3318487256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0100ee0 msgr2=0x7fe8f0198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.628 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8f5ef7700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe8f0100ee0 0x7fe8f0198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8f5ef7700 1 -- 172.21.15.8:0/3318487256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f01055e0 msgr2=0x7fe8f019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8f5ef7700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe8f01055e0 0x7fe8f019d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8f5ef7700 1 -- 172.21.15.8:0/3318487256 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec0095f0 con 0x7fe8f0100540 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8f5ef7700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0100540 0x7fe8f0198660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec006010 tx=0x7fe8ec005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8e6ffd700 1 -- 172.21.15.8:0/3318487256 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec01d070 con 0x7fe8f0100540 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8e6ffd700 1 -- 172.21.15.8:0/3318487256 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8ec004120 con 0x7fe8f0100540 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f0110370 con 0x7fe8f0100540 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.124+0000 7fe8e6ffd700 1 -- 172.21.15.8:0/3318487256 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec005290 con 0x7fe8f0100540 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.125+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f01101e0 con 0x7fe8f0100540 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.126+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8f0073ff0 con 0x7fe8f0100540 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.128+0000 7fe8e6ffd700 1 -- 172.21.15.8:0/3318487256 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe8ec0048e0 con 0x7fe8f0100540 2023-11-27T15:24:28.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.131+0000 7fe8e6ffd700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8dc0a11e0 0x7fe8dc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.131+0000 7fe8e6ffd700 1 -- 172.21.15.8:0/3318487256 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe8ec0cdc40 con 0x7fe8f0100540 2023-11-27T15:24:28.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.131+0000 7fe8f56f6700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8dc0a11e0 0x7fe8dc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.131+0000 7fe8e6ffd700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] conn(0x7fe8dc0abc20 0x7fe8dc0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.131+0000 7fe8e6ffd700 1 -- 172.21.15.8:0/3318487256 --> [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8dc0ae720 con 0x7fe8dc0abc20 2023-11-27T15:24:28.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.131+0000 7fe8e6ffd700 1 -- 172.21.15.8:0/3318487256 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8ec0d2240 con 0x7fe8f0100540 2023-11-27T15:24:28.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.132+0000 7fe8f56f6700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8dc0a11e0 0x7fe8dc0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005950 tx=0x7fe8e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.132+0000 7fe8f66f8700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] conn(0x7fe8dc0abc20 0x7fe8dc0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.132+0000 7fe8f66f8700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] conn(0x7fe8dc0abc20 0x7fe8dc0ae050 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.133+0000 7fe8e6ffd700 1 -- 172.21.15.8:0/3318487256 <== osd.11 v2:172.21.15.60:6824/1307710245 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe8dc0ae720 con 0x7fe8dc0abc20 2023-11-27T15:24:28.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.157+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 --> [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8f0110500 con 0x7fe8dc0abc20 2023-11-27T15:24:28.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.157+0000 7fe8e6ffd700 1 -- 172.21.15.8:0/3318487256 <== osd.11 v2:172.21.15.60:6824/1307710245 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe8f0110500 con 0x7fe8dc0abc20 2023-11-27T15:24:28.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 >> [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] conn(0x7fe8dc0abc20 msgr2=0x7fe8dc0ae050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f815b700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] conn(0x7fe8dc0abc20 0x7fe8dc0ae050 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8dc0a11e0 msgr2=0x7fe8dc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f815b700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe8dc0a11e0 0x7fe8dc0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005950 tx=0x7fe8e00058e0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0100540 msgr2=0x7fe8f0198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f815b700 1 --2- 172.21.15.8:0/3318487256 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0100540 0x7fe8f0198660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec006010 tx=0x7fe8ec005ef0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f66f8700 1 -- 172.21.15.8:0/3318487256 reap_dead start 2023-11-27T15:24:28.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 shutdown_connections 2023-11-27T15:24:28.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 >> 172.21.15.8:0/3318487256 conn(0x7fe8f00fbdc0 msgr2=0x7fe8f00fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:28.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 shutdown_connections 2023-11-27T15:24:28.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.158+0000 7fe8f815b700 1 -- 172.21.15.8:0/3318487256 wait complete. 2023-11-27T15:24:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.634+0000 7f79eb408700 1 -- 172.21.15.8:0/3351369014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e4100ee0 msgr2=0x7f79e4104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.634+0000 7f79eb408700 1 --2- 172.21.15.8:0/3351369014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e4100ee0 0x7f79e4104eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009990 tx=0x7f79d4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.634+0000 7f79eb408700 1 -- 172.21.15.8:0/3351369014 shutdown_connections 2023-11-27T15:24:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.634+0000 7f79eb408700 1 --2- 172.21.15.8:0/3351369014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e4100540 0x7f79e4100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.634+0000 7f79eb408700 1 --2- 172.21.15.8:0/3351369014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e41055e0 0x7f79e410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.634+0000 7f79eb408700 1 --2- 172.21.15.8:0/3351369014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e4100ee0 0x7f79e4104eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.634+0000 7f79eb408700 1 -- 172.21.15.8:0/3351369014 >> 172.21.15.8:0/3351369014 conn(0x7f79e40fbdc0 msgr2=0x7f79e40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.634+0000 7f79eb408700 1 -- 172.21.15.8:0/3351369014 shutdown_connections 2023-11-27T15:24:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.634+0000 7f79eb408700 1 -- 172.21.15.8:0/3351369014 wait complete. 2023-11-27T15:24:28.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.635+0000 7f79eb408700 1 Processor -- start 2023-11-27T15:24:28.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.635+0000 7f79eb408700 1 -- start start 2023-11-27T15:24:28.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79eb408700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e4100540 0x7f79e4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79eb408700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e4100ee0 0x7f79e4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79eb408700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e41055e0 0x7f79e419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79eb408700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e419d550 con 0x7f79e4100ee0 2023-11-27T15:24:28.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79eb408700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e419d6c0 con 0x7f79e4100540 2023-11-27T15:24:28.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79eb408700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e419d880 con 0x7f79e41055e0 2023-11-27T15:24:28.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79e89a3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e4100ee0 0x7f79e4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79e89a3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e4100ee0 0x7f79e4198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60152/0 (socket says 172.21.15.8:60152) 2023-11-27T15:24:28.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79e89a3700 1 -- 172.21.15.8:0/2742521008 learned_addr learned my addr 172.21.15.8:0/2742521008 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:28.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79e91a4700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e4100540 0x7f79e4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.636+0000 7f79e99a5700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e41055e0 0x7f79e419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79e91a4700 1 -- 172.21.15.8:0/2742521008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e41055e0 msgr2=0x7f79e419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79e91a4700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f79e41055e0 0x7f79e419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79e91a4700 1 -- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e4100ee0 msgr2=0x7f79e4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79e91a4700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79e4100ee0 0x7f79e4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79e91a4700 1 -- 172.21.15.8:0/2742521008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc009c40 con 0x7f79e4100540 2023-11-27T15:24:28.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79e91a4700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e4100540 0x7f79e4198590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79e000e9f0 tx=0x7f79e000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79da7fc700 1 -- 172.21.15.8:0/2742521008 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000c9c0 con 0x7f79e4100540 2023-11-27T15:24:28.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d40095f0 con 0x7f79e4100540 2023-11-27T15:24:28.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79da7fc700 1 -- 172.21.15.8:0/2742521008 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79e0010430 con 0x7f79e4100540 2023-11-27T15:24:28.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79da7fc700 1 -- 172.21.15.8:0/2742521008 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0018680 con 0x7f79e4100540 2023-11-27T15:24:28.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.637+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e4110310 con 0x7f79e4100540 2023-11-27T15:24:28.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.638+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79e4074070 con 0x7f79e4100540 2023-11-27T15:24:28.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.640+0000 7f79da7fc700 1 -- 172.21.15.8:0/2742521008 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f79e0010860 con 0x7f79e4100540 2023-11-27T15:24:28.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.642+0000 7f79da7fc700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f79d00a11e0 0x7f79d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.642+0000 7f79da7fc700 1 -- 172.21.15.8:0/2742521008 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f79e0014070 con 0x7f79e4100540 2023-11-27T15:24:28.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.642+0000 7f79e89a3700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f79d00a11e0 0x7f79d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.642+0000 7f79da7fc700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] conn(0x7f79d00abd50 0x7f79d00ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:28.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.642+0000 7f79da7fc700 1 -- 172.21.15.8:0/2742521008 --> [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79d00ae850 con 0x7f79d00abd50 2023-11-27T15:24:28.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.642+0000 7f79da7fc700 1 -- 172.21.15.8:0/2742521008 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79e00cd650 con 0x7f79e4100540 2023-11-27T15:24:28.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.643+0000 7f79e99a5700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] conn(0x7f79d00abd50 0x7f79d00ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:28.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.643+0000 7f79e89a3700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f79d00a11e0 0x7f79d00a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79d4005cd0 tx=0x7f79d4005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.643+0000 7f79e99a5700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] conn(0x7f79d00abd50 0x7f79d00ae180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:28.648 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.644+0000 7f79da7fc700 1 -- 172.21.15.8:0/2742521008 <== osd.9 v2:172.21.15.8:6826/2087690159 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f79d00ae850 con 0x7f79d00abd50 2023-11-27T15:24:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.662+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 --> [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f79e4110180 con 0x7f79d00abd50 2023-11-27T15:24:28.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.662+0000 7f79da7fc700 1 -- 172.21.15.8:0/2742521008 <== osd.9 v2:172.21.15.8:6826/2087690159 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f79e4110180 con 0x7f79d00abd50 2023-11-27T15:24:28.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.663+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] conn(0x7f79d00abd50 msgr2=0x7f79d00ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.663+0000 7f79eb408700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] conn(0x7f79d00abd50 0x7f79d00ae180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.663+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f79d00a11e0 msgr2=0x7f79d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.663+0000 7f79eb408700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f79d00a11e0 0x7f79d00a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f79d4005cd0 tx=0x7f79d4005bc0 comp rx=0 tx=0).stop 2023-11-27T15:24:28.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.663+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e4100540 msgr2=0x7f79e4198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:28.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.663+0000 7f79eb408700 1 --2- 172.21.15.8:0/2742521008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79e4100540 0x7f79e4198590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79e000e9f0 tx=0x7f79e000ed00 comp rx=0 tx=0).stop 2023-11-27T15:24:28.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.663+0000 7f79e99a5700 1 -- 172.21.15.8:0/2742521008 reap_dead start 2023-11-27T15:24:28.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.663+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 shutdown_connections 2023-11-27T15:24:28.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.663+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 >> 172.21.15.8:0/2742521008 conn(0x7f79e40fbdc0 msgr2=0x7f79e40fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:28.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.663+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 shutdown_connections 2023-11-27T15:24:28.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:28.664+0000 7f79eb408700 1 -- 172.21.15.8:0/2742521008 wait complete. 2023-11-27T15:24:29.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:29 smithi060 ceph-mon[131518]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-27T15:24:29.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:29 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:24:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:29 smithi008 ceph-mon[128788]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-27T15:24:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:29 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:24:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:29 smithi029 ceph-mon[131166]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-27T15:24:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:29 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:24:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.655+0000 7fd7c8988700 1 -- 172.21.15.8:0/2617825670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c4100ee0 msgr2=0x7fd7c4104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.655+0000 7fd7c8988700 1 --2- 172.21.15.8:0/2617825670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c4100ee0 0x7fd7c4104eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009990 tx=0x7fd7b8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.655+0000 7fd7c8988700 1 -- 172.21.15.8:0/2617825670 shutdown_connections 2023-11-27T15:24:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.655+0000 7fd7c8988700 1 --2- 172.21.15.8:0/2617825670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c41055e0 0x7fd7c410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.655+0000 7fd7c8988700 1 --2- 172.21.15.8:0/2617825670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7c4100540 0x7fd7c4100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.655+0000 7fd7c8988700 1 --2- 172.21.15.8:0/2617825670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c4100ee0 0x7fd7c4104eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:29.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.655+0000 7fd7c8988700 1 -- 172.21.15.8:0/2617825670 >> 172.21.15.8:0/2617825670 conn(0x7fd7c40fbdc0 msgr2=0x7fd7c40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.655+0000 7fd7c8988700 1 -- 172.21.15.8:0/2617825670 shutdown_connections 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.655+0000 7fd7c8988700 1 -- 172.21.15.8:0/2617825670 wait complete. 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.656+0000 7fd7c8988700 1 Processor -- start 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.657+0000 7fd7c8988700 1 -- start start 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.657+0000 7fd7c8988700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c4100540 0x7fd7c4072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.657+0000 7fd7c8988700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7c4100ee0 0x7fd7c4073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.657+0000 7fd7c8988700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c41055e0 0x7fd7c40738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.657+0000 7fd7c8988700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c41a4810 con 0x7fd7c4100ee0 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.657+0000 7fd7c8988700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c41a4980 con 0x7fd7c4100540 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.657+0000 7fd7c8988700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c41a4b40 con 0x7fd7c41055e0 2023-11-27T15:24:29.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.657+0000 7fd7c259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c4100540 0x7fd7c4072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.657+0000 7fd7c259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c4100540 0x7fd7c4072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:36868/0 (socket says 172.21.15.8:36868) 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.657+0000 7fd7c259c700 1 -- 172.21.15.8:0/2946608151 learned_addr learned my addr 172.21.15.8:0/2946608151 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.658+0000 7fd7c1d9b700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7c4100ee0 0x7fd7c4073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.658+0000 7fd7c2d9d700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c41055e0 0x7fd7c40738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.658+0000 7fd7c259c700 1 -- 172.21.15.8:0/2946608151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c41055e0 msgr2=0x7fd7c40738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.658+0000 7fd7c259c700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd7c41055e0 0x7fd7c40738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.658+0000 7fd7c259c700 1 -- 172.21.15.8:0/2946608151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7c4100ee0 msgr2=0x7fd7c4073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.658+0000 7fd7c259c700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7c4100ee0 0x7fd7c4073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.658+0000 7fd7c259c700 1 -- 172.21.15.8:0/2946608151 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80095f0 con 0x7fd7c4100540 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.658+0000 7fd7c259c700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c4100540 0x7fd7c4072df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400e9a0 tx=0x7fd7b400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.658+0000 7fd7b37fe700 1 -- 172.21.15.8:0/2946608151 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4009790 con 0x7fd7c4100540 2023-11-27T15:24:29.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.658+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c4110310 con 0x7fd7c4100540 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.659+0000 7fd7b37fe700 1 -- 172.21.15.8:0/2946608151 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7b4010430 con 0x7fd7c4100540 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.659+0000 7fd7b37fe700 1 -- 172.21.15.8:0/2946608151 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4018680 con 0x7fd7c4100540 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.659+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c4110180 con 0x7fd7c4100540 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.659+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd7c40fd220 con 0x7fd7c4100540 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.662+0000 7fd7b37fe700 1 -- 172.21.15.8:0/2946608151 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd7b40105a0 con 0x7fd7c4100540 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.663+0000 7fd7b37fe700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd7ac0a1260 0x7fd7ac0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.663+0000 7fd7b37fe700 1 -- 172.21.15.8:0/2946608151 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd7b4014070 con 0x7fd7c4100540 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.663+0000 7fd7c1d9b700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd7ac0a1260 0x7fd7ac0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.663+0000 7fd7b37fe700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] conn(0x7fd7ac0abdd0 0x7fd7ac0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.663+0000 7fd7b37fe700 1 -- 172.21.15.8:0/2946608151 --> [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7ac0ae8d0 con 0x7fd7ac0abdd0 2023-11-27T15:24:29.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.663+0000 7fd7b37fe700 1 -- 172.21.15.8:0/2946608151 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b40cd430 con 0x7fd7c4100540 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.664+0000 7fd7c2d9d700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] conn(0x7fd7ac0abdd0 0x7fd7ac0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.664+0000 7fd7c1d9b700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd7ac0a1260 0x7fd7ac0a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8006010 tx=0x7fd7b8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.664+0000 7fd7c2d9d700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] conn(0x7fd7ac0abdd0 0x7fd7ac0ae200 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.665+0000 7fd7b37fe700 1 -- 172.21.15.8:0/2946608151 <== osd.2 v2:172.21.15.60:6800/3602448664 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd7ac0ae8d0 con 0x7fd7ac0abdd0 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.681+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 --> [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd7c41104a0 con 0x7fd7ac0abdd0 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.682+0000 7fd7b37fe700 1 -- 172.21.15.8:0/2946608151 <== osd.2 v2:172.21.15.60:6800/3602448664 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd7c41104a0 con 0x7fd7ac0abdd0 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.682+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 >> [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] conn(0x7fd7ac0abdd0 msgr2=0x7fd7ac0ae200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.682+0000 7fd7c8988700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] conn(0x7fd7ac0abdd0 0x7fd7ac0ae200 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.682+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd7ac0a1260 msgr2=0x7fd7ac0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.682+0000 7fd7c8988700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd7ac0a1260 0x7fd7ac0a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8006010 tx=0x7fd7b8005960 comp rx=0 tx=0).stop 2023-11-27T15:24:29.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.683+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c4100540 msgr2=0x7fd7c4072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.683+0000 7fd7c8988700 1 --2- 172.21.15.8:0/2946608151 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd7c4100540 0x7fd7c4072df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400e9a0 tx=0x7fd7b400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.683+0000 7fd7c2d9d700 1 -- 172.21.15.8:0/2946608151 reap_dead start 2023-11-27T15:24:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.683+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 shutdown_connections 2023-11-27T15:24:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.683+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 >> 172.21.15.8:0/2946608151 conn(0x7fd7c40fbdc0 msgr2=0x7fd7c40fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.683+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 shutdown_connections 2023-11-27T15:24:29.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:29.683+0000 7fd7c8988700 1 -- 172.21.15.8:0/2946608151 wait complete. 2023-11-27T15:24:30.520 INFO:teuthology.orchestra.run.smithi008.stdout:111669149758 2023-11-27T15:24:30.521 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-11-27T15:24:31.030 INFO:teuthology.orchestra.run.smithi008.stdout:287762808865 2023-11-27T15:24:31.030 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-11-27T15:24:31.083 INFO:teuthology.orchestra.run.smithi008.stdout:339302416408 2023-11-27T15:24:31.083 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-11-27T15:24:31.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.148+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/2632327915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9c108560 msgr2=0x7f3a9c108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.148+0000 7f3aa3bc3700 1 --2- 172.21.15.8:0/2632327915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9c108560 0x7f3a9c108930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009990 tx=0x7f3a98009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.152 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.148+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/2632327915 shutdown_connections 2023-11-27T15:24:31.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.148+0000 7f3aa3bc3700 1 --2- 172.21.15.8:0/2632327915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9c108560 0x7f3a9c108930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.148+0000 7f3aa3bc3700 1 --2- 172.21.15.8:0/2632327915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a9c102ef0 0x7f3a9c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.148+0000 7f3aa3bc3700 1 --2- 172.21.15.8:0/2632327915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a9c102540 0x7f3a9c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.148+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/2632327915 >> 172.21.15.8:0/2632327915 conn(0x7f3a9c0fe040 msgr2=0x7f3a9c100450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:31.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.148+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/2632327915 shutdown_connections 2023-11-27T15:24:31.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.148+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/2632327915 wait complete. 2023-11-27T15:24:31.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.149+0000 7f3aa3bc3700 1 Processor -- start 2023-11-27T15:24:31.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.150+0000 7f3aa3bc3700 1 -- start start 2023-11-27T15:24:31.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.150+0000 7f3aa3bc3700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9c102540 0x7f3a9c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.150+0000 7f3aa3bc3700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a9c102ef0 0x7f3a9c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.150+0000 7f3aa3bc3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a9c108560 0x7f3a9c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.150+0000 7f3aa3bc3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c19d590 con 0x7f3a9c108560 2023-11-27T15:24:31.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.150+0000 7f3aa3bc3700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c19d700 con 0x7f3a9c102ef0 2023-11-27T15:24:31.155 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.150+0000 7f3aa3bc3700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c19d8c0 con 0x7f3a9c102540 2023-11-27T15:24:31.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.150+0000 7f3aa195f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9c102540 0x7f3a9c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.150+0000 7f3aa195f700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9c102540 0x7f3a9c1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:40530/0 (socket says 172.21.15.8:40530) 2023-11-27T15:24:31.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.150+0000 7f3aa195f700 1 -- 172.21.15.8:0/626447749 learned_addr learned my addr 172.21.15.8:0/626447749 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:31.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.151+0000 7f3aa2160700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a9c108560 0x7f3a9c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.151+0000 7f3aa195f700 1 -- 172.21.15.8:0/626447749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a9c102ef0 msgr2=0x7f3a9c198b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-27T15:24:31.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.151+0000 7f3aa195f700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a9c102ef0 0x7f3a9c198b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.151+0000 7f3aa195f700 1 -- 172.21.15.8:0/626447749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a9c108560 msgr2=0x7f3a9c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.156 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.151+0000 7f3aa195f700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a9c108560 0x7f3a9c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.151+0000 7f3aa195f700 1 -- 172.21.15.8:0/626447749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a980095f0 con 0x7f3a9c102540 2023-11-27T15:24:31.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.151+0000 7f3aa195f700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9c102540 0x7f3a9c1985d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a98004760 tx=0x7f3a98004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.152+0000 7f3a92ffd700 1 -- 172.21.15.8:0/626447749 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9801d070 con 0x7f3a9c102540 2023-11-27T15:24:31.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.152+0000 7f3a92ffd700 1 -- 172.21.15.8:0/626447749 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a98004830 con 0x7f3a9c102540 2023-11-27T15:24:31.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.152+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c112660 con 0x7f3a9c102540 2023-11-27T15:24:31.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.152+0000 7f3a92ffd700 1 -- 172.21.15.8:0/626447749 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98005260 con 0x7f3a9c102540 2023-11-27T15:24:31.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.152+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c1124d0 con 0x7f3a9c102540 2023-11-27T15:24:31.159 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.154+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a9c109d10 con 0x7f3a9c102540 2023-11-27T15:24:31.161 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.157+0000 7f3a92ffd700 1 -- 172.21.15.8:0/626447749 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f3a98005480 con 0x7f3a9c102540 2023-11-27T15:24:31.163 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.159+0000 7f3a92ffd700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a880a12a0 0x7f3a880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.159+0000 7f3a92ffd700 1 -- 172.21.15.8:0/626447749 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3a980cdb70 con 0x7f3a9c102540 2023-11-27T15:24:31.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.160+0000 7f3a92ffd700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] conn(0x7f3a880abe10 0x7f3a880ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.160+0000 7f3a92ffd700 1 -- 172.21.15.8:0/626447749 --> [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a880ae910 con 0x7f3a880abe10 2023-11-27T15:24:31.164 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.160+0000 7f3a92ffd700 1 -- 172.21.15.8:0/626447749 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a980d22c0 con 0x7f3a9c102540 2023-11-27T15:24:31.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.160+0000 7f3aa2160700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] conn(0x7f3a880abe10 0x7f3a880ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.160+0000 7f3aa115e700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a880a12a0 0x7f3a880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.161+0000 7f3aa2160700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] conn(0x7f3a880abe10 0x7f3a880ae240 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.165 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.161+0000 7f3aa115e700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a880a12a0 0x7f3a880a3750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c199bf0 tx=0x7f3a8c006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.166 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.162+0000 7f3a92ffd700 1 -- 172.21.15.8:0/626447749 <== osd.8 v2:172.21.15.60:6816/2261251565 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3a880ae910 con 0x7f3a880abe10 2023-11-27T15:24:31.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.181+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 --> [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a9c1127f0 con 0x7f3a880abe10 2023-11-27T15:24:31.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.182+0000 7f3a92ffd700 1 -- 172.21.15.8:0/626447749 <== osd.8 v2:172.21.15.60:6816/2261251565 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3a9c1127f0 con 0x7f3a880abe10 2023-11-27T15:24:31.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.182+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 >> [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] conn(0x7f3a880abe10 msgr2=0x7f3a880ae240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.182+0000 7f3aa3bc3700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] conn(0x7f3a880abe10 0x7f3a880ae240 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.182+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a880a12a0 msgr2=0x7f3a880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.182+0000 7f3aa3bc3700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a880a12a0 0x7f3a880a3750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c199bf0 tx=0x7f3a8c006c60 comp rx=0 tx=0).stop 2023-11-27T15:24:31.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.182+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9c102540 msgr2=0x7f3a9c1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.182+0000 7f3aa3bc3700 1 --2- 172.21.15.8:0/626447749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a9c102540 0x7f3a9c1985d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a98004760 tx=0x7f3a98004790 comp rx=0 tx=0).stop 2023-11-27T15:24:31.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.182+0000 7f3aa2160700 1 -- 172.21.15.8:0/626447749 reap_dead start 2023-11-27T15:24:31.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.183+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 shutdown_connections 2023-11-27T15:24:31.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.183+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 >> 172.21.15.8:0/626447749 conn(0x7f3a9c0fe040 msgr2=0x7f3a9c109300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:31.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.183+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 shutdown_connections 2023-11-27T15:24:31.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.183+0000 7f3aa3bc3700 1 -- 172.21.15.8:0/626447749 wait complete. 2023-11-27T15:24:31.400 INFO:teuthology.orchestra.run.smithi008.stdout:201863462960 2023-11-27T15:24:31.400 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-11-27T15:24:31.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:30.999+0000 7f9516e80700 1 -- 172.21.15.8:0/1129212933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9510102f50 msgr2=0x7f951010f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:30.999+0000 7f9516e80700 1 --2- 172.21.15.8:0/1129212933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9510102f50 0x7f951010f3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f950800b0a0 tx=0x7f950800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:30.999+0000 7f9516e80700 1 -- 172.21.15.8:0/1129212933 shutdown_connections 2023-11-27T15:24:31.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:30.999+0000 7f9516e80700 1 --2- 172.21.15.8:0/1129212933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9510102f50 0x7f951010f3d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:30.999+0000 7f9516e80700 1 --2- 172.21.15.8:0/1129212933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95101086b0 0x7f9510108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:30.999+0000 7f9516e80700 1 --2- 172.21.15.8:0/1129212933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95101025c0 0x7f9510102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:30.999+0000 7f9516e80700 1 -- 172.21.15.8:0/1129212933 >> 172.21.15.8:0/1129212933 conn(0x7f95100fe100 msgr2=0x7f9510100510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:31.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:30.999+0000 7f9516e80700 1 -- 172.21.15.8:0/1129212933 shutdown_connections 2023-11-27T15:24:31.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:30.999+0000 7f9516e80700 1 -- 172.21.15.8:0/1129212933 wait complete. 2023-11-27T15:24:31.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.000+0000 7f9516e80700 1 Processor -- start 2023-11-27T15:24:31.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.000+0000 7f9516e80700 1 -- start start 2023-11-27T15:24:31.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f9516e80700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95101025c0 0x7f95101986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f9516e80700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9510102f50 0x7f9510198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f9516e80700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95101086b0 0x7f951019bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f9516e80700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951019c5f0 con 0x7f95101086b0 2023-11-27T15:24:31.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f9516e80700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951019c760 con 0x7f9510102f50 2023-11-27T15:24:31.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f9516e80700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951019c920 con 0x7f95101025c0 2023-11-27T15:24:31.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f9514c1c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95101025c0 0x7f95101986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f9514c1c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95101025c0 0x7f95101986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:40502/0 (socket says 172.21.15.8:40502) 2023-11-27T15:24:31.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f9514c1c700 1 -- 172.21.15.8:0/3719836047 learned_addr learned my addr 172.21.15.8:0/3719836047 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:31.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f951541d700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95101086b0 0x7f951019bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.001+0000 7f950ffff700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9510102f50 0x7f9510198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f9514c1c700 1 -- 172.21.15.8:0/3719836047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9510102f50 msgr2=0x7f9510198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f9514c1c700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9510102f50 0x7f9510198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f9514c1c700 1 -- 172.21.15.8:0/3719836047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95101086b0 msgr2=0x7f951019bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.507 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f9514c1c700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95101086b0 0x7f951019bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f9514c1c700 1 -- 172.21.15.8:0/3719836047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9508009d00 con 0x7f95101025c0 2023-11-27T15:24:31.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f9514c1c700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95101025c0 0x7f95101986f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f950400e9a0 tx=0x7f950400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f950dffb700 1 -- 172.21.15.8:0/3719836047 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504009790 con 0x7f95101025c0 2023-11-27T15:24:31.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f950dffb700 1 -- 172.21.15.8:0/3719836047 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9504010430 con 0x7f95101025c0 2023-11-27T15:24:31.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9510112620 con 0x7f95101025c0 2023-11-27T15:24:31.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f950dffb700 1 -- 172.21.15.8:0/3719836047 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504018680 con 0x7f95101025c0 2023-11-27T15:24:31.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.002+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9510112490 con 0x7f95101025c0 2023-11-27T15:24:31.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.003+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9510100170 con 0x7f95101025c0 2023-11-27T15:24:31.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.005+0000 7f950dffb700 1 -- 172.21.15.8:0/3719836047 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f95040105a0 con 0x7f95101025c0 2023-11-27T15:24:31.508 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.007+0000 7f950dffb700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95000a1260 0x7f95000a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.007+0000 7f950dffb700 1 -- 172.21.15.8:0/3719836047 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9504014070 con 0x7f95101025c0 2023-11-27T15:24:31.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.007+0000 7f950ffff700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95000a1260 0x7f95000a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.007+0000 7f950dffb700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] conn(0x7f95000abdd0 0x7f95000ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.007+0000 7f950dffb700 1 -- 172.21.15.8:0/3719836047 --> [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f95000ae8d0 con 0x7f95000abdd0 2023-11-27T15:24:31.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.007+0000 7f950dffb700 1 -- 172.21.15.8:0/3719836047 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95040cd3f0 con 0x7f95101025c0 2023-11-27T15:24:31.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.008+0000 7f951541d700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] conn(0x7f95000abdd0 0x7f95000ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.008+0000 7f950ffff700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95000a1260 0x7f95000a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9510199d10 tx=0x7f94fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.008+0000 7f951541d700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] conn(0x7f95000abdd0 0x7f95000ae200 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.509 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.009+0000 7f950dffb700 1 -- 172.21.15.8:0/3719836047 <== osd.4 v2:172.21.15.29:6808/3472444298 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f95000ae8d0 con 0x7f95000abdd0 2023-11-27T15:24:31.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.028+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 --> [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f95101127b0 con 0x7f95000abdd0 2023-11-27T15:24:31.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.029+0000 7f950dffb700 1 -- 172.21.15.8:0/3719836047 <== osd.4 v2:172.21.15.29:6808/3472444298 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f95101127b0 con 0x7f95000abdd0 2023-11-27T15:24:31.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.029+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 >> [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] conn(0x7f95000abdd0 msgr2=0x7f95000ae200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.029+0000 7f9516e80700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] conn(0x7f95000abdd0 0x7f95000ae200 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.030+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95000a1260 msgr2=0x7f95000a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.030+0000 7f9516e80700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95000a1260 0x7f95000a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9510199d10 tx=0x7f94fc009260 comp rx=0 tx=0).stop 2023-11-27T15:24:31.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.030+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95101025c0 msgr2=0x7f95101986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.510 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.030+0000 7f9516e80700 1 --2- 172.21.15.8:0/3719836047 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95101025c0 0x7f95101986f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f950400e9a0 tx=0x7f950400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.030+0000 7f951541d700 1 -- 172.21.15.8:0/3719836047 reap_dead start 2023-11-27T15:24:31.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.030+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 shutdown_connections 2023-11-27T15:24:31.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.030+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 >> 172.21.15.8:0/3719836047 conn(0x7f95100fe100 msgr2=0x7f95100ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:31.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.030+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 shutdown_connections 2023-11-27T15:24:31.511 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.030+0000 7f9516e80700 1 -- 172.21.15.8:0/3719836047 wait complete. 2023-11-27T15:24:31.589 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:31 smithi008 ceph-mon[128788]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:24:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:31 smithi060 ceph-mon[131518]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:24:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:31 smithi029 ceph-mon[131166]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:24:31.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.907+0000 7fcc80344700 1 -- 172.21.15.8:0/3707715521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc78100540 msgr2=0x7fcc78100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.907+0000 7fcc80344700 1 --2- 172.21.15.8:0/3707715521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc78100540 0x7fcc78100910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009990 tx=0x7fcc74009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.908+0000 7fcc80344700 1 -- 172.21.15.8:0/3707715521 shutdown_connections 2023-11-27T15:24:31.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.908+0000 7fcc80344700 1 --2- 172.21.15.8:0/3707715521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc781055e0 0x7fcc7810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.908+0000 7fcc80344700 1 --2- 172.21.15.8:0/3707715521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc78100540 0x7fcc78100910 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.908+0000 7fcc80344700 1 --2- 172.21.15.8:0/3707715521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc78100ee0 0x7fcc78104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.908+0000 7fcc80344700 1 -- 172.21.15.8:0/3707715521 >> 172.21.15.8:0/3707715521 conn(0x7fcc780fbdc0 msgr2=0x7fcc780fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:31.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.908+0000 7fcc80344700 1 -- 172.21.15.8:0/3707715521 shutdown_connections 2023-11-27T15:24:31.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.908+0000 7fcc80344700 1 -- 172.21.15.8:0/3707715521 wait complete. 2023-11-27T15:24:31.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.909+0000 7fcc80344700 1 Processor -- start 2023-11-27T15:24:31.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.909+0000 7fcc80344700 1 -- start start 2023-11-27T15:24:31.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.910+0000 7fcc80344700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc78100540 0x7fcc781a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.910+0000 7fcc80344700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc78100ee0 0x7fcc781a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.910+0000 7fcc80344700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc781055e0 0x7fcc7819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.910+0000 7fcc80344700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7819ced0 con 0x7fcc781055e0 2023-11-27T15:24:31.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.910+0000 7fcc80344700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7819d040 con 0x7fcc78100ee0 2023-11-27T15:24:31.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.910+0000 7fcc80344700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7819d200 con 0x7fcc78100540 2023-11-27T15:24:31.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.910+0000 7fcc7e0e0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc78100540 0x7fcc781a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.910+0000 7fcc7d8df700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc78100ee0 0x7fcc781a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.910+0000 7fcc7e0e0700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc78100540 0x7fcc781a27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:40570/0 (socket says 172.21.15.8:40570) 2023-11-27T15:24:31.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.910+0000 7fcc7e0e0700 1 -- 172.21.15.8:0/3357576146 learned_addr learned my addr 172.21.15.8:0/3357576146 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:31.917 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.911+0000 7fcc7e8e1700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc781055e0 0x7fcc7819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.911+0000 7fcc7d8df700 1 -- 172.21.15.8:0/3357576146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc78100540 msgr2=0x7fcc781a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.911+0000 7fcc7d8df700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc78100540 0x7fcc781a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.911+0000 7fcc7d8df700 1 -- 172.21.15.8:0/3357576146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc781055e0 msgr2=0x7fcc7819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.911+0000 7fcc7d8df700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc781055e0 0x7fcc7819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.918 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.911+0000 7fcc7d8df700 1 -- 172.21.15.8:0/3357576146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc740095f0 con 0x7fcc78100ee0 2023-11-27T15:24:31.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.911+0000 7fcc7e0e0700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fcc78100540 0x7fcc781a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:31.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.912+0000 7fcc7d8df700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc78100ee0 0x7fcc781a2d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc7806a3c0 tx=0x7fcc6800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.912+0000 7fcc6f7fe700 1 -- 172.21.15.8:0/3357576146 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800f890 con 0x7fcc78100ee0 2023-11-27T15:24:31.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.912+0000 7fcc6f7fe700 1 -- 172.21.15.8:0/3357576146 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc68010460 con 0x7fcc78100ee0 2023-11-27T15:24:31.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.912+0000 7fcc6f7fe700 1 -- 172.21.15.8:0/3357576146 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800e610 con 0x7fcc78100ee0 2023-11-27T15:24:31.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.912+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc78116a60 con 0x7fcc78100ee0 2023-11-27T15:24:31.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.912+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc781168d0 con 0x7fcc78100ee0 2023-11-27T15:24:31.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.913+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc780fd1f0 con 0x7fcc78100ee0 2023-11-27T15:24:31.921 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.917+0000 7fcc6f7fe700 1 -- 172.21.15.8:0/3357576146 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fcc68010890 con 0x7fcc78100ee0 2023-11-27T15:24:31.922 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.918+0000 7fcc6f7fe700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fcc640a1290 0x7fcc640a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.919+0000 7fcc6f7fe700 1 -- 172.21.15.8:0/3357576146 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcc680ccdb0 con 0x7fcc78100ee0 2023-11-27T15:24:31.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.919+0000 7fcc7e0e0700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fcc640a1290 0x7fcc640a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.919+0000 7fcc6f7fe700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] conn(0x7fcc640abe00 0x7fcc640ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.919+0000 7fcc6f7fe700 1 -- 172.21.15.8:0/3357576146 --> [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc640ae900 con 0x7fcc640abe00 2023-11-27T15:24:31.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.919+0000 7fcc6f7fe700 1 -- 172.21.15.8:0/3357576146 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc680cd120 con 0x7fcc78100ee0 2023-11-27T15:24:31.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.919+0000 7fcc7e8e1700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] conn(0x7fcc640abe00 0x7fcc640ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.924 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.920+0000 7fcc7e0e0700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fcc640a1290 0x7fcc640a3740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400b6f0 tx=0x7fcc74005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.920+0000 7fcc7e8e1700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] conn(0x7fcc640abe00 0x7fcc640ae230 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.925 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.921+0000 7fcc6f7fe700 1 -- 172.21.15.8:0/3357576146 <== osd.10 v2:172.21.15.29:6824/3342205417 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcc640ae900 con 0x7fcc640abe00 2023-11-27T15:24:31.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.939+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 --> [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc78116bf0 con 0x7fcc640abe00 2023-11-27T15:24:31.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.940+0000 7fcc6f7fe700 1 -- 172.21.15.8:0/3357576146 <== osd.10 v2:172.21.15.29:6824/3342205417 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcc78116bf0 con 0x7fcc640abe00 2023-11-27T15:24:31.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.940+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 >> [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] conn(0x7fcc640abe00 msgr2=0x7fcc640ae230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.940+0000 7fcc80344700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] conn(0x7fcc640abe00 0x7fcc640ae230 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.940+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fcc640a1290 msgr2=0x7fcc640a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.940+0000 7fcc80344700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fcc640a1290 0x7fcc640a3740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400b6f0 tx=0x7fcc74005fb0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.940+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc78100ee0 msgr2=0x7fcc781a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.940+0000 7fcc80344700 1 --2- 172.21.15.8:0/3357576146 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fcc78100ee0 0x7fcc781a2d20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc7806a3c0 tx=0x7fcc6800dc30 comp rx=0 tx=0).stop 2023-11-27T15:24:31.945 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.940+0000 7fcc7e8e1700 1 -- 172.21.15.8:0/3357576146 reap_dead start 2023-11-27T15:24:31.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.940+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 shutdown_connections 2023-11-27T15:24:31.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.940+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 >> 172.21.15.8:0/3357576146 conn(0x7fcc780fbdc0 msgr2=0x7fcc780fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:31.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.941+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 shutdown_connections 2023-11-27T15:24:31.946 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.941+0000 7fcc80344700 1 -- 172.21.15.8:0/3357576146 wait complete. 2023-11-27T15:24:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.717+0000 7faab4697700 1 -- 172.21.15.8:0/2719933505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaac072e30 msgr2=0x7faaac073200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.717+0000 7faab4697700 1 --2- 172.21.15.8:0/2719933505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaac072e30 0x7faaac073200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009990 tx=0x7faaa8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.717+0000 7faab4697700 1 -- 172.21.15.8:0/2719933505 shutdown_connections 2023-11-27T15:24:31.951 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.717+0000 7faab4697700 1 --2- 172.21.15.8:0/2719933505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac10cb80 0x7faaac10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.717+0000 7faab4697700 1 --2- 172.21.15.8:0/2719933505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaac072e30 0x7faaac073200 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.717+0000 7faab4697700 1 --2- 172.21.15.8:0/2719933505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faaac073740 0x7faaac10c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.717+0000 7faab4697700 1 -- 172.21.15.8:0/2719933505 >> 172.21.15.8:0/2719933505 conn(0x7faaac0fbdc0 msgr2=0x7faaac0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.717+0000 7faab4697700 1 -- 172.21.15.8:0/2719933505 shutdown_connections 2023-11-27T15:24:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.717+0000 7faab4697700 1 -- 172.21.15.8:0/2719933505 wait complete. 2023-11-27T15:24:31.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.718+0000 7faab4697700 1 Processor -- start 2023-11-27T15:24:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.719+0000 7faab4697700 1 -- start start 2023-11-27T15:24:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.719+0000 7faab4697700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac072e30 0x7faaac105780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.719+0000 7faab4697700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faaac073740 0x7faaac105ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.719+0000 7faab4697700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaac10cb80 0x7faaac100830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.719+0000 7faab4697700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac101000 con 0x7faaac073740 2023-11-27T15:24:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.719+0000 7faab4697700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac101170 con 0x7faaac10cb80 2023-11-27T15:24:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.719+0000 7faab4697700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac101330 con 0x7faaac072e30 2023-11-27T15:24:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.719+0000 7faab1c32700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faaac073740 0x7faaac105ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.719+0000 7faab2c34700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaac10cb80 0x7faaac100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.954 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.720+0000 7faab2c34700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaac10cb80 0x7faaac100830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:49968/0 (socket says 172.21.15.8:49968) 2023-11-27T15:24:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.720+0000 7faab2c34700 1 -- 172.21.15.8:0/1897044911 learned_addr learned my addr 172.21.15.8:0/1897044911 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.720+0000 7faab2433700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac072e30 0x7faaac105780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.720+0000 7faab1c32700 1 -- 172.21.15.8:0/1897044911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac072e30 msgr2=0x7faaac105780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.720+0000 7faab1c32700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faaac072e30 0x7faaac105780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.955 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.720+0000 7faab1c32700 1 -- 172.21.15.8:0/1897044911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaac10cb80 msgr2=0x7faaac100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.720+0000 7faab1c32700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaac10cb80 0x7faaac100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.720+0000 7faab1c32700 1 -- 172.21.15.8:0/1897044911 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80095f0 con 0x7faaac073740 2023-11-27T15:24:31.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.720+0000 7faab2c34700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faaac10cb80 0x7faaac100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:31.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.720+0000 7faab1c32700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faaac073740 0x7faaac105ce0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00e9f0 tx=0x7faa9c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.956 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.721+0000 7faaa37fe700 1 -- 172.21.15.8:0/1897044911 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00eee0 con 0x7faaac073740 2023-11-27T15:24:31.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.721+0000 7faaa37fe700 1 -- 172.21.15.8:0/1897044911 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa9c00cbd0 con 0x7faaac073740 2023-11-27T15:24:31.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.721+0000 7faaa37fe700 1 -- 172.21.15.8:0/1897044911 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c018680 con 0x7faaac073740 2023-11-27T15:24:31.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.721+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac112570 con 0x7faaac073740 2023-11-27T15:24:31.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.721+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac1123e0 con 0x7faaac073740 2023-11-27T15:24:31.957 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.722+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faaac0611f0 con 0x7faaac073740 2023-11-27T15:24:31.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.725+0000 7faaa37fe700 1 -- 172.21.15.8:0/1897044911 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7faa9c00cd40 con 0x7faaac073740 2023-11-27T15:24:31.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.726+0000 7faaa37fe700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa980a1260 0x7faa980a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.726+0000 7faaa37fe700 1 -- 172.21.15.8:0/1897044911 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faa9c014070 con 0x7faaac073740 2023-11-27T15:24:31.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.726+0000 7faaa37fe700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] conn(0x7faa980abdd0 0x7faa980ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:31.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.727+0000 7faaa37fe700 1 -- 172.21.15.8:0/1897044911 --> [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa980ae8d0 con 0x7faa980abdd0 2023-11-27T15:24:31.958 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.727+0000 7faaa37fe700 1 -- 172.21.15.8:0/1897044911 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa9c0cd590 con 0x7faaac073740 2023-11-27T15:24:31.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.727+0000 7faab2c34700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] conn(0x7faa980abdd0 0x7faa980ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.727+0000 7faab2c34700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] conn(0x7faa980abdd0 0x7faa980ae200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.728+0000 7faab2433700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa980a1260 0x7faa980a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:31.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.728+0000 7faaa37fe700 1 -- 172.21.15.8:0/1897044911 <== osd.7 v2:172.21.15.29:6816/2172314430 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faa980ae8d0 con 0x7faa980abdd0 2023-11-27T15:24:31.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.731+0000 7faab2433700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa980a1260 0x7faa980a3710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faaa800b2e0 tx=0x7faaa8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:31.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.752+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 --> [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faaac112700 con 0x7faa980abdd0 2023-11-27T15:24:31.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.753+0000 7faaa37fe700 1 -- 172.21.15.8:0/1897044911 <== osd.7 v2:172.21.15.29:6816/2172314430 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faaac112700 con 0x7faa980abdd0 2023-11-27T15:24:31.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.754+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 >> [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] conn(0x7faa980abdd0 msgr2=0x7faa980ae200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.754+0000 7faab4697700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] conn(0x7faa980abdd0 0x7faa980ae200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.754+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa980a1260 msgr2=0x7faa980a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.754+0000 7faab4697700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7faa980a1260 0x7faa980a3710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faaa800b2e0 tx=0x7faaa8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:24:31.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.754+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faaac073740 msgr2=0x7faaac105ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:31.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.754+0000 7faab4697700 1 --2- 172.21.15.8:0/1897044911 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faaac073740 0x7faaac105ce0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00e9f0 tx=0x7faa9c00ed00 comp rx=0 tx=0).stop 2023-11-27T15:24:31.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.754+0000 7faab2c34700 1 -- 172.21.15.8:0/1897044911 reap_dead start 2023-11-27T15:24:31.961 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.755+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 shutdown_connections 2023-11-27T15:24:31.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.755+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 >> 172.21.15.8:0/1897044911 conn(0x7faaac0fbdc0 msgr2=0x7faaac197b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:31.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.756+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 shutdown_connections 2023-11-27T15:24:31.962 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:31.756+0000 7faab4697700 1 -- 172.21.15.8:0/1897044911 wait complete. 2023-11-27T15:24:32.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.105+0000 7f1de6e5a700 1 -- 172.21.15.8:0/3157441121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de0100ee0 msgr2=0x7f1de0104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:32.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.105+0000 7f1de6e5a700 1 --2- 172.21.15.8:0/3157441121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de0100ee0 0x7f1de0104eb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009990 tx=0x7f1dcc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:32.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.105+0000 7f1de6e5a700 1 -- 172.21.15.8:0/3157441121 shutdown_connections 2023-11-27T15:24:32.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.105+0000 7f1de6e5a700 1 --2- 172.21.15.8:0/3157441121 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de0100540 0x7f1de0100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:32.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.105+0000 7f1de6e5a700 1 --2- 172.21.15.8:0/3157441121 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1de01055e0 0x7f1de010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:32.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.105+0000 7f1de6e5a700 1 --2- 172.21.15.8:0/3157441121 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de0100ee0 0x7f1de0104eb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:32.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.105+0000 7f1de6e5a700 1 -- 172.21.15.8:0/3157441121 >> 172.21.15.8:0/3157441121 conn(0x7f1de00fbdc0 msgr2=0x7f1de00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:32.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.105+0000 7f1de6e5a700 1 -- 172.21.15.8:0/3157441121 shutdown_connections 2023-11-27T15:24:32.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.105+0000 7f1de6e5a700 1 -- 172.21.15.8:0/3157441121 wait complete. 2023-11-27T15:24:32.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.106+0000 7f1de6e5a700 1 Processor -- start 2023-11-27T15:24:32.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1de6e5a700 1 -- start start 2023-11-27T15:24:32.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1de6e5a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de0100540 0x7f1de0196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:32.112 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1de6e5a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de0100ee0 0x7f1de0196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:32.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1de6e5a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1de01055e0 0x7f1de019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:32.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1de6e5a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de019b3c0 con 0x7f1de01055e0 2023-11-27T15:24:32.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1de6e5a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de019b530 con 0x7f1de0100ee0 2023-11-27T15:24:32.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1de6e5a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de019b6f0 con 0x7f1de0100540 2023-11-27T15:24:32.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1ddffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de0100ee0 0x7f1de0196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:32.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1ddffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de0100ee0 0x7f1de0196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50000/0 (socket says 172.21.15.8:50000) 2023-11-27T15:24:32.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1ddffff700 1 -- 172.21.15.8:0/2744990761 learned_addr learned my addr 172.21.15.8:0/2744990761 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:32.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1de53f7700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1de01055e0 0x7f1de019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:32.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.107+0000 7f1de4bf6700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de0100540 0x7f1de0196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:32.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.108+0000 7f1ddffff700 1 -- 172.21.15.8:0/2744990761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de0100540 msgr2=0x7f1de0196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:32.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.108+0000 7f1ddffff700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1de0100540 0x7f1de0196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:32.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.108+0000 7f1ddffff700 1 -- 172.21.15.8:0/2744990761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1de01055e0 msgr2=0x7f1de019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:32.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.108+0000 7f1ddffff700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1de01055e0 0x7f1de019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:32.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.108+0000 7f1ddffff700 1 -- 172.21.15.8:0/2744990761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd8009c40 con 0x7f1de0100ee0 2023-11-27T15:24:32.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.108+0000 7f1ddffff700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de0100ee0 0x7f1de0196940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc004740 tx=0x7f1dcc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:32.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.108+0000 7f1dddffb700 1 -- 172.21.15.8:0/2744990761 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc01d070 con 0x7f1de0100ee0 2023-11-27T15:24:32.114 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.108+0000 7f1dddffb700 1 -- 172.21.15.8:0/2744990761 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dcc004830 con 0x7f1de0100ee0 2023-11-27T15:24:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.108+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc0095f0 con 0x7f1de0100ee0 2023-11-27T15:24:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.108+0000 7f1dddffb700 1 -- 172.21.15.8:0/2744990761 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0052c0 con 0x7f1de0100ee0 2023-11-27T15:24:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.109+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de0110360 con 0x7f1de0100ee0 2023-11-27T15:24:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.109+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1de00fd2b0 con 0x7f1de0100ee0 2023-11-27T15:24:32.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.126+0000 7f1dddffb700 1 -- 172.21.15.8:0/2744990761 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1dcc005420 con 0x7f1de0100ee0 2023-11-27T15:24:32.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.128+0000 7f1dddffb700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dd00a1260 0x7f1dd00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:32.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.128+0000 7f1dddffb700 1 -- 172.21.15.8:0/2744990761 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1dcc0ce4c0 con 0x7f1de0100ee0 2023-11-27T15:24:32.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.128+0000 7f1de4bf6700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dd00a1260 0x7f1dd00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:32.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.128+0000 7f1dddffb700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] conn(0x7f1dd00abdd0 0x7f1dd00ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:32.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.128+0000 7f1dddffb700 1 -- 172.21.15.8:0/2744990761 --> [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1dd00ae8d0 con 0x7f1dd00abdd0 2023-11-27T15:24:32.132 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.128+0000 7f1dddffb700 1 -- 172.21.15.8:0/2744990761 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dcc0ce830 con 0x7f1de0100ee0 2023-11-27T15:24:32.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.128+0000 7f1de53f7700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] conn(0x7f1dd00abdd0 0x7f1dd00ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:32.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.129+0000 7f1de4bf6700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dd00a1260 0x7f1dd00a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009c60 tx=0x7f1dd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:32.148 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.145+0000 7f1de53f7700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] conn(0x7f1dd00abdd0 0x7f1dd00ae200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:32.149 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.145+0000 7f1dddffb700 1 -- 172.21.15.8:0/2744990761 <== osd.5 v2:172.21.15.60:6808/1250742602 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1dd00ae8d0 con 0x7f1dd00abdd0 2023-11-27T15:24:32.167 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.163+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 --> [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1de01101d0 con 0x7f1dd00abdd0 2023-11-27T15:24:32.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.164+0000 7f1dddffb700 1 -- 172.21.15.8:0/2744990761 <== osd.5 v2:172.21.15.60:6808/1250742602 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1de01101d0 con 0x7f1dd00abdd0 2023-11-27T15:24:32.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.164+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 >> [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] conn(0x7f1dd00abdd0 msgr2=0x7f1dd00ae200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:32.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.164+0000 7f1de6e5a700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] conn(0x7f1dd00abdd0 0x7f1dd00ae200 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:32.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.164+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dd00a1260 msgr2=0x7f1dd00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:32.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.164+0000 7f1de6e5a700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dd00a1260 0x7f1dd00a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009c60 tx=0x7f1dd4009260 comp rx=0 tx=0).stop 2023-11-27T15:24:32.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.164+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de0100ee0 msgr2=0x7f1de0196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:32.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.164+0000 7f1de6e5a700 1 --2- 172.21.15.8:0/2744990761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1de0100ee0 0x7f1de0196940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc004740 tx=0x7f1dcc009580 comp rx=0 tx=0).stop 2023-11-27T15:24:32.169 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.164+0000 7f1de53f7700 1 -- 172.21.15.8:0/2744990761 reap_dead start 2023-11-27T15:24:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.165+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 shutdown_connections 2023-11-27T15:24:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.165+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 >> 172.21.15.8:0/2744990761 conn(0x7f1de00fbdc0 msgr2=0x7f1de00fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.165+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 shutdown_connections 2023-11-27T15:24:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:32.165+0000 7f1de6e5a700 1 -- 172.21.15.8:0/2744990761 wait complete. 2023-11-27T15:24:32.824 INFO:teuthology.orchestra.run.smithi008.stdout:223338299435 2023-11-27T15:24:32.824 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-11-27T15:24:33.110 INFO:teuthology.orchestra.run.smithi008.stdout:146028888124 2023-11-27T15:24:33.111 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-11-27T15:24:33.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.324 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.325 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:32 smithi060 ceph-mon[131518]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:24:33.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.364 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:32 smithi008 ceph-mon[128788]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:24:33.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.397 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:33.398 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:32 smithi029 ceph-mon[131166]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:24:33.580 INFO:teuthology.orchestra.run.smithi008.stdout:77309411397 2023-11-27T15:24:33.581 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-11-27T15:24:33.633 INFO:teuthology.orchestra.run.smithi008.stdout:253403070504 2023-11-27T15:24:33.633 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-11-27T15:24:33.859 INFO:teuthology.orchestra.run.smithi008.stdout:167503724598 2023-11-27T15:24:33.859 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-11-27T15:24:33.934 INFO:teuthology.orchestra.run.smithi008.stdout:309237645343 2023-11-27T15:24:33.935 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-11-27T15:24:34.133 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:34.135 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:34.322 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:34.333 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:34.794 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:35.021 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:35.146 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:35 smithi060 ceph-mon[131518]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:24:35.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:35 smithi008 ceph-mon[128788]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:24:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:35 smithi029 ceph-mon[131166]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:24:36.192 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:37.468 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:37.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:37 smithi008 ceph-mon[128788]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2023-11-27T15:24:37.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.632+0000 7f78b20ad700 1 -- 172.21.15.8:0/3740491604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78ac104660 msgr2=0x7f78ac104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:37.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.632+0000 7f78b20ad700 1 --2- 172.21.15.8:0/3740491604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78ac104660 0x7f78ac104a90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f78a400b0a0 tx=0x7f78a400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.632+0000 7f78b20ad700 1 -- 172.21.15.8:0/3740491604 shutdown_connections 2023-11-27T15:24:37.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.632+0000 7f78b20ad700 1 --2- 172.21.15.8:0/3740491604 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78ac0ff330 0x7f78ac0ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.632+0000 7f78b20ad700 1 --2- 172.21.15.8:0/3740491604 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78ac104660 0x7f78ac104a90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.632+0000 7f78b20ad700 1 --2- 172.21.15.8:0/3740491604 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78ac0ffc40 0x7f78ac104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.632+0000 7f78b20ad700 1 -- 172.21.15.8:0/3740491604 >> 172.21.15.8:0/3740491604 conn(0x7f78ac0770a0 msgr2=0x7f78ac0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:37.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.632+0000 7f78b20ad700 1 -- 172.21.15.8:0/3740491604 shutdown_connections 2023-11-27T15:24:37.636 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.632+0000 7f78b20ad700 1 -- 172.21.15.8:0/3740491604 wait complete. 2023-11-27T15:24:37.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.633+0000 7f78b20ad700 1 Processor -- start 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.633+0000 7f78b20ad700 1 -- start start 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78b20ad700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78ac0ff330 0x7f78ac1a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78b20ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78ac0ffc40 0x7f78ac1a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78b20ad700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78ac104660 0x7f78ac19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78b20ad700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac19cfb0 con 0x7f78ac0ffc40 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78b20ad700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac19d120 con 0x7f78ac0ff330 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78b20ad700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac19d2e0 con 0x7f78ac104660 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78ab7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78ac0ff330 0x7f78ac1a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78aaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78ac0ffc40 0x7f78ac1a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78ab7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78ac0ff330 0x7f78ac1a28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50018/0 (socket says 172.21.15.8:50018) 2023-11-27T15:24:37.638 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78ab7fe700 1 -- 172.21.15.8:0/588747254 learned_addr learned my addr 172.21.15.8:0/588747254 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:37.639 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78abfff700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78ac104660 0x7f78ac19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78ab7fe700 1 -- 172.21.15.8:0/588747254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78ac104660 msgr2=0x7f78ac19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78ab7fe700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78ac104660 0x7f78ac19c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78ab7fe700 1 -- 172.21.15.8:0/588747254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78ac0ffc40 msgr2=0x7f78ac1a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78ab7fe700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78ac0ffc40 0x7f78ac1a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78ab7fe700 1 -- 172.21.15.8:0/588747254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0009520 con 0x7f78ac0ff330 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.634+0000 7f78aaffd700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78ac0ffc40 0x7f78ac1a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.635+0000 7f78ab7fe700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78ac0ff330 0x7f78ac1a28c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f789c00e9a0 tx=0x7f789c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.635+0000 7f78a8ff9700 1 -- 172.21.15.8:0/588747254 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c009790 con 0x7f78ac0ff330 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.635+0000 7f78a8ff9700 1 -- 172.21.15.8:0/588747254 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f789c010430 con 0x7f78ac0ff330 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.635+0000 7f78a8ff9700 1 -- 172.21.15.8:0/588747254 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c018680 con 0x7f78ac0ff330 2023-11-27T15:24:37.640 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.635+0000 7f78b20ad700 1 -- 172.21.15.8:0/588747254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a4009d00 con 0x7f78ac0ff330 2023-11-27T15:24:37.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.635+0000 7f78b20ad700 1 -- 172.21.15.8:0/588747254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac116960 con 0x7f78ac0ff330 2023-11-27T15:24:37.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.636+0000 7f78b20ad700 1 -- 172.21.15.8:0/588747254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ac1167d0 con 0x7f78ac0ff330 2023-11-27T15:24:37.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.638+0000 7f78a8ff9700 1 -- 172.21.15.8:0/588747254 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f789c0105a0 con 0x7f78ac0ff330 2023-11-27T15:24:37.643 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.640+0000 7f78a8ff9700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78940a1260 0x7f78940a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:37.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.640+0000 7f78a8ff9700 1 -- 172.21.15.8:0/588747254 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f789c014070 con 0x7f78ac0ff330 2023-11-27T15:24:37.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.640+0000 7f78aaffd700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78940a1260 0x7f78940a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:37.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.641+0000 7f78aaffd700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78940a1260 0x7f78940a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78ac1044c0 tx=0x7f78a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:37.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.642+0000 7f78a8ff9700 1 -- 172.21.15.8:0/588747254 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f789c096220 con 0x7f78ac0ff330 2023-11-27T15:24:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:37 smithi060 ceph-mon[131518]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2023-11-27T15:24:37.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.855+0000 7f78b20ad700 1 -- 172.21.15.8:0/588747254 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f78ac066ea0 con 0x7f78ac0ff330 2023-11-27T15:24:37.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.856+0000 7f78a8ff9700 1 -- 172.21.15.8:0/588747254 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f789c099840 con 0x7f78ac0ff330 2023-11-27T15:24:37.862 INFO:teuthology.orchestra.run.smithi008.stdout:34359738446 2023-11-27T15:24:37.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 -- 172.21.15.8:0/588747254 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78940a1260 msgr2=0x7f78940a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:37.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78940a1260 0x7f78940a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f78ac1044c0 tx=0x7f78a0009260 comp rx=0 tx=0).stop 2023-11-27T15:24:37.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 -- 172.21.15.8:0/588747254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78ac0ff330 msgr2=0x7f78ac1a28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:37.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78ac0ff330 0x7f78ac1a28c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f789c00e9a0 tx=0x7f789c00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 -- 172.21.15.8:0/588747254 shutdown_connections 2023-11-27T15:24:37.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78940a1260 0x7f78940a3710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f78ac104660 0x7f78ac19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f78ac0ff330 0x7f78ac1a28c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 --2- 172.21.15.8:0/588747254 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78ac0ffc40 0x7f78ac1a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:37.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 -- 172.21.15.8:0/588747254 >> 172.21.15.8:0/588747254 conn(0x7f78ac0770a0 msgr2=0x7f78ac1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:37.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 -- 172.21.15.8:0/588747254 shutdown_connections 2023-11-27T15:24:37.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:37.859+0000 7f78b20ad700 1 -- 172.21.15.8:0/588747254 wait complete. 2023-11-27T15:24:37.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:37 smithi029 ceph-mon[131166]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2023-11-27T15:24:38.420 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/588747254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-27T15:24:38.420 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2023-11-27T15:24:38.420 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-11-27T15:24:38.420 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-11-27T15:24:38.421 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-11-27T15:24:38.421 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-11-27T15:24:38.421 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-11-27T15:24:38.421 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-11-27T15:24:38.421 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-11-27T15:24:38.421 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2023-11-27T15:24:38.421 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:38 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2023-11-27T15:24:38.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.526+0000 7fb063ea8700 1 -- 172.21.15.8:0/2755311962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb05c1055e0 msgr2=0x7fb05c10d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:38.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.526+0000 7fb063ea8700 1 --2- 172.21.15.8:0/2755311962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb05c1055e0 0x7fb05c10d160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb05400b0d0 tx=0x7fb05400b3e0 comp rx=0 tx=0).stop 2023-11-27T15:24:38.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.527+0000 7fb063ea8700 1 -- 172.21.15.8:0/2755311962 shutdown_connections 2023-11-27T15:24:38.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.527+0000 7fb063ea8700 1 --2- 172.21.15.8:0/2755311962 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c100540 0x7fb05c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:38.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.527+0000 7fb063ea8700 1 --2- 172.21.15.8:0/2755311962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb05c1055e0 0x7fb05c10d160 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:38.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.527+0000 7fb063ea8700 1 --2- 172.21.15.8:0/2755311962 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb05c100ee0 0x7fb05c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:38.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.527+0000 7fb063ea8700 1 -- 172.21.15.8:0/2755311962 >> 172.21.15.8:0/2755311962 conn(0x7fb05c0fbdc0 msgr2=0x7fb05c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:38.542 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.527+0000 7fb063ea8700 1 -- 172.21.15.8:0/2755311962 shutdown_connections 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.527+0000 7fb063ea8700 1 -- 172.21.15.8:0/2755311962 wait complete. 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.528+0000 7fb063ea8700 1 Processor -- start 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.528+0000 7fb063ea8700 1 -- start start 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.528+0000 7fb063ea8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c100540 0x7fb05c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.528+0000 7fb063ea8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb05c100ee0 0x7fb05c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.528+0000 7fb063ea8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb05c1055e0 0x7fb05c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.528+0000 7fb063ea8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c073c20 con 0x7fb05c100ee0 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.528+0000 7fb063ea8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c1a4c10 con 0x7fb05c1055e0 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.528+0000 7fb063ea8700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c1a4d80 con 0x7fb05c100540 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.528+0000 7fb061443700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb05c100ee0 0x7fb05c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb061443700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb05c100ee0 0x7fb05c073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35174/0 (socket says 172.21.15.8:35174) 2023-11-27T15:24:38.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb061443700 1 -- 172.21.15.8:0/3136956756 learned_addr learned my addr 172.21.15.8:0/3136956756 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb061c44700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c100540 0x7fb05c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb062445700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb05c1055e0 0x7fb05c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb061443700 1 -- 172.21.15.8:0/3136956756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c100540 msgr2=0x7fb05c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb061443700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c100540 0x7fb05c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb061443700 1 -- 172.21.15.8:0/3136956756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb05c1055e0 msgr2=0x7fb05c1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb061443700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb05c1055e0 0x7fb05c1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb061443700 1 -- 172.21.15.8:0/3136956756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb054009d00 con 0x7fb05c100ee0 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb061443700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb05c100ee0 0x7fb05c073330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00e970 tx=0x7fb04c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb052ffd700 1 -- 172.21.15.8:0/3136956756 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c009790 con 0x7fb05c100ee0 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb052ffd700 1 -- 172.21.15.8:0/3136956756 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb04c00c7e0 con 0x7fb05c100ee0 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.529+0000 7fb063ea8700 1 -- 172.21.15.8:0/3136956756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05c110320 con 0x7fb05c100ee0 2023-11-27T15:24:38.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.530+0000 7fb052ffd700 1 -- 172.21.15.8:0/3136956756 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0054e0 con 0x7fb05c100ee0 2023-11-27T15:24:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.530+0000 7fb063ea8700 1 -- 172.21.15.8:0/3136956756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05c110190 con 0x7fb05c100ee0 2023-11-27T15:24:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.531+0000 7fb063ea8700 1 -- 172.21.15.8:0/3136956756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05c1104b0 con 0x7fb05c100ee0 2023-11-27T15:24:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.533+0000 7fb052ffd700 1 -- 172.21.15.8:0/3136956756 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb04c00c950 con 0x7fb05c100ee0 2023-11-27T15:24:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.534+0000 7fb052ffd700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb0480a11e0 0x7fb0480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.534+0000 7fb052ffd700 1 -- 172.21.15.8:0/3136956756 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb04c014070 con 0x7fb05c100ee0 2023-11-27T15:24:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.534+0000 7fb061c44700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb0480a11e0 0x7fb0480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.535+0000 7fb061c44700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb0480a11e0 0x7fb0480a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb058009c60 tx=0x7fb058009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:38.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.539+0000 7fb052ffd700 1 -- 172.21.15.8:0/3136956756 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb04c096210 con 0x7fb05c100ee0 2023-11-27T15:24:38.772 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.769+0000 7fb063ea8700 1 -- 172.21.15.8:0/3136956756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb05c066ea0 con 0x7fb05c100ee0 2023-11-27T15:24:38.773 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.769+0000 7fb052ffd700 1 -- 172.21.15.8:0/3136956756 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb04c099830 con 0x7fb05c100ee0 2023-11-27T15:24:38.773 INFO:teuthology.orchestra.run.smithi008.stdout:55834574921 2023-11-27T15:24:38.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.771+0000 7fb063ea8700 1 -- 172.21.15.8:0/3136956756 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb0480a11e0 msgr2=0x7fb0480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:38.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.771+0000 7fb063ea8700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb0480a11e0 0x7fb0480a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb058009c60 tx=0x7fb058009260 comp rx=0 tx=0).stop 2023-11-27T15:24:38.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.771+0000 7fb063ea8700 1 -- 172.21.15.8:0/3136956756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb05c100ee0 msgr2=0x7fb05c073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:38.775 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.771+0000 7fb063ea8700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb05c100ee0 0x7fb05c073330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00e970 tx=0x7fb04c00ec80 comp rx=0 tx=0).stop 2023-11-27T15:24:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.771+0000 7fb063ea8700 1 -- 172.21.15.8:0/3136956756 shutdown_connections 2023-11-27T15:24:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.771+0000 7fb063ea8700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb0480a11e0 0x7fb0480a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.771+0000 7fb063ea8700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb05c100540 0x7fb05c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.771+0000 7fb063ea8700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb05c1055e0 0x7fb05c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.771+0000 7fb063ea8700 1 --2- 172.21.15.8:0/3136956756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb05c100ee0 0x7fb05c073330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.771+0000 7fb063ea8700 1 -- 172.21.15.8:0/3136956756 >> 172.21.15.8:0/3136956756 conn(0x7fb05c0fbdc0 msgr2=0x7fb05c0fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.772+0000 7fb063ea8700 1 -- 172.21.15.8:0/3136956756 shutdown_connections 2023-11-27T15:24:38.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:38.772+0000 7fb063ea8700 1 -- 172.21.15.8:0/3136956756 wait complete. 2023-11-27T15:24:38.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/588747254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-27T15:24:38.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2023-11-27T15:24:38.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-11-27T15:24:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-11-27T15:24:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-11-27T15:24:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-11-27T15:24:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-11-27T15:24:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-11-27T15:24:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-11-27T15:24:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2023-11-27T15:24:38.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:38 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2023-11-27T15:24:38.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/588747254' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-27T15:24:38.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2023-11-27T15:24:38.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-11-27T15:24:38.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-11-27T15:24:38.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2023-11-27T15:24:38.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-11-27T15:24:38.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-11-27T15:24:38.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-11-27T15:24:38.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2023-11-27T15:24:38.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2023-11-27T15:24:38.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:38 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]: dispatch 2023-11-27T15:24:39.811 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:24:39.811 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3136956756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-27T15:24:39.811 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2023-11-27T15:24:39.811 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-11-27T15:24:39.811 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-11-27T15:24:39.812 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-11-27T15:24:39.812 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-11-27T15:24:39.812 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-11-27T15:24:39.812 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-11-27T15:24:39.812 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-11-27T15:24:39.812 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2023-11-27T15:24:39.812 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2023-11-27T15:24:39.812 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:39 smithi008 ceph-mon[128788]: osdmap e89: 12 total, 12 up, 12 in 2023-11-27T15:24:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:24:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3136956756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-27T15:24:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2023-11-27T15:24:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-11-27T15:24:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-11-27T15:24:39.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-11-27T15:24:39.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-11-27T15:24:39.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-11-27T15:24:39.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-11-27T15:24:39.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-11-27T15:24:39.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2023-11-27T15:24:39.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2023-11-27T15:24:39.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:39 smithi060 ceph-mon[131518]: osdmap e89: 12 total, 12 up, 12 in 2023-11-27T15:24:39.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:24:39.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3136956756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-27T15:24:39.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2023-11-27T15:24:39.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-11-27T15:24:39.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-11-27T15:24:39.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2023-11-27T15:24:39.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-11-27T15:24:39.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-11-27T15:24:39.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-11-27T15:24:39.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2023-11-27T15:24:39.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2023-11-27T15:24:39.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 10]}]': finished 2023-11-27T15:24:39.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:39 smithi029 ceph-mon[131166]: osdmap e89: 12 total, 12 up, 12 in 2023-11-27T15:24:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.355+0000 7fb000bb6700 1 -- 172.21.15.8:0/2904121853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faffc100ee0 msgr2=0x7faffc104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:40.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.355+0000 7fb000bb6700 1 --2- 172.21.15.8:0/2904121853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faffc100ee0 0x7faffc104eb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faff00099c0 tx=0x7faff0009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.355+0000 7fb000bb6700 1 -- 172.21.15.8:0/2904121853 shutdown_connections 2023-11-27T15:24:40.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.355+0000 7fb000bb6700 1 --2- 172.21.15.8:0/2904121853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faffc1055e0 0x7faffc10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.355+0000 7fb000bb6700 1 --2- 172.21.15.8:0/2904121853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc100540 0x7faffc100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.355+0000 7fb000bb6700 1 --2- 172.21.15.8:0/2904121853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faffc100ee0 0x7faffc104eb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.355+0000 7fb000bb6700 1 -- 172.21.15.8:0/2904121853 >> 172.21.15.8:0/2904121853 conn(0x7faffc0fbdc0 msgr2=0x7faffc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:40.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.355+0000 7fb000bb6700 1 -- 172.21.15.8:0/2904121853 shutdown_connections 2023-11-27T15:24:40.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.355+0000 7fb000bb6700 1 -- 172.21.15.8:0/2904121853 wait complete. 2023-11-27T15:24:40.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.356+0000 7fb000bb6700 1 Processor -- start 2023-11-27T15:24:40.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.356+0000 7fb000bb6700 1 -- start start 2023-11-27T15:24:40.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7fb000bb6700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc100540 0x7faffc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:40.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7fb000bb6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faffc100ee0 0x7faffc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:40.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7fb000bb6700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faffc1055e0 0x7faffc19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:40.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7fb000bb6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc19d5a0 con 0x7faffc100ee0 2023-11-27T15:24:40.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7fb000bb6700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc19d710 con 0x7faffc100540 2023-11-27T15:24:40.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7fb000bb6700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc19d8d0 con 0x7faffc1055e0 2023-11-27T15:24:40.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7faffa59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc100540 0x7faffc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:40.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7faffa59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc100540 0x7faffc1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:58788/0 (socket says 172.21.15.8:58788) 2023-11-27T15:24:40.740 INFO:teuthology.orchestra.run.smithi008.stdout:223338299437 2023-11-27T15:24:40.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7faffa59c700 1 -- 172.21.15.8:0/253496236 learned_addr learned my addr 172.21.15.8:0/253496236 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:40.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7faffad9d700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faffc1055e0 0x7faffc19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:40.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7faff9d9b700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faffc100ee0 0x7faffc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:40.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.357+0000 7faffa59c700 1 -- 172.21.15.8:0/253496236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faffc1055e0 msgr2=0x7faffc19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:40.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7faffa59c700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faffc1055e0 0x7faffc19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7faffa59c700 1 -- 172.21.15.8:0/253496236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faffc100ee0 msgr2=0x7faffc198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:40.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7faffa59c700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faffc100ee0 0x7faffc198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7faffa59c700 1 -- 172.21.15.8:0/253496236 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff00095f0 con 0x7faffc100540 2023-11-27T15:24:40.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7faffad9d700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faffc1055e0 0x7faffc19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:40.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7faffa59c700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc100540 0x7faffc1985e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fafec00e9a0 tx=0x7fafec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:40.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7fafeb7fe700 1 -- 172.21.15.8:0/253496236 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec009790 con 0x7faffc100540 2023-11-27T15:24:40.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7fb000bb6700 1 -- 172.21.15.8:0/253496236 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc110360 con 0x7faffc100540 2023-11-27T15:24:40.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7fafeb7fe700 1 -- 172.21.15.8:0/253496236 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafec010430 con 0x7faffc100540 2023-11-27T15:24:40.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7fafeb7fe700 1 -- 172.21.15.8:0/253496236 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec018680 con 0x7faffc100540 2023-11-27T15:24:40.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.358+0000 7fb000bb6700 1 -- 172.21.15.8:0/253496236 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc1101d0 con 0x7faffc100540 2023-11-27T15:24:40.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.359+0000 7fb000bb6700 1 -- 172.21.15.8:0/253496236 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc1104f0 con 0x7faffc100540 2023-11-27T15:24:40.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.361+0000 7fafeb7fe700 1 -- 172.21.15.8:0/253496236 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fafec0105a0 con 0x7faffc100540 2023-11-27T15:24:40.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.363+0000 7fafeb7fe700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fafe40a11e0 0x7fafe40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:40.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.363+0000 7fafeb7fe700 1 -- 172.21.15.8:0/253496236 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fafec014070 con 0x7faffc100540 2023-11-27T15:24:40.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.363+0000 7faff9d9b700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fafe40a11e0 0x7fafe40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:40.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.364+0000 7faff9d9b700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fafe40a11e0 0x7fafe40a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faff0009fd0 tx=0x7faff0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:40.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.365+0000 7fafeb7fe700 1 -- 172.21.15.8:0/253496236 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fafec096440 con 0x7faffc100540 2023-11-27T15:24:40.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.587+0000 7fb000bb6700 1 -- 172.21.15.8:0/253496236 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7faffc066ea0 con 0x7faffc100540 2023-11-27T15:24:40.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.588+0000 7fafeb7fe700 1 -- 172.21.15.8:0/253496236 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fafec026090 con 0x7faffc100540 2023-11-27T15:24:40.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 -- 172.21.15.8:0/253496236 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fafe40a11e0 msgr2=0x7fafe40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:40.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fafe40a11e0 0x7fafe40a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faff0009fd0 tx=0x7faff0005fb0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 -- 172.21.15.8:0/253496236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc100540 msgr2=0x7faffc1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:40.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc100540 0x7faffc1985e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fafec00e9a0 tx=0x7fafec00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.743 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 -- 172.21.15.8:0/253496236 shutdown_connections 2023-11-27T15:24:40.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fafe40a11e0 0x7fafe40a3690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faffc1055e0 0x7faffc19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faffc100540 0x7faffc1985e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 --2- 172.21.15.8:0/253496236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faffc100ee0 0x7faffc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 -- 172.21.15.8:0/253496236 >> 172.21.15.8:0/253496236 conn(0x7faffc0fbdc0 msgr2=0x7faffc0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:40.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.591+0000 7fb000bb6700 1 -- 172.21.15.8:0/253496236 shutdown_connections 2023-11-27T15:24:40.744 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.592+0000 7fb000bb6700 1 -- 172.21.15.8:0/253496236 wait complete. 2023-11-27T15:24:40.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.659+0000 7f7eb9c72700 1 -- 172.21.15.8:0/3122843187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7eb41025c0 msgr2=0x7f7eb4102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:40.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.659+0000 7f7eb9c72700 1 --2- 172.21.15.8:0/3122843187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7eb41025c0 0x7f7eb4102a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009990 tx=0x7f7ea8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.659+0000 7f7eb9c72700 1 -- 172.21.15.8:0/3122843187 shutdown_connections 2023-11-27T15:24:40.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.659+0000 7f7eb9c72700 1 --2- 172.21.15.8:0/3122843187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb4102f50 0x7f7eb410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.659+0000 7f7eb9c72700 1 --2- 172.21.15.8:0/3122843187 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb41086b0 0x7f7eb4108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.659+0000 7f7eb9c72700 1 --2- 172.21.15.8:0/3122843187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7eb41025c0 0x7f7eb4102a10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.659+0000 7f7eb9c72700 1 -- 172.21.15.8:0/3122843187 >> 172.21.15.8:0/3122843187 conn(0x7f7eb40fe100 msgr2=0x7f7eb4100510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:40.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.659+0000 7f7eb9c72700 1 -- 172.21.15.8:0/3122843187 shutdown_connections 2023-11-27T15:24:40.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.659+0000 7f7eb9c72700 1 -- 172.21.15.8:0/3122843187 wait complete. 2023-11-27T15:24:40.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.660+0000 7f7eb9c72700 1 Processor -- start 2023-11-27T15:24:40.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.660+0000 7f7eb9c72700 1 -- start start 2023-11-27T15:24:40.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb9c72700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb41025c0 0x7f7eb4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:40.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb9c72700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb4102f50 0x7f7eb4198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:40.784 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb9c72700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7eb41086b0 0x7f7eb419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb9c72700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb419d600 con 0x7f7eb41086b0 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb9c72700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb419d770 con 0x7f7eb4102f50 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb9c72700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb419d930 con 0x7f7eb41025c0 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7eb41086b0 0x7f7eb419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7eb41086b0 0x7f7eb419cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38440/0 (socket says 172.21.15.8:38440) 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb3fff700 1 -- 172.21.15.8:0/2455904319 learned_addr learned my addr 172.21.15.8:0/2455904319 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb37fe700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb41025c0 0x7f7eb4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb3fff700 1 -- 172.21.15.8:0/2455904319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb41025c0 msgr2=0x7f7eb4198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.661+0000 7f7eb3fff700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb41025c0 0x7f7eb4198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.662+0000 7f7eb3fff700 1 -- 172.21.15.8:0/2455904319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb4102f50 msgr2=0x7f7eb4198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:40.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.662+0000 7f7eb3fff700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb4102f50 0x7f7eb4198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.662+0000 7f7eb3fff700 1 -- 172.21.15.8:0/2455904319 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac009c40 con 0x7f7eb41086b0 2023-11-27T15:24:40.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.662+0000 7f7eb3fff700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7eb41086b0 0x7f7eb419cff0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00c370 tx=0x7f7eac00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:40.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.662+0000 7f7eb0ff9700 1 -- 172.21.15.8:0/2455904319 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac017070 con 0x7f7eb41086b0 2023-11-27T15:24:40.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.662+0000 7f7eb0ff9700 1 -- 172.21.15.8:0/2455904319 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7eac00f040 con 0x7f7eb41086b0 2023-11-27T15:24:40.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.662+0000 7f7eb9c72700 1 -- 172.21.15.8:0/2455904319 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea4009520 con 0x7f7eb41086b0 2023-11-27T15:24:40.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.662+0000 7f7eb0ff9700 1 -- 172.21.15.8:0/2455904319 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00e050 con 0x7f7eb41086b0 2023-11-27T15:24:40.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.662+0000 7f7eb9c72700 1 -- 172.21.15.8:0/2455904319 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea80095f0 con 0x7f7eb41086b0 2023-11-27T15:24:40.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.664+0000 7f7eb9c72700 1 -- 172.21.15.8:0/2455904319 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb4066ea0 con 0x7f7eb41086b0 2023-11-27T15:24:40.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.665+0000 7f7eb0ff9700 1 -- 172.21.15.8:0/2455904319 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f7eac004460 con 0x7f7eb41086b0 2023-11-27T15:24:40.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.667+0000 7f7eb0ff9700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e9c0a11e0 0x7f7e9c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:40.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.667+0000 7f7eb0ff9700 1 -- 172.21.15.8:0/2455904319 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7eac0cc7c0 con 0x7f7eb41086b0 2023-11-27T15:24:40.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.668+0000 7f7eb37fe700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e9c0a11e0 0x7f7e9c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:40.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.668+0000 7f7eb37fe700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e9c0a11e0 0x7f7e9c0a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009d70 tx=0x7f7ea4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:40.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.671+0000 7f7eb0ff9700 1 -- 172.21.15.8:0/2455904319 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7eac095a40 con 0x7f7eb41086b0 2023-11-27T15:24:40.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.930+0000 7f7eb9c72700 1 -- 172.21.15.8:0/2455904319 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7eb41a5340 con 0x7f7eb41086b0 2023-11-27T15:24:40.934 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.930+0000 7f7eb0ff9700 1 -- 172.21.15.8:0/2455904319 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7eac099060 con 0x7f7eb41086b0 2023-11-27T15:24:40.935 INFO:teuthology.orchestra.run.smithi008.stdout:111669149761 2023-11-27T15:24:40.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 -- 172.21.15.8:0/2455904319 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e9c0a11e0 msgr2=0x7f7e9c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:40.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e9c0a11e0 0x7f7e9c0a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4009d70 tx=0x7f7ea4009260 comp rx=0 tx=0).stop 2023-11-27T15:24:40.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 -- 172.21.15.8:0/2455904319 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7eb41086b0 msgr2=0x7f7eb419cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:40.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7eb41086b0 0x7f7eb419cff0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00c370 tx=0x7f7eac00c680 comp rx=0 tx=0).stop 2023-11-27T15:24:40.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 -- 172.21.15.8:0/2455904319 shutdown_connections 2023-11-27T15:24:40.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e9c0a11e0 0x7f7e9c0a3690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7eb41025c0 0x7f7eb4198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7eb41086b0 0x7f7eb419cff0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 --2- 172.21.15.8:0/2455904319 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7eb4102f50 0x7f7eb4198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:40.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 -- 172.21.15.8:0/2455904319 >> 172.21.15.8:0/2455904319 conn(0x7f7eb40fe100 msgr2=0x7f7eb40ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:40.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.933+0000 7f7eb9c72700 1 -- 172.21.15.8:0/2455904319 shutdown_connections 2023-11-27T15:24:40.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:40.934+0000 7f7eb9c72700 1 -- 172.21.15.8:0/2455904319 wait complete. 2023-11-27T15:24:41.522 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:41 smithi008 ceph-mon[128788]: osdmap e90: 12 total, 12 up, 12 in 2023-11-27T15:24:41.522 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:41 smithi008 ceph-mon[128788]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2023-11-27T15:24:41.522 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:41 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/253496236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-27T15:24:41.522 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:41 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2455904319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-27T15:24:41.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:41 smithi060 ceph-mon[131518]: osdmap e90: 12 total, 12 up, 12 in 2023-11-27T15:24:41.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:41 smithi060 ceph-mon[131518]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2023-11-27T15:24:41.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:41 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/253496236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-27T15:24:41.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:41 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2455904319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-27T15:24:41.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:41 smithi029 ceph-mon[131166]: osdmap e90: 12 total, 12 up, 12 in 2023-11-27T15:24:41.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:41 smithi029 ceph-mon[131166]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2023-11-27T15:24:41.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:41 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/253496236' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-27T15:24:41.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:41 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2455904319' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-27T15:24:42.244 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574921 for osd.1 2023-11-27T15:24:42.245 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:42.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299437 for osd.7 2023-11-27T15:24:42.245 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:42.539 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149758 got 111669149761 for osd.3 2023-11-27T15:24:42.540 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:42.608 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738446 for osd.0 2023-11-27T15:24:42.608 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.859+0000 7fc82e2fe700 1 -- 172.21.15.8:0/189435580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc828100540 msgr2=0x7fc828100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:42.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.859+0000 7fc82e2fe700 1 --2- 172.21.15.8:0/189435580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc828100540 0x7fc828100910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc8180098f0 tx=0x7fc818009c00 comp rx=0 tx=0).stop 2023-11-27T15:24:42.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.859+0000 7fc82e2fe700 1 -- 172.21.15.8:0/189435580 shutdown_connections 2023-11-27T15:24:42.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.859+0000 7fc82e2fe700 1 --2- 172.21.15.8:0/189435580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8281055e0 0x7fc82810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:42.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.859+0000 7fc82e2fe700 1 --2- 172.21.15.8:0/189435580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc828100540 0x7fc828100910 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:42.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.859+0000 7fc82e2fe700 1 --2- 172.21.15.8:0/189435580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc828100ee0 0x7fc828104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:42.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.859+0000 7fc82e2fe700 1 -- 172.21.15.8:0/189435580 >> 172.21.15.8:0/189435580 conn(0x7fc8280fbdc0 msgr2=0x7fc8280fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:42.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.859+0000 7fc82e2fe700 1 -- 172.21.15.8:0/189435580 shutdown_connections 2023-11-27T15:24:42.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.859+0000 7fc82e2fe700 1 -- 172.21.15.8:0/189435580 wait complete. 2023-11-27T15:24:42.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.860+0000 7fc82e2fe700 1 Processor -- start 2023-11-27T15:24:42.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.860+0000 7fc82e2fe700 1 -- start start 2023-11-27T15:24:42.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.860+0000 7fc82e2fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc828100540 0x7fc82819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:42.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.860+0000 7fc82e2fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc828100ee0 0x7fc82819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:42.864 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.860+0000 7fc82e2fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8281055e0 0x7fc8281985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:42.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.860+0000 7fc82e2fe700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828198c10 con 0x7fc8281055e0 2023-11-27T15:24:42.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc82e2fe700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828198d80 con 0x7fc828100540 2023-11-27T15:24:42.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc82e2fe700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828198f40 con 0x7fc828100ee0 2023-11-27T15:24:42.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc82c89b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8281055e0 0x7fc8281985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:42.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc82c89b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8281055e0 0x7fc8281985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38458/0 (socket says 172.21.15.8:38458) 2023-11-27T15:24:42.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc8277fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc828100ee0 0x7fc82819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:42.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc82c89b700 1 -- 172.21.15.8:0/3518080315 learned_addr learned my addr 172.21.15.8:0/3518080315 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:42.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc827fff700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc828100540 0x7fc82819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc82c89b700 1 -- 172.21.15.8:0/3518080315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc828100ee0 msgr2=0x7fc82819e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc82c89b700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc828100ee0 0x7fc82819e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc82c89b700 1 -- 172.21.15.8:0/3518080315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc828100540 msgr2=0x7fc82819e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc82c89b700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc828100540 0x7fc82819e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc82c89b700 1 -- 172.21.15.8:0/3518080315 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818009520 con 0x7fc8281055e0 2023-11-27T15:24:42.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc8277fe700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc828100ee0 0x7fc82819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.861+0000 7fc827fff700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc828100540 0x7fc82819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.862+0000 7fc82c89b700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8281055e0 0x7fc8281985a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc82000ef30 tx=0x7fc82000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.862+0000 7fc8257fa700 1 -- 172.21.15.8:0/3518080315 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820018070 con 0x7fc8281055e0 2023-11-27T15:24:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.862+0000 7fc8257fa700 1 -- 172.21.15.8:0/3518080315 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc82000f040 con 0x7fc8281055e0 2023-11-27T15:24:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.862+0000 7fc82e2fe700 1 -- 172.21.15.8:0/3518080315 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc828110370 con 0x7fc8281055e0 2023-11-27T15:24:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.862+0000 7fc82e2fe700 1 -- 172.21.15.8:0/3518080315 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8281101e0 con 0x7fc8281055e0 2023-11-27T15:24:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.862+0000 7fc8257fa700 1 -- 172.21.15.8:0/3518080315 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82000ccf0 con 0x7fc8281055e0 2023-11-27T15:24:42.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.863+0000 7fc82e2fe700 1 -- 172.21.15.8:0/3518080315 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc828110500 con 0x7fc8281055e0 2023-11-27T15:24:42.869 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.866+0000 7fc8257fa700 1 -- 172.21.15.8:0/3518080315 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc820003a00 con 0x7fc8281055e0 2023-11-27T15:24:42.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.867+0000 7fc8257fa700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8100a11e0 0x7fc8100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:42.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.867+0000 7fc8257fa700 1 -- 172.21.15.8:0/3518080315 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc8200cc8a0 con 0x7fc8281055e0 2023-11-27T15:24:42.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.867+0000 7fc827fff700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8100a11e0 0x7fc8100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:42.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.868+0000 7fc827fff700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8100a11e0 0x7fc8100a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc818005e90 tx=0x7fc818019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:42.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.871+0000 7fc8257fa700 1 -- 172.21.15.8:0/3518080315 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc820095bb0 con 0x7fc8281055e0 2023-11-27T15:24:42.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.930+0000 7f2d90b2e700 1 -- 172.21.15.8:0/2146680783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c101e50 msgr2=0x7f2d8c10a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:42.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.930+0000 7f2d90b2e700 1 --2- 172.21.15.8:0/2146680783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c101e50 0x7f2d8c10a350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d800098f0 tx=0x7f2d80009c00 comp rx=0 tx=0).stop 2023-11-27T15:24:42.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.930+0000 7f2d90b2e700 1 -- 172.21.15.8:0/2146680783 shutdown_connections 2023-11-27T15:24:42.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.930+0000 7f2d90b2e700 1 --2- 172.21.15.8:0/2146680783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8c101540 0x7f2d8c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:42.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.930+0000 7f2d90b2e700 1 --2- 172.21.15.8:0/2146680783 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d8c10a890 0x7f2d8c10ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:42.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.930+0000 7f2d90b2e700 1 --2- 172.21.15.8:0/2146680783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c101e50 0x7f2d8c10a350 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:42.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.930+0000 7f2d90b2e700 1 -- 172.21.15.8:0/2146680783 >> 172.21.15.8:0/2146680783 conn(0x7f2d8c0770a0 msgr2=0x7f2d8c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:42.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.930+0000 7f2d90b2e700 1 -- 172.21.15.8:0/2146680783 shutdown_connections 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.930+0000 7f2d90b2e700 1 -- 172.21.15.8:0/2146680783 wait complete. 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.931+0000 7f2d90b2e700 1 Processor -- start 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.932+0000 7f2d90b2e700 1 -- start start 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.932+0000 7f2d90b2e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d8c101540 0x7f2d8c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.932+0000 7f2d90b2e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8c101e50 0x7f2d8c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.932+0000 7f2d90b2e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c10a890 0x7f2d8c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.932+0000 7f2d90b2e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c198c00 con 0x7f2d8c101540 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.932+0000 7f2d90b2e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c198d70 con 0x7f2d8c10a890 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.932+0000 7f2d90b2e700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c198f30 con 0x7f2d8c101e50 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.932+0000 7f2d8a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d8c101540 0x7f2d8c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.932+0000 7f2d8a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d8c101540 0x7f2d8c19e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38466/0 (socket says 172.21.15.8:38466) 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.932+0000 7f2d8a59c700 1 -- 172.21.15.8:0/236495280 learned_addr learned my addr 172.21.15.8:0/236495280 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:42.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.933+0000 7f2d89d9b700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8c101e50 0x7f2d8c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.933+0000 7f2d8a59c700 1 -- 172.21.15.8:0/236495280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8c101e50 msgr2=0x7f2d8c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.933+0000 7f2d8a59c700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8c101e50 0x7f2d8c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.933+0000 7f2d8a59c700 1 -- 172.21.15.8:0/236495280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c10a890 msgr2=0x7f2d8c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.933+0000 7f2d8a59c700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c10a890 0x7f2d8c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.933+0000 7f2d8a59c700 1 -- 172.21.15.8:0/236495280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c0095f0 con 0x7f2d8c101540 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.933+0000 7f2d8a59c700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d8c101540 0x7f2d8c19e510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00ec10 tx=0x7f2d7c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.933+0000 7f2d7b7fe700 1 -- 172.21.15.8:0/236495280 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c009a10 con 0x7f2d8c101540 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.934+0000 7f2d90b2e700 1 -- 172.21.15.8:0/236495280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d80009520 con 0x7f2d8c101540 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.934+0000 7f2d90b2e700 1 -- 172.21.15.8:0/236495280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c110360 con 0x7f2d8c101540 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.934+0000 7f2d7b7fe700 1 -- 172.21.15.8:0/236495280 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d7c0044e0 con 0x7f2d8c101540 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.934+0000 7f2d7b7fe700 1 -- 172.21.15.8:0/236495280 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c010450 con 0x7f2d8c101540 2023-11-27T15:24:42.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.936+0000 7f2d90b2e700 1 -- 172.21.15.8:0/236495280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c005320 con 0x7f2d8c101540 2023-11-27T15:24:42.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.938+0000 7f2d7b7fe700 1 -- 172.21.15.8:0/236495280 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f2d7c004020 con 0x7f2d8c101540 2023-11-27T15:24:42.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.941+0000 7f2d7b7fe700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2d740a1260 0x7f2d740a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:42.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.941+0000 7f2d89d9b700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2d740a1260 0x7f2d740a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:42.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.943+0000 7f2d7b7fe700 1 -- 172.21.15.8:0/236495280 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2d7c014070 con 0x7f2d8c101540 2023-11-27T15:24:42.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.943+0000 7f2d89d9b700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2d740a1260 0x7f2d740a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0febb0 tx=0x7f2d80019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:42.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:42.950+0000 7f2d7b7fe700 1 -- 172.21.15.8:0/236495280 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2d7c096550 con 0x7f2d8c101540 2023-11-27T15:24:43.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.073+0000 7f5ae9d71700 1 -- 172.21.15.8:0/795881384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ae40ff330 msgr2=0x7f5ae40ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.077 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.073+0000 7f5ae9d71700 1 --2- 172.21.15.8:0/795881384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ae40ff330 0x7f5ae40ff700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009990 tx=0x7f5ad4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.074+0000 7f5ae9d71700 1 -- 172.21.15.8:0/795881384 shutdown_connections 2023-11-27T15:24:43.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.074+0000 7f5ae9d71700 1 --2- 172.21.15.8:0/795881384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ae4104660 0x7f5ae4104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.074+0000 7f5ae9d71700 1 --2- 172.21.15.8:0/795881384 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ae40ff330 0x7f5ae40ff700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.074+0000 7f5ae9d71700 1 --2- 172.21.15.8:0/795881384 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae40ffc40 0x7f5ae4104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.074+0000 7f5ae9d71700 1 -- 172.21.15.8:0/795881384 >> 172.21.15.8:0/795881384 conn(0x7f5ae40770a0 msgr2=0x7f5ae40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:43.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.074+0000 7f5ae9d71700 1 -- 172.21.15.8:0/795881384 shutdown_connections 2023-11-27T15:24:43.078 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.074+0000 7f5ae9d71700 1 -- 172.21.15.8:0/795881384 wait complete. 2023-11-27T15:24:43.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.075+0000 7f5ae9d71700 1 Processor -- start 2023-11-27T15:24:43.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.075+0000 7f5ae9d71700 1 -- start start 2023-11-27T15:24:43.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.075+0000 7f5ae9d71700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ae40ff330 0x7f5ae41a2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:43.079 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.075+0000 7f5ae9d71700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ae40ffc40 0x7f5ae41a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:43.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae9d71700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4104660 0x7f5ae419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:43.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae9d71700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae419ce70 con 0x7f5ae40ff330 2023-11-27T15:24:43.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae9d71700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae419cfe0 con 0x7f5ae40ffc40 2023-11-27T15:24:43.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae9d71700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae419d1a0 con 0x7f5ae4104660 2023-11-27T15:24:43.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ae40ff330 0x7f5ae41a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:43.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ae40ff330 0x7f5ae41a2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38496/0 (socket says 172.21.15.8:38496) 2023-11-27T15:24:43.081 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae37fe700 1 -- 172.21.15.8:0/161358312 learned_addr learned my addr 172.21.15.8:0/161358312 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae2ffd700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ae40ffc40 0x7f5ae41a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae37fe700 1 -- 172.21.15.8:0/161358312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4104660 msgr2=0x7f5ae419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae3fff700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4104660 0x7f5ae419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae37fe700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4104660 0x7f5ae419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae37fe700 1 -- 172.21.15.8:0/161358312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ae40ffc40 msgr2=0x7f5ae41a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae37fe700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ae40ffc40 0x7f5ae41a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.076+0000 7f5ae37fe700 1 -- 172.21.15.8:0/161358312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad40095f0 con 0x7f5ae40ff330 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.077+0000 7f5ae3fff700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4104660 0x7f5ae419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.077+0000 7f5ae2ffd700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ae40ffc40 0x7f5ae41a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.077+0000 7f5ae37fe700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ae40ff330 0x7f5ae41a2780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40095c0 tx=0x7f5ad400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:43.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.077+0000 7f5ae0ff9700 1 -- 172.21.15.8:0/161358312 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad401d070 con 0x7f5ae40ff330 2023-11-27T15:24:43.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.077+0000 7f5ae9d71700 1 -- 172.21.15.8:0/161358312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae4116950 con 0x7f5ae40ff330 2023-11-27T15:24:43.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.077+0000 7f5ae0ff9700 1 -- 172.21.15.8:0/161358312 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ad4005510 con 0x7f5ae40ff330 2023-11-27T15:24:43.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.077+0000 7f5ae0ff9700 1 -- 172.21.15.8:0/161358312 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4004bd0 con 0x7f5ae40ff330 2023-11-27T15:24:43.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.078+0000 7f5ae9d71700 1 -- 172.21.15.8:0/161358312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae41167c0 con 0x7f5ae40ff330 2023-11-27T15:24:43.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.079+0000 7f5aca7fc700 1 -- 172.21.15.8:0/161358312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae4116ae0 con 0x7f5ae40ff330 2023-11-27T15:24:43.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.081+0000 7f5ae0ff9700 1 -- 172.21.15.8:0/161358312 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f5ad4004d50 con 0x7f5ae40ff330 2023-11-27T15:24:43.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.083+0000 7f5ae0ff9700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5acc0a1260 0x7f5acc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:43.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.083+0000 7f5ae0ff9700 1 -- 172.21.15.8:0/161358312 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5ad40cdca0 con 0x7f5ae40ff330 2023-11-27T15:24:43.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.084+0000 7f5ae2ffd700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5acc0a1260 0x7f5acc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:43.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.085+0000 7f5ae2ffd700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5acc0a1260 0x7f5acc0a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8005d90 tx=0x7f5ad8005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:43.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.091+0000 7f5ae0ff9700 1 -- 172.21.15.8:0/161358312 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5ad4096fb0 con 0x7f5ae40ff330 2023-11-27T15:24:43.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.185+0000 7f2d90b2e700 1 -- 172.21.15.8:0/236495280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2d6c005190 con 0x7f2d8c101540 2023-11-27T15:24:43.189 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.185+0000 7f2d7b7fe700 1 -- 172.21.15.8:0/236495280 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d7c099b70 con 0x7f2d8c101540 2023-11-27T15:24:43.189 INFO:teuthology.orchestra.run.smithi008.stdout:201863462963 2023-11-27T15:24:43.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 -- 172.21.15.8:0/236495280 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2d740a1260 msgr2=0x7f2d740a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2d740a1260 0x7f2d740a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0febb0 tx=0x7f2d80019040 comp rx=0 tx=0).stop 2023-11-27T15:24:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 -- 172.21.15.8:0/236495280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d8c101540 msgr2=0x7f2d8c19e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d8c101540 0x7f2d8c19e510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00ec10 tx=0x7f2d7c00c2d0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 -- 172.21.15.8:0/236495280 shutdown_connections 2023-11-27T15:24:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2d8c10a890 0x7f2d8c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d8c101540 0x7f2d8c19e510 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f2d740a1260 0x7f2d740a3710 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 --2- 172.21.15.8:0/236495280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2d8c101e50 0x7f2d8c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 -- 172.21.15.8:0/236495280 >> 172.21.15.8:0/236495280 conn(0x7f2d8c0770a0 msgr2=0x7f2d8c0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.188+0000 7f2d90b2e700 1 -- 172.21.15.8:0/236495280 shutdown_connections 2023-11-27T15:24:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.189+0000 7f2d90b2e700 1 -- 172.21.15.8:0/236495280 wait complete. 2023-11-27T15:24:43.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.195+0000 7fc82e2fe700 1 -- 172.21.15.8:0/3518080315 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fc8281a5720 con 0x7fc8281055e0 2023-11-27T15:24:43.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.196+0000 7fc8257fa700 1 -- 172.21.15.8:0/3518080315 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc8200991d0 con 0x7fc8281055e0 2023-11-27T15:24:43.199 INFO:teuthology.orchestra.run.smithi008.stdout:339302416411 2023-11-27T15:24:43.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.198+0000 7fc82e2fe700 1 -- 172.21.15.8:0/3518080315 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8100a11e0 msgr2=0x7fc8100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.198+0000 7fc82e2fe700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8100a11e0 0x7fc8100a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc818005e90 tx=0x7fc818019040 comp rx=0 tx=0).stop 2023-11-27T15:24:43.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.198+0000 7fc82e2fe700 1 -- 172.21.15.8:0/3518080315 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8281055e0 msgr2=0x7fc8281985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.198+0000 7fc82e2fe700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8281055e0 0x7fc8281985a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc82000ef30 tx=0x7fc82000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.199+0000 7fc82e2fe700 1 -- 172.21.15.8:0/3518080315 shutdown_connections 2023-11-27T15:24:43.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.199+0000 7fc82e2fe700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8281055e0 0x7fc8281985a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.199+0000 7fc82e2fe700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc828100540 0x7fc82819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.199+0000 7fc82e2fe700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8100a11e0 0x7fc8100a3690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.199+0000 7fc82e2fe700 1 --2- 172.21.15.8:0/3518080315 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc828100ee0 0x7fc82819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.199+0000 7fc82e2fe700 1 -- 172.21.15.8:0/3518080315 >> 172.21.15.8:0/3518080315 conn(0x7fc8280fbdc0 msgr2=0x7fc8280fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:43.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.199+0000 7fc82e2fe700 1 -- 172.21.15.8:0/3518080315 shutdown_connections 2023-11-27T15:24:43.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.199+0000 7fc82e2fe700 1 -- 172.21.15.8:0/3518080315 wait complete. 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.233+0000 7fee4a075700 1 -- 172.21.15.8:0/626930547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee44100ee0 msgr2=0x7fee44104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.233+0000 7fee4a075700 1 --2- 172.21.15.8:0/626930547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee44100ee0 0x7fee44104eb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fee380099c0 tx=0x7fee38009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.233+0000 7fee4a075700 1 -- 172.21.15.8:0/626930547 shutdown_connections 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.233+0000 7fee4a075700 1 --2- 172.21.15.8:0/626930547 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee44100540 0x7fee44100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.233+0000 7fee4a075700 1 --2- 172.21.15.8:0/626930547 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee441055e0 0x7fee4410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.233+0000 7fee4a075700 1 --2- 172.21.15.8:0/626930547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee44100ee0 0x7fee44104eb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.233+0000 7fee4a075700 1 -- 172.21.15.8:0/626930547 >> 172.21.15.8:0/626930547 conn(0x7fee440fbdc0 msgr2=0x7fee440fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.233+0000 7fee4a075700 1 -- 172.21.15.8:0/626930547 shutdown_connections 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.233+0000 7fee4a075700 1 -- 172.21.15.8:0/626930547 wait complete. 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.234+0000 7fee4a075700 1 Processor -- start 2023-11-27T15:24:43.251 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.234+0000 7fee4a075700 1 -- start start 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.234+0000 7fee4a075700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee44100540 0x7fee441986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.234+0000 7fee4a075700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee44100ee0 0x7fee44198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.234+0000 7fee4a075700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee441055e0 0x7fee4419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee4a075700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4419d5c0 con 0x7fee44100540 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee4a075700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4419d730 con 0x7fee441055e0 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee4a075700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4419d8f0 con 0x7fee44100ee0 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee42ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee44100ee0 0x7fee44198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee42ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee44100ee0 0x7fee44198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:37060/0 (socket says 172.21.15.8:37060) 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee42ffd700 1 -- 172.21.15.8:0/818983505 learned_addr learned my addr 172.21.15.8:0/818983505 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee43fff700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee441055e0 0x7fee4419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee437fe700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee44100540 0x7fee441986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:43.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee42ffd700 1 -- 172.21.15.8:0/818983505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee441055e0 msgr2=0x7fee4419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee42ffd700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee441055e0 0x7fee4419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee42ffd700 1 -- 172.21.15.8:0/818983505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee44100540 msgr2=0x7fee441986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee42ffd700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee44100540 0x7fee441986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.235+0000 7fee42ffd700 1 -- 172.21.15.8:0/818983505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee380095f0 con 0x7fee44100ee0 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.236+0000 7fee42ffd700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee44100ee0 0x7fee44198be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fee38009fd0 tx=0x7fee3800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.236+0000 7fee40ff9700 1 -- 172.21.15.8:0/818983505 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3801d070 con 0x7fee44100ee0 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.236+0000 7fee4a075700 1 -- 172.21.15.8:0/818983505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee44110360 con 0x7fee44100ee0 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.236+0000 7fee40ff9700 1 -- 172.21.15.8:0/818983505 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee38005510 con 0x7fee44100ee0 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.236+0000 7fee40ff9700 1 -- 172.21.15.8:0/818983505 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38004b30 con 0x7fee44100ee0 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.236+0000 7fee4a075700 1 -- 172.21.15.8:0/818983505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee441101d0 con 0x7fee44100ee0 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.237+0000 7fee4a075700 1 -- 172.21.15.8:0/818983505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee441104f0 con 0x7fee44100ee0 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.239+0000 7fee40ff9700 1 -- 172.21.15.8:0/818983505 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fee3800fb80 con 0x7fee44100ee0 2023-11-27T15:24:43.253 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.241+0000 7fee40ff9700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee2c0a1260 0x7fee2c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:43.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.241+0000 7fee40ff9700 1 -- 172.21.15.8:0/818983505 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fee380cdb20 con 0x7fee44100ee0 2023-11-27T15:24:43.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.241+0000 7fee437fe700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee2c0a1260 0x7fee2c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:43.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.242+0000 7fee437fe700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee2c0a1260 0x7fee2c0a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee34009bf0 tx=0x7fee34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:43.254 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.243+0000 7fee40ff9700 1 -- 172.21.15.8:0/818983505 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fee38096e30 con 0x7fee44100ee0 2023-11-27T15:24:43.268 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:43.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.357+0000 7f5aca7fc700 1 -- 172.21.15.8:0/161358312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5ae4066ea0 con 0x7f5ae40ff330 2023-11-27T15:24:43.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.358+0000 7f5ae0ff9700 1 -- 172.21.15.8:0/161358312 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ad409a5d0 con 0x7f5ae40ff330 2023-11-27T15:24:43.362 INFO:teuthology.orchestra.run.smithi008.stdout:146028888126 2023-11-27T15:24:43.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.362+0000 7f5aca7fc700 1 -- 172.21.15.8:0/161358312 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5acc0a1260 msgr2=0x7f5acc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.362+0000 7f5aca7fc700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5acc0a1260 0x7f5acc0a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8005d90 tx=0x7f5ad8005d00 comp rx=0 tx=0).stop 2023-11-27T15:24:43.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.362+0000 7f5aca7fc700 1 -- 172.21.15.8:0/161358312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ae40ff330 msgr2=0x7f5ae41a2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.362+0000 7f5aca7fc700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ae40ff330 0x7f5ae41a2780 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40095c0 tx=0x7f5ad400f690 comp rx=0 tx=0).stop 2023-11-27T15:24:43.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.362+0000 7f5aca7fc700 1 -- 172.21.15.8:0/161358312 shutdown_connections 2023-11-27T15:24:43.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.362+0000 7f5aca7fc700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f5acc0a1260 0x7f5acc0a3710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.362+0000 7f5aca7fc700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f5ae4104660 0x7f5ae419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.362+0000 7f5aca7fc700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ae40ff330 0x7f5ae41a2780 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.362+0000 7f5aca7fc700 1 --2- 172.21.15.8:0/161358312 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f5ae40ffc40 0x7f5ae41a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.362+0000 7f5aca7fc700 1 -- 172.21.15.8:0/161358312 >> 172.21.15.8:0/161358312 conn(0x7f5ae40770a0 msgr2=0x7f5ae41021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:43.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.363+0000 7f5aca7fc700 1 -- 172.21.15.8:0/161358312 shutdown_connections 2023-11-27T15:24:43.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.364+0000 7f5aca7fc700 1 -- 172.21.15.8:0/161358312 wait complete. 2023-11-27T15:24:43.489 INFO:teuthology.orchestra.run.smithi008.stdout:77309411400 2023-11-27T15:24:43.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.485+0000 7fee4a075700 1 -- 172.21.15.8:0/818983505 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fee44066ea0 con 0x7fee44100ee0 2023-11-27T15:24:43.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.485+0000 7fee40ff9700 1 -- 172.21.15.8:0/818983505 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fee3809a450 con 0x7fee44100ee0 2023-11-27T15:24:43.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.488+0000 7fee4a075700 1 -- 172.21.15.8:0/818983505 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee2c0a1260 msgr2=0x7fee2c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.488+0000 7fee4a075700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee2c0a1260 0x7fee2c0a3710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fee34009bf0 tx=0x7fee34009260 comp rx=0 tx=0).stop 2023-11-27T15:24:43.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.488+0000 7fee4a075700 1 -- 172.21.15.8:0/818983505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee44100ee0 msgr2=0x7fee44198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:43.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.488+0000 7fee4a075700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee44100ee0 0x7fee44198be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fee38009fd0 tx=0x7fee3800f690 comp rx=0 tx=0).stop 2023-11-27T15:24:43.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.488+0000 7fee4a075700 1 -- 172.21.15.8:0/818983505 shutdown_connections 2023-11-27T15:24:43.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.488+0000 7fee4a075700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee441055e0 0x7fee4419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.488+0000 7fee4a075700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee44100540 0x7fee441986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.488+0000 7fee4a075700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee2c0a1260 0x7fee2c0a3710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.488+0000 7fee4a075700 1 --2- 172.21.15.8:0/818983505 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee44100ee0 0x7fee44198be0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:43.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.488+0000 7fee4a075700 1 -- 172.21.15.8:0/818983505 >> 172.21.15.8:0/818983505 conn(0x7fee440fbdc0 msgr2=0x7fee440fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:43.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.489+0000 7fee4a075700 1 -- 172.21.15.8:0/818983505 shutdown_connections 2023-11-27T15:24:43.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:43.489+0000 7fee4a075700 1 -- 172.21.15.8:0/818983505 wait complete. 2023-11-27T15:24:43.511 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:43 smithi008 ceph-mon[128788]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 13 B/s, 0 objects/s recovering 2023-11-27T15:24:43.511 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:43 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/236495280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-27T15:24:43.512 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:43 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3518080315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-27T15:24:43.512 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:43 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/161358312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-27T15:24:43.773 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:43 smithi060 ceph-mon[131518]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 13 B/s, 0 objects/s recovering 2023-11-27T15:24:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:43 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/236495280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-27T15:24:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:43 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3518080315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-27T15:24:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:43 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/161358312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-27T15:24:43.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:43 smithi029 ceph-mon[131166]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 13 B/s, 0 objects/s recovering 2023-11-27T15:24:43.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:43 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/236495280' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-27T15:24:43.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:43 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3518080315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-27T15:24:43.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:43 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/161358312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-27T15:24:44.276 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:44.311 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416408 got 339302416411 for osd.11 2023-11-27T15:24:44.312 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:44.314 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462960 got 201863462963 for osd.6 2023-11-27T15:24:44.314 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:44.566 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:44 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/818983505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-27T15:24:44.570 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888124 got 146028888126 for osd.4 2023-11-27T15:24:44.571 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:44.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:44 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/818983505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-27T15:24:44.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:44 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/818983505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-27T15:24:44.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.903+0000 7fe4a0288700 1 -- 172.21.15.8:0/769424670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4980ffe00 msgr2=0x7fe498103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:44.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.903+0000 7fe4a0288700 1 --2- 172.21.15.8:0/769424670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4980ffe00 0x7fe498103dd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe488009990 tx=0x7fe488009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:44.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.903+0000 7fe4a0288700 1 -- 172.21.15.8:0/769424670 shutdown_connections 2023-11-27T15:24:44.907 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.903+0000 7fe4a0288700 1 --2- 172.21.15.8:0/769424670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4980ff460 0x7fe4980ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:44.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.903+0000 7fe4a0288700 1 --2- 172.21.15.8:0/769424670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe498104500 0x7fe49810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:44.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.903+0000 7fe4a0288700 1 --2- 172.21.15.8:0/769424670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4980ffe00 0x7fe498103dd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:44.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.903+0000 7fe4a0288700 1 -- 172.21.15.8:0/769424670 >> 172.21.15.8:0/769424670 conn(0x7fe4980770a0 msgr2=0x7fe4980774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:44.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.903+0000 7fe4a0288700 1 -- 172.21.15.8:0/769424670 shutdown_connections 2023-11-27T15:24:44.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.903+0000 7fe4a0288700 1 -- 172.21.15.8:0/769424670 wait complete. 2023-11-27T15:24:44.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.904+0000 7fe4a0288700 1 Processor -- start 2023-11-27T15:24:44.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.905+0000 7fe4a0288700 1 -- start start 2023-11-27T15:24:44.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.905+0000 7fe4a0288700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4980ff460 0x7fe498100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:44.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.905+0000 7fe4a0288700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4980ffe00 0x7fe4981013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:44.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.905+0000 7fe4a0288700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe498104500 0x7fe498101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:44.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.905+0000 7fe4a0288700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4981a4890 con 0x7fe4980ff460 2023-11-27T15:24:44.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.905+0000 7fe4a0288700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4981a4a00 con 0x7fe498104500 2023-11-27T15:24:44.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.905+0000 7fe4a0288700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4981a4bc0 con 0x7fe4980ffe00 2023-11-27T15:24:44.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.905+0000 7fe49e024700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4980ff460 0x7fe498100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:44.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.905+0000 7fe49e024700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4980ff460 0x7fe498100e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38522/0 (socket says 172.21.15.8:38522) 2023-11-27T15:24:44.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.905+0000 7fe49e024700 1 -- 172.21.15.8:0/2703817263 learned_addr learned my addr 172.21.15.8:0/2703817263 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:44.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.906+0000 7fe49e825700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe498104500 0x7fe498101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:44.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.906+0000 7fe49d823700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4980ffe00 0x7fe4981013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:44.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.906+0000 7fe49e024700 1 -- 172.21.15.8:0/2703817263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4980ffe00 msgr2=0x7fe4981013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:44.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.906+0000 7fe49e024700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4980ffe00 0x7fe4981013b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:44.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.906+0000 7fe49e024700 1 -- 172.21.15.8:0/2703817263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe498104500 msgr2=0x7fe498101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:44.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.906+0000 7fe49e024700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe498104500 0x7fe498101930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:44.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.906+0000 7fe49e024700 1 -- 172.21.15.8:0/2703817263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4880095f0 con 0x7fe4980ff460 2023-11-27T15:24:44.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.906+0000 7fe49d823700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4980ffe00 0x7fe4981013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:44.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.906+0000 7fe49e024700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4980ff460 0x7fe498100e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe49400e9a0 tx=0x7fe49400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:44.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.907+0000 7fe48f7fe700 1 -- 172.21.15.8:0/2703817263 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494009790 con 0x7fe4980ff460 2023-11-27T15:24:44.914 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.907+0000 7fe48f7fe700 1 -- 172.21.15.8:0/2703817263 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe49400cbd0 con 0x7fe4980ff460 2023-11-27T15:24:44.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.907+0000 7fe4a0288700 1 -- 172.21.15.8:0/2703817263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49810f280 con 0x7fe4980ff460 2023-11-27T15:24:44.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.907+0000 7fe48f7fe700 1 -- 172.21.15.8:0/2703817263 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494018680 con 0x7fe4980ff460 2023-11-27T15:24:44.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.907+0000 7fe4a0288700 1 -- 172.21.15.8:0/2703817263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49810f0f0 con 0x7fe4980ff460 2023-11-27T15:24:44.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.908+0000 7fe4a0288700 1 -- 172.21.15.8:0/2703817263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49810f410 con 0x7fe4980ff460 2023-11-27T15:24:44.916 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.912+0000 7fe48f7fe700 1 -- 172.21.15.8:0/2703817263 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe49400cd40 con 0x7fe4980ff460 2023-11-27T15:24:44.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.915+0000 7fe48f7fe700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4840a12e0 0x7fe4840a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:44.919 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.915+0000 7fe48f7fe700 1 -- 172.21.15.8:0/2703817263 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe494014070 con 0x7fe4980ff460 2023-11-27T15:24:44.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.916+0000 7fe49d823700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4840a12e0 0x7fe4840a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:44.920 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.917+0000 7fe49d823700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4840a12e0 0x7fe4840a3790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe498102660 tx=0x7fe488005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:44.923 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:44.919+0000 7fe48f7fe700 1 -- 172.21.15.8:0/2703817263 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe494096550 con 0x7fe4980ff460 2023-11-27T15:24:45.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.138+0000 7fe4a0288700 1 -- 172.21.15.8:0/2703817263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fe4981a5740 con 0x7fe4980ff460 2023-11-27T15:24:45.142 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.139+0000 7fe48f7fe700 1 -- 172.21.15.8:0/2703817263 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe494026020 con 0x7fe4980ff460 2023-11-27T15:24:45.143 INFO:teuthology.orchestra.run.smithi008.stdout:287762808868 2023-11-27T15:24:45.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 -- 172.21.15.8:0/2703817263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4840a12e0 msgr2=0x7fe4840a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:45.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4840a12e0 0x7fe4840a3790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe498102660 tx=0x7fe488005fb0 comp rx=0 tx=0).stop 2023-11-27T15:24:45.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 -- 172.21.15.8:0/2703817263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4980ff460 msgr2=0x7fe498100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:45.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4980ff460 0x7fe498100e70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe49400e9a0 tx=0x7fe49400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:45.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 -- 172.21.15.8:0/2703817263 shutdown_connections 2023-11-27T15:24:45.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe498104500 0x7fe498101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:45.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4980ff460 0x7fe498100e70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:45.145 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4840a12e0 0x7fe4840a3790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:45.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 --2- 172.21.15.8:0/2703817263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4980ffe00 0x7fe4981013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:45.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 -- 172.21.15.8:0/2703817263 >> 172.21.15.8:0/2703817263 conn(0x7fe4980770a0 msgr2=0x7fe498068650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:45.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 -- 172.21.15.8:0/2703817263 shutdown_connections 2023-11-27T15:24:45.146 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:45.141+0000 7fe4a0288700 1 -- 172.21.15.8:0/2703817263 wait complete. 2023-11-27T15:24:45.273 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411400 for osd.2 2023-11-27T15:24:45.273 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:45 smithi008 ceph-mon[128788]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 86 B/s, 1 objects/s recovering 2023-11-27T15:24:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:45 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:45.770 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:45 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2703817263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-27T15:24:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:45 smithi060 ceph-mon[131518]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 86 B/s, 1 objects/s recovering 2023-11-27T15:24:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:45 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:45 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2703817263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-27T15:24:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:45 smithi029 ceph-mon[131166]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 86 B/s, 1 objects/s recovering 2023-11-27T15:24:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:45 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:45 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2703817263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-27T15:24:46.056 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808868 for osd.9 2023-11-27T15:24:46.056 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:46.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:24:46.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:24:46.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:46.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:24:46.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:24:46.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:46.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:24:46.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:24:46.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:24:47.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.572+0000 7fb048648700 1 -- 172.21.15.8:0/2603738761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0401055e0 msgr2=0x7fb04010d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:47.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.572+0000 7fb048648700 1 --2- 172.21.15.8:0/2603738761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0401055e0 0x7fb04010d160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb03800b0d0 tx=0x7fb03800b3e0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.572+0000 7fb048648700 1 -- 172.21.15.8:0/2603738761 shutdown_connections 2023-11-27T15:24:47.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.572+0000 7fb048648700 1 --2- 172.21.15.8:0/2603738761 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0401055e0 0x7fb04010d160 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.573+0000 7fb048648700 1 --2- 172.21.15.8:0/2603738761 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb040100540 0x7fb040100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.573+0000 7fb048648700 1 --2- 172.21.15.8:0/2603738761 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb040100ee0 0x7fb040104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.573+0000 7fb048648700 1 -- 172.21.15.8:0/2603738761 >> 172.21.15.8:0/2603738761 conn(0x7fb0400fbdc0 msgr2=0x7fb0400fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:47.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.573+0000 7fb048648700 1 -- 172.21.15.8:0/2603738761 shutdown_connections 2023-11-27T15:24:47.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.573+0000 7fb048648700 1 -- 172.21.15.8:0/2603738761 wait complete. 2023-11-27T15:24:47.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.574+0000 7fb048648700 1 Processor -- start 2023-11-27T15:24:47.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.574+0000 7fb048648700 1 -- start start 2023-11-27T15:24:47.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.574+0000 7fb048648700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb040100540 0x7fb040196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:47.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.574+0000 7fb048648700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb040100ee0 0x7fb040196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:47.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.574+0000 7fb048648700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0401055e0 0x7fb04019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:47.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb048648700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04019b500 con 0x7fb040100ee0 2023-11-27T15:24:47.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb048648700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04019b670 con 0x7fb040100540 2023-11-27T15:24:47.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb048648700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04019b830 con 0x7fb0401055e0 2023-11-27T15:24:47.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb045be3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb040100ee0 0x7fb040196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb045be3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb040100ee0 0x7fb040196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38548/0 (socket says 172.21.15.8:38548) 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb045be3700 1 -- 172.21.15.8:0/847569549 learned_addr learned my addr 172.21.15.8:0/847569549 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb0463e4700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb040100540 0x7fb040196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb046be5700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0401055e0 0x7fb04019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb045be3700 1 -- 172.21.15.8:0/847569549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0401055e0 msgr2=0x7fb04019aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb045be3700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0401055e0 0x7fb04019aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb045be3700 1 -- 172.21.15.8:0/847569549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb040100540 msgr2=0x7fb040196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb045be3700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb040100540 0x7fb040196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.575+0000 7fb045be3700 1 -- 172.21.15.8:0/847569549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb038009d00 con 0x7fb040100ee0 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.576+0000 7fb045be3700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb040100ee0 0x7fb040196a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb03000d950 tx=0x7fb03000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:47.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.576+0000 7fb0377fe700 1 -- 172.21.15.8:0/847569549 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030004220 con 0x7fb040100ee0 2023-11-27T15:24:47.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.576+0000 7fb0377fe700 1 -- 172.21.15.8:0/847569549 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb030007620 con 0x7fb040100ee0 2023-11-27T15:24:47.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.576+0000 7fb0377fe700 1 -- 172.21.15.8:0/847569549 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030005670 con 0x7fb040100ee0 2023-11-27T15:24:47.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.576+0000 7fb048648700 1 -- 172.21.15.8:0/847569549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb040110370 con 0x7fb040100ee0 2023-11-27T15:24:47.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.576+0000 7fb048648700 1 -- 172.21.15.8:0/847569549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0401101e0 con 0x7fb040100ee0 2023-11-27T15:24:47.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.577+0000 7fb048648700 1 -- 172.21.15.8:0/847569549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb040110500 con 0x7fb040100ee0 2023-11-27T15:24:47.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.579+0000 7fb0377fe700 1 -- 172.21.15.8:0/847569549 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb030003680 con 0x7fb040100ee0 2023-11-27T15:24:47.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.581+0000 7fb0377fe700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb02c0a1260 0x7fb02c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:47.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.582+0000 7fb0377fe700 1 -- 172.21.15.8:0/847569549 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb0300cca00 con 0x7fb040100ee0 2023-11-27T15:24:47.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.582+0000 7fb0463e4700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb02c0a1260 0x7fb02c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:47.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.583+0000 7fb0463e4700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb02c0a1260 0x7fb02c0a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb03c007be0 tx=0x7fb03c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:47.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.586+0000 7fb0377fe700 1 -- 172.21.15.8:0/847569549 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb030095c80 con 0x7fb040100ee0 2023-11-27T15:24:47.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:47 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:24:47.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:47 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:24:47.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:47 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:24:47.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:47 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:24:47.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:47 smithi008 ceph-mon[128788]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 86 B/s, 1 objects/s recovering 2023-11-27T15:24:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:47 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:24:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:47 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:24:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:47 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:24:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:47 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:24:47.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:47 smithi060 ceph-mon[131518]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 86 B/s, 1 objects/s recovering 2023-11-27T15:24:47.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.840+0000 7fb048648700 1 -- 172.21.15.8:0/847569549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fb040066ea0 con 0x7fb040100ee0 2023-11-27T15:24:47.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.840+0000 7fb0377fe700 1 -- 172.21.15.8:0/847569549 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fb0300992a0 con 0x7fb040100ee0 2023-11-27T15:24:47.844 INFO:teuthology.orchestra.run.smithi008.stdout:309237645346 2023-11-27T15:24:47.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 -- 172.21.15.8:0/847569549 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb02c0a1260 msgr2=0x7fb02c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:47.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb02c0a1260 0x7fb02c0a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb03c007be0 tx=0x7fb03c0073d0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 -- 172.21.15.8:0/847569549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb040100ee0 msgr2=0x7fb040196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:47.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb040100ee0 0x7fb040196a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb03000d950 tx=0x7fb03000dc60 comp rx=0 tx=0).stop 2023-11-27T15:24:47.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 -- 172.21.15.8:0/847569549 shutdown_connections 2023-11-27T15:24:47.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb02c0a1260 0x7fb02c0a3710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb0401055e0 0x7fb04019aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb040100540 0x7fb040196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 --2- 172.21.15.8:0/847569549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb040100ee0 0x7fb040196a80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 -- 172.21.15.8:0/847569549 >> 172.21.15.8:0/847569549 conn(0x7fb0400fbdc0 msgr2=0x7fb0400fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:47.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 -- 172.21.15.8:0/847569549 shutdown_connections 2023-11-27T15:24:47.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.843+0000 7fb048648700 1 -- 172.21.15.8:0/847569549 wait complete. 2023-11-27T15:24:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:47 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:24:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:47 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:24:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:47 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:24:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:47 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:24:47.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:47 smithi029 ceph-mon[131166]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 86 B/s, 1 objects/s recovering 2023-11-27T15:24:47.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.966+0000 7f448ac67700 1 -- 172.21.15.8:0/3589345580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4484101540 msgr2=0x7f4484101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:47.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.966+0000 7f448ac67700 1 --2- 172.21.15.8:0/3589345580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4484101540 0x7f4484101910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4478009990 tx=0x7f4478009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.966+0000 7f448ac67700 1 -- 172.21.15.8:0/3589345580 shutdown_connections 2023-11-27T15:24:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.966+0000 7f448ac67700 1 --2- 172.21.15.8:0/3589345580 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4484101540 0x7f4484101910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.966+0000 7f448ac67700 1 --2- 172.21.15.8:0/3589345580 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f448410a890 0x7f448410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.966+0000 7f448ac67700 1 --2- 172.21.15.8:0/3589345580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4484101e50 0x7f448410a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.966+0000 7f448ac67700 1 -- 172.21.15.8:0/3589345580 >> 172.21.15.8:0/3589345580 conn(0x7f44840770a0 msgr2=0x7f44840774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.966+0000 7f448ac67700 1 -- 172.21.15.8:0/3589345580 shutdown_connections 2023-11-27T15:24:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.967+0000 7f448ac67700 1 -- 172.21.15.8:0/3589345580 wait complete. 2023-11-27T15:24:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.968+0000 7f448ac67700 1 Processor -- start 2023-11-27T15:24:47.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.968+0000 7f448ac67700 1 -- start start 2023-11-27T15:24:47.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.968+0000 7f448ac67700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4484101540 0x7f4484194260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:47.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.968+0000 7f448ac67700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4484101e50 0x7f44841947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:47.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.968+0000 7f448ac67700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f448410a890 0x7f4484198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:47.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.968+0000 7f448ac67700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4484199220 con 0x7f4484101540 2023-11-27T15:24:47.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.968+0000 7f448ac67700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4484199390 con 0x7f4484101e50 2023-11-27T15:24:47.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.968+0000 7f448ac67700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4484199550 con 0x7f448410a890 2023-11-27T15:24:47.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4483fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4484101e50 0x7f44841947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:47.973 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4488a03700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4484101540 0x7f4484194260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4489204700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f448410a890 0x7f4484198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4488a03700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4484101540 0x7f4484194260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38560/0 (socket says 172.21.15.8:38560) 2023-11-27T15:24:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4488a03700 1 -- 172.21.15.8:0/4056951355 learned_addr learned my addr 172.21.15.8:0/4056951355 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4489204700 1 -- 172.21.15.8:0/4056951355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4484101e50 msgr2=0x7f44841947a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4489204700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4484101e50 0x7f44841947a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4489204700 1 -- 172.21.15.8:0/4056951355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4484101540 msgr2=0x7f4484194260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4489204700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4484101540 0x7f4484194260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4489204700 1 -- 172.21.15.8:0/4056951355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780095f0 con 0x7f448410a890 2023-11-27T15:24:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.969+0000 7f4488a03700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4484101540 0x7f4484194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.970+0000 7f4489204700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f448410a890 0x7f4484198c10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f447c00efc0 tx=0x7f447c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:47.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.970+0000 7f4481ffb700 1 -- 172.21.15.8:0/4056951355 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c018070 con 0x7f448410a890 2023-11-27T15:24:47.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.970+0000 7f448ac67700 1 -- 172.21.15.8:0/4056951355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448410e210 con 0x7f448410a890 2023-11-27T15:24:47.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.970+0000 7f4481ffb700 1 -- 172.21.15.8:0/4056951355 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f447c00f040 con 0x7f448410a890 2023-11-27T15:24:47.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.970+0000 7f4481ffb700 1 -- 172.21.15.8:0/4056951355 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00cd40 con 0x7f448410a890 2023-11-27T15:24:47.975 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.971+0000 7f448ac67700 1 -- 172.21.15.8:0/4056951355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448410e080 con 0x7f448410a890 2023-11-27T15:24:47.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.972+0000 7f448ac67700 1 -- 172.21.15.8:0/4056951355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448410e3a0 con 0x7f448410a890 2023-11-27T15:24:47.978 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.975+0000 7f4481ffb700 1 -- 172.21.15.8:0/4056951355 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f447c003a00 con 0x7f448410a890 2023-11-27T15:24:47.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.976+0000 7f4481ffb700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f44740a1260 0x7f44740a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:47.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.976+0000 7f4481ffb700 1 -- 172.21.15.8:0/4056951355 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f447c0cc4c0 con 0x7f448410a890 2023-11-27T15:24:47.980 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.977+0000 7f4488a03700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f44740a1260 0x7f44740a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:47.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.977+0000 7f4488a03700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f44740a1260 0x7f44740a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4478006010 tx=0x7f447800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:47.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:47.981+0000 7f4481ffb700 1 -- 172.21.15.8:0/4056951355 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f447c0957d0 con 0x7f448410a890 2023-11-27T15:24:48.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.082+0000 7fd452857700 1 -- 172.21.15.8:0/1083295226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd44c10a890 msgr2=0x7fd44c10ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:48.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.082+0000 7fd452857700 1 --2- 172.21.15.8:0/1083295226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd44c10a890 0x7fd44c10ccd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd444009f90 tx=0x7fd44400b330 comp rx=0 tx=0).stop 2023-11-27T15:24:48.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.082+0000 7fd452857700 1 -- 172.21.15.8:0/1083295226 shutdown_connections 2023-11-27T15:24:48.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.082+0000 7fd452857700 1 --2- 172.21.15.8:0/1083295226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c101540 0x7fd44c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.082+0000 7fd452857700 1 --2- 172.21.15.8:0/1083295226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd44c10a890 0x7fd44c10ccd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.082+0000 7fd452857700 1 --2- 172.21.15.8:0/1083295226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd44c101e50 0x7fd44c10a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.082+0000 7fd452857700 1 -- 172.21.15.8:0/1083295226 >> 172.21.15.8:0/1083295226 conn(0x7fd44c0770a0 msgr2=0x7fd44c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:48.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.082+0000 7fd452857700 1 -- 172.21.15.8:0/1083295226 shutdown_connections 2023-11-27T15:24:48.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.082+0000 7fd452857700 1 -- 172.21.15.8:0/1083295226 wait complete. 2023-11-27T15:24:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.083+0000 7fd452857700 1 Processor -- start 2023-11-27T15:24:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd452857700 1 -- start start 2023-11-27T15:24:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd452857700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd44c101540 0x7fd44c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd452857700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd44c101e50 0x7fd44c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd452857700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c10a890 0x7fd44c19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd452857700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c19c600 con 0x7fd44c101e50 2023-11-27T15:24:48.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd452857700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c19c770 con 0x7fd44c101540 2023-11-27T15:24:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd452857700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c19c930 con 0x7fd44c10a890 2023-11-27T15:24:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd44bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd44c101540 0x7fd44c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd44bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd44c101540 0x7fd44c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:58916/0 (socket says 172.21.15.8:58916) 2023-11-27T15:24:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd44bfff700 1 -- 172.21.15.8:0/3689415812 learned_addr learned my addr 172.21.15.8:0/3689415812 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:48.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.084+0000 7fd450df4700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c10a890 0x7fd44c19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:48.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.085+0000 7fd44b7fe700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd44c101e50 0x7fd44c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:48.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.085+0000 7fd44bfff700 1 -- 172.21.15.8:0/3689415812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c10a890 msgr2=0x7fd44c19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:48.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.085+0000 7fd44bfff700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c10a890 0x7fd44c19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.085+0000 7fd44bfff700 1 -- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd44c101e50 msgr2=0x7fd44c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:48.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.085+0000 7fd44bfff700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd44c101e50 0x7fd44c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.085+0000 7fd44bfff700 1 -- 172.21.15.8:0/3689415812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4400095f0 con 0x7fd44c101540 2023-11-27T15:24:48.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.085+0000 7fd44b7fe700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd44c101e50 0x7fd44c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:48.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.085+0000 7fd44bfff700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd44c101540 0x7fd44c198700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00e9a0 tx=0x7fd43c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:48.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.085+0000 7fd4497fa700 1 -- 172.21.15.8:0/3689415812 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c009790 con 0x7fd44c101540 2023-11-27T15:24:48.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.086+0000 7fd452857700 1 -- 172.21.15.8:0/3689415812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd444009c40 con 0x7fd44c101540 2023-11-27T15:24:48.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.086+0000 7fd4497fa700 1 -- 172.21.15.8:0/3689415812 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd43c00cbd0 con 0x7fd44c101540 2023-11-27T15:24:48.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.086+0000 7fd4497fa700 1 -- 172.21.15.8:0/3689415812 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c018680 con 0x7fd44c101540 2023-11-27T15:24:48.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.086+0000 7fd452857700 1 -- 172.21.15.8:0/3689415812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c110350 con 0x7fd44c101540 2023-11-27T15:24:48.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.087+0000 7fd452857700 1 -- 172.21.15.8:0/3689415812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c1101c0 con 0x7fd44c101540 2023-11-27T15:24:48.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.091+0000 7fd4497fa700 1 -- 172.21.15.8:0/3689415812 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd43c00cd40 con 0x7fd44c101540 2023-11-27T15:24:48.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.092+0000 7fd4497fa700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd4340a11e0 0x7fd4340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.092+0000 7fd4497fa700 1 -- 172.21.15.8:0/3689415812 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd43c014070 con 0x7fd44c101540 2023-11-27T15:24:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.092+0000 7fd44b7fe700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd4340a11e0 0x7fd4340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:48.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.093+0000 7fd4497fa700 1 -- 172.21.15.8:0/3689415812 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd43c0965c0 con 0x7fd44c101540 2023-11-27T15:24:48.097 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.093+0000 7fd44b7fe700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd4340a11e0 0x7fd4340a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd44c199d20 tx=0x7fd440009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.197+0000 7f448ac67700 1 -- 172.21.15.8:0/4056951355 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4484066ea0 con 0x7f448410a890 2023-11-27T15:24:48.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.197+0000 7f4481ffb700 1 -- 172.21.15.8:0/4056951355 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f447c098df0 con 0x7f448410a890 2023-11-27T15:24:48.201 INFO:teuthology.orchestra.run.smithi008.stdout:167503724601 2023-11-27T15:24:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 -- 172.21.15.8:0/4056951355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f44740a1260 msgr2=0x7f44740a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f44740a1260 0x7f44740a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4478006010 tx=0x7f447800b260 comp rx=0 tx=0).stop 2023-11-27T15:24:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 -- 172.21.15.8:0/4056951355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f448410a890 msgr2=0x7f4484198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f448410a890 0x7f4484198c10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f447c00efc0 tx=0x7f447c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 -- 172.21.15.8:0/4056951355 shutdown_connections 2023-11-27T15:24:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f44740a1260 0x7f44740a3710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f448410a890 0x7f4484198c10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4484101540 0x7f4484194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 --2- 172.21.15.8:0/4056951355 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4484101e50 0x7f44841947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 -- 172.21.15.8:0/4056951355 >> 172.21.15.8:0/4056951355 conn(0x7f44840770a0 msgr2=0x7f44840fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 -- 172.21.15.8:0/4056951355 shutdown_connections 2023-11-27T15:24:48.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.200+0000 7f448ac67700 1 -- 172.21.15.8:0/4056951355 wait complete. 2023-11-27T15:24:48.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.324+0000 7fd452857700 1 -- 172.21.15.8:0/3689415812 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fd44c1104e0 con 0x7fd44c101540 2023-11-27T15:24:48.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.325+0000 7fd4497fa700 1 -- 172.21.15.8:0/3689415812 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd43c099be0 con 0x7fd44c101540 2023-11-27T15:24:48.328 INFO:teuthology.orchestra.run.smithi008.stdout:253403070507 2023-11-27T15:24:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 -- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd4340a11e0 msgr2=0x7fd4340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd4340a11e0 0x7fd4340a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd44c199d20 tx=0x7fd440009310 comp rx=0 tx=0).stop 2023-11-27T15:24:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 -- 172.21.15.8:0/3689415812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd44c101540 msgr2=0x7fd44c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:48.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd44c101540 0x7fd44c198700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00e9a0 tx=0x7fd43c00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 -- 172.21.15.8:0/3689415812 shutdown_connections 2023-11-27T15:24:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd4340a11e0 0x7fd4340a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd44c10a890 0x7fd44c19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd44c101540 0x7fd44c198700 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 --2- 172.21.15.8:0/3689415812 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd44c101e50 0x7fd44c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 -- 172.21.15.8:0/3689415812 >> 172.21.15.8:0/3689415812 conn(0x7fd44c0770a0 msgr2=0x7fd44c0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 -- 172.21.15.8:0/3689415812 shutdown_connections 2023-11-27T15:24:48.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:48.327+0000 7fd452857700 1 -- 172.21.15.8:0/3689415812 wait complete. 2023-11-27T15:24:48.505 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645343 got 309237645346 for osd.10 2023-11-27T15:24:48.505 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:48.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:48 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/847569549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-27T15:24:48.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:48 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4056951355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-27T15:24:48.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:48 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3689415812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-27T15:24:48.794 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724598 got 167503724601 for osd.5 2023-11-27T15:24:48.797 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:48.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:48 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/847569549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-27T15:24:48.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:48 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/4056951355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-27T15:24:48.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:48 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3689415812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-27T15:24:48.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:48 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/847569549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-27T15:24:48.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:48 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/4056951355' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-27T15:24:48.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:48 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3689415812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-27T15:24:49.197 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070504 got 253403070507 for osd.8 2023-11-27T15:24:49.198 DEBUG:teuthology.parallel:result is None 2023-11-27T15:24:49.198 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-11-27T15:24:49.198 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-11-27T15:24:49.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:49 smithi060 ceph-mon[131518]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 76 B/s, 1 objects/s recovering 2023-11-27T15:24:49.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:49 smithi008 ceph-mon[128788]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 76 B/s, 1 objects/s recovering 2023-11-27T15:24:49.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:49 smithi029 ceph-mon[131166]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 76 B/s, 1 objects/s recovering 2023-11-27T15:24:50.062 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:51.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.606+0000 7f1a890dd700 1 -- 172.21.15.8:0/1932299963 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a84100d10 msgr2=0x7f1a84104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.606+0000 7f1a890dd700 1 --2- 172.21.15.8:0/1932299963 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a84100d10 0x7f1a84104ce0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1a780099c0 tx=0x7f1a78009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.606+0000 7f1a890dd700 1 -- 172.21.15.8:0/1932299963 shutdown_connections 2023-11-27T15:24:51.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.606+0000 7f1a890dd700 1 --2- 172.21.15.8:0/1932299963 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a84105410 0x7f1a8410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.606+0000 7f1a890dd700 1 --2- 172.21.15.8:0/1932299963 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a84100370 0x7f1a84100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.606+0000 7f1a890dd700 1 --2- 172.21.15.8:0/1932299963 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a84100d10 0x7f1a84104ce0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.606+0000 7f1a890dd700 1 -- 172.21.15.8:0/1932299963 >> 172.21.15.8:0/1932299963 conn(0x7f1a840fbbd0 msgr2=0x7f1a840fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.606+0000 7f1a890dd700 1 -- 172.21.15.8:0/1932299963 shutdown_connections 2023-11-27T15:24:51.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.606+0000 7f1a890dd700 1 -- 172.21.15.8:0/1932299963 wait complete. 2023-11-27T15:24:51.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.607+0000 7f1a890dd700 1 Processor -- start 2023-11-27T15:24:51.611 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.607+0000 7f1a890dd700 1 -- start start 2023-11-27T15:24:51.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.607+0000 7f1a890dd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a84100370 0x7f1a84196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.607+0000 7f1a890dd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a84100d10 0x7f1a84196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.607+0000 7f1a890dd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a84105410 0x7f1a8419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.607+0000 7f1a890dd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8419b210 con 0x7f1a84100370 2023-11-27T15:24:51.613 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.607+0000 7f1a890dd700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8419b380 con 0x7f1a84100d10 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.607+0000 7f1a890dd700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8419b540 con 0x7f1a84105410 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a83fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a84100370 0x7f1a84196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a83fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a84100370 0x7f1a84196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45236/0 (socket says 172.21.15.8:45236) 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a83fff700 1 -- 172.21.15.8:0/1461159650 learned_addr learned my addr 172.21.15.8:0/1461159650 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a837fe700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a84100d10 0x7f1a84196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a888dc700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a84105410 0x7f1a8419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a83fff700 1 -- 172.21.15.8:0/1461159650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a84105410 msgr2=0x7f1a8419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a83fff700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a84105410 0x7f1a8419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a83fff700 1 -- 172.21.15.8:0/1461159650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a84100d10 msgr2=0x7f1a84196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a83fff700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a84100d10 0x7f1a84196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a83fff700 1 -- 172.21.15.8:0/1461159650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a780095f0 con 0x7f1a84100370 2023-11-27T15:24:51.614 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a888dc700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a84105410 0x7f1a8419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:51.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.608+0000 7f1a83fff700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a84100370 0x7f1a84196250 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400e9a0 tx=0x7f1a7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.609+0000 7f1a817fa700 1 -- 172.21.15.8:0/1461159650 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74009790 con 0x7f1a84100370 2023-11-27T15:24:51.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.609+0000 7f1a817fa700 1 -- 172.21.15.8:0/1461159650 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a7400cbd0 con 0x7f1a84100370 2023-11-27T15:24:51.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.609+0000 7f1a890dd700 1 -- 172.21.15.8:0/1461159650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a84110190 con 0x7f1a84100370 2023-11-27T15:24:51.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.609+0000 7f1a817fa700 1 -- 172.21.15.8:0/1461159650 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a74018680 con 0x7f1a84100370 2023-11-27T15:24:51.615 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.609+0000 7f1a890dd700 1 -- 172.21.15.8:0/1461159650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a84110000 con 0x7f1a84100370 2023-11-27T15:24:51.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:51 smithi008 ceph-mon[128788]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 68 B/s, 1 objects/s recovering 2023-11-27T15:24:51.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.610+0000 7f1a890dd700 1 -- 172.21.15.8:0/1461159650 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a84110320 con 0x7f1a84100370 2023-11-27T15:24:51.616 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.612+0000 7f1a817fa700 1 -- 172.21.15.8:0/1461159650 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1a7400cd40 con 0x7f1a84100370 2023-11-27T15:24:51.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.613+0000 7f1a817fa700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1a6c0a1260 0x7f1a6c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:51.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.614+0000 7f1a817fa700 1 -- 172.21.15.8:0/1461159650 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1a74014070 con 0x7f1a84100370 2023-11-27T15:24:51.617 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.614+0000 7f1a837fe700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1a6c0a1260 0x7f1a6c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:51.618 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.614+0000 7f1a837fe700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1a6c0a1260 0x7f1a6c0a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009fd0 tx=0x7f1a78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:51.621 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.617+0000 7f1a817fa700 1 -- 172.21.15.8:0/1461159650 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1a74051730 con 0x7f1a84100370 2023-11-27T15:24:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:51 smithi060 ceph-mon[131518]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 68 B/s, 1 objects/s recovering 2023-11-27T15:24:51.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.823+0000 7f1a890dd700 1 -- 172.21.15.8:0/1461159650 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a840fcdd0 con 0x7f1a6c0a1260 2023-11-27T15:24:51.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.838+0000 7f1a817fa700 1 -- 172.21.15.8:0/1461159650 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426489 (secure 0 0 0) 0x7f1a840fcdd0 con 0x7f1a6c0a1260 2023-11-27T15:24:51.849 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:51.852 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.843+0000 7f1a890dd700 1 -- 172.21.15.8:0/1461159650 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1a6c0a1260 msgr2=0x7f1a6c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.843+0000 7f1a890dd700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1a6c0a1260 0x7f1a6c0a3710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009fd0 tx=0x7f1a78005fb0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.843+0000 7f1a890dd700 1 -- 172.21.15.8:0/1461159650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a84100370 msgr2=0x7f1a84196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.843+0000 7f1a890dd700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a84100370 0x7f1a84196250 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400e9a0 tx=0x7f1a7400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.843+0000 7f1a890dd700 1 -- 172.21.15.8:0/1461159650 shutdown_connections 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.843+0000 7f1a890dd700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1a6c0a1260 0x7f1a6c0a3710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.843+0000 7f1a890dd700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1a84105410 0x7f1a8419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.843+0000 7f1a890dd700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1a84100370 0x7f1a84196250 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.843+0000 7f1a890dd700 1 --2- 172.21.15.8:0/1461159650 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1a84100d10 0x7f1a84196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.843+0000 7f1a890dd700 1 -- 172.21.15.8:0/1461159650 >> 172.21.15.8:0/1461159650 conn(0x7f1a840fbbd0 msgr2=0x7f1a840fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.844+0000 7f1a890dd700 1 -- 172.21.15.8:0/1461159650 shutdown_connections 2023-11-27T15:24:51.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:51.844+0000 7f1a890dd700 1 -- 172.21.15.8:0/1461159650 wait complete. 2023-11-27T15:24:51.854 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-11-27T15:24:51.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:51 smithi029 ceph-mon[131166]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 68 B/s, 1 objects/s recovering 2023-11-27T15:24:52.368 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-11-27T15:24:50.348284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656330,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":453,"num_read_kb":345,"num_write":338,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":83168,"kb_used_data":13824,"kb_used_omap":0,"kb_used_meta":69248,"kb_avail":1124858656,"statfs":{"total":1151940427776,"available":1151855263744,"internally_reserved":0,"allocated":14155776,"data_stored":7833715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70909952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":28,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.094243"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261233+0000","last_change":"2023-11-27T15:23:27.211699+0000","last_active":"2023-11-27T15:24:40.261233+0000","last_peered":"2023-11-27T15:24:40.261233+0000","last_clean":"2023-11-27T15:24:40.261233+0000","last_became_active":"2023-11-27T15:23:27.211372+0000","last_became_peered":"2023-11-27T15:23:27.211372+0000","last_unstale":"2023-11-27T15:24:40.261233+0000","last_undegraded":"2023-11-27T15:24:40.261233+0000","last_fullsized":"2023-11-27T15:24:40.261233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:04:28.645927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.262233+0000","last_change":"2023-11-27T15:23:27.210567+0000","last_active":"2023-11-27T15:24:40.262233+0000","last_peered":"2023-11-27T15:24:40.262233+0000","last_clean":"2023-11-27T15:24:40.262233+0000","last_became_active":"2023-11-27T15:23:27.208340+0000","last_became_peered":"2023-11-27T15:23:27.208340+0000","last_unstale":"2023-11-27T15:24:40.262233+0000","last_undegraded":"2023-11-27T15:24:40.262233+0000","last_fullsized":"2023-11-27T15:24:40.262233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:55:01.081997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257911+0000","last_change":"2023-11-27T15:23:27.196040+0000","last_active":"2023-11-27T15:24:40.257911+0000","last_peered":"2023-11-27T15:24:40.257911+0000","last_clean":"2023-11-27T15:24:40.257911+0000","last_became_active":"2023-11-27T15:23:27.195195+0000","last_became_peered":"2023-11-27T15:23:27.195195+0000","last_unstale":"2023-11-27T15:24:40.257911+0000","last_undegraded":"2023-11-27T15:24:40.257911+0000","last_fullsized":"2023-11-27T15:24:40.257911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:07:01.884552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320800+0000","last_change":"2023-11-27T15:23:27.190096+0000","last_active":"2023-11-27T15:24:40.320800+0000","last_peered":"2023-11-27T15:24:40.320800+0000","last_clean":"2023-11-27T15:24:40.320800+0000","last_became_active":"2023-11-27T15:23:27.189768+0000","last_became_peered":"2023-11-27T15:23:27.189768+0000","last_unstale":"2023-11-27T15:24:40.320800+0000","last_undegraded":"2023-11-27T15:24:40.320800+0000","last_fullsized":"2023-11-27T15:24:40.320800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:00:55.203158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259735+0000","last_change":"2023-11-27T15:23:27.200974+0000","last_active":"2023-11-27T15:24:40.259735+0000","last_peered":"2023-11-27T15:24:40.259735+0000","last_clean":"2023-11-27T15:24:40.259735+0000","last_became_active":"2023-11-27T15:23:27.200647+0000","last_became_peered":"2023-11-27T15:23:27.200647+0000","last_unstale":"2023-11-27T15:24:40.259735+0000","last_undegraded":"2023-11-27T15:24:40.259735+0000","last_fullsized":"2023-11-27T15:24:40.259735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:05:54.959172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.315298+0000","last_change":"2023-11-27T15:23:27.188417+0000","last_active":"2023-11-27T15:24:40.315298+0000","last_peered":"2023-11-27T15:24:40.315298+0000","last_clean":"2023-11-27T15:24:40.315298+0000","last_became_active":"2023-11-27T15:23:27.188079+0000","last_became_peered":"2023-11-27T15:23:27.188079+0000","last_unstale":"2023-11-27T15:24:40.315298+0000","last_undegraded":"2023-11-27T15:24:40.315298+0000","last_fullsized":"2023-11-27T15:24:40.315298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:22:04.529440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.390841+0000","last_change":"2023-11-27T15:23:27.192410+0000","last_active":"2023-11-27T15:24:40.390841+0000","last_peered":"2023-11-27T15:24:40.390841+0000","last_clean":"2023-11-27T15:24:40.390841+0000","last_became_active":"2023-11-27T15:23:27.192026+0000","last_became_peered":"2023-11-27T15:23:27.192026+0000","last_unstale":"2023-11-27T15:24:40.390841+0000","last_undegraded":"2023-11-27T15:24:40.390841+0000","last_fullsized":"2023-11-27T15:24:40.390841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:58:54.291338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268554+0000","last_change":"2023-11-27T15:23:27.205778+0000","last_active":"2023-11-27T15:24:40.268554+0000","last_peered":"2023-11-27T15:24:40.268554+0000","last_clean":"2023-11-27T15:24:40.268554+0000","last_became_active":"2023-11-27T15:23:27.205467+0000","last_became_peered":"2023-11-27T15:23:27.205467+0000","last_unstale":"2023-11-27T15:24:40.268554+0000","last_undegraded":"2023-11-27T15:24:40.268554+0000","last_fullsized":"2023-11-27T15:24:40.268554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:58:16.116452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257004+0000","last_change":"2023-11-27T15:23:27.187920+0000","last_active":"2023-11-27T15:24:40.257004+0000","last_peered":"2023-11-27T15:24:40.257004+0000","last_clean":"2023-11-27T15:24:40.257004+0000","last_became_active":"2023-11-27T15:23:27.186989+0000","last_became_peered":"2023-11-27T15:23:27.186989+0000","last_unstale":"2023-11-27T15:24:40.257004+0000","last_undegraded":"2023-11-27T15:24:40.257004+0000","last_fullsized":"2023-11-27T15:24:40.257004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:25:30.816437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.315866+0000","last_change":"2023-11-27T15:23:27.181330+0000","last_active":"2023-11-27T15:24:40.315866+0000","last_peered":"2023-11-27T15:24:40.315866+0000","last_clean":"2023-11-27T15:24:40.315866+0000","last_became_active":"2023-11-27T15:23:27.180986+0000","last_became_peered":"2023-11-27T15:23:27.180986+0000","last_unstale":"2023-11-27T15:24:40.315866+0000","last_undegraded":"2023-11-27T15:24:40.315866+0000","last_fullsized":"2023-11-27T15:24:40.315866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:00:04.601663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391622+0000","last_change":"2023-11-27T15:23:27.190673+0000","last_active":"2023-11-27T15:24:40.391622+0000","last_peered":"2023-11-27T15:24:40.391622+0000","last_clean":"2023-11-27T15:24:40.391622+0000","last_became_active":"2023-11-27T15:23:27.186256+0000","last_became_peered":"2023-11-27T15:23:27.186256+0000","last_unstale":"2023-11-27T15:24:40.391622+0000","last_undegraded":"2023-11-27T15:24:40.391622+0000","last_fullsized":"2023-11-27T15:24:40.391622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:42:50.554823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.317407+0000","last_change":"2023-11-27T15:24:40.316671+0000","last_active":"2023-11-27T15:24:40.317407+0000","last_peered":"2023-11-27T15:24:40.317407+0000","last_clean":"2023-11-27T15:24:40.317407+0000","last_became_active":"2023-11-27T15:24:40.316245+0000","last_became_peered":"2023-11-27T15:24:40.316245+0000","last_unstale":"2023-11-27T15:24:40.317407+0000","last_undegraded":"2023-11-27T15:24:40.317407+0000","last_fullsized":"2023-11-27T15:24:40.317407+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:21:28.469719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260086+0000","last_change":"2023-11-27T15:23:27.190146+0000","last_active":"2023-11-27T15:24:40.260086+0000","last_peered":"2023-11-27T15:24:40.260086+0000","last_clean":"2023-11-27T15:24:40.260086+0000","last_became_active":"2023-11-27T15:23:27.189133+0000","last_became_peered":"2023-11-27T15:23:27.189133+0000","last_unstale":"2023-11-27T15:24:40.260086+0000","last_undegraded":"2023-11-27T15:24:40.260086+0000","last_fullsized":"2023-11-27T15:24:40.260086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:11:57.964507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.258139+0000","last_change":"2023-11-27T15:23:27.207461+0000","last_active":"2023-11-27T15:24:40.258139+0000","last_peered":"2023-11-27T15:24:40.258139+0000","last_clean":"2023-11-27T15:24:40.258139+0000","last_became_active":"2023-11-27T15:23:27.206837+0000","last_became_peered":"2023-11-27T15:23:27.206837+0000","last_unstale":"2023-11-27T15:24:40.258139+0000","last_undegraded":"2023-11-27T15:24:40.258139+0000","last_fullsized":"2023-11-27T15:24:40.258139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:36:53.427531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257998+0000","last_change":"2023-11-27T15:23:27.211206+0000","last_active":"2023-11-27T15:24:40.257998+0000","last_peered":"2023-11-27T15:24:40.257998+0000","last_clean":"2023-11-27T15:24:40.257998+0000","last_became_active":"2023-11-27T15:23:27.209933+0000","last_became_peered":"2023-11-27T15:23:27.209933+0000","last_unstale":"2023-11-27T15:24:40.257998+0000","last_undegraded":"2023-11-27T15:24:40.257998+0000","last_fullsized":"2023-11-27T15:24:40.257998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:01:16.466116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256415+0000","last_change":"2023-11-27T15:23:27.188863+0000","last_active":"2023-11-27T15:24:40.256415+0000","last_peered":"2023-11-27T15:24:40.256415+0000","last_clean":"2023-11-27T15:24:40.256415+0000","last_became_active":"2023-11-27T15:23:27.188534+0000","last_became_peered":"2023-11-27T15:23:27.188534+0000","last_unstale":"2023-11-27T15:24:40.256415+0000","last_undegraded":"2023-11-27T15:24:40.256415+0000","last_fullsized":"2023-11-27T15:24:40.256415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:29:05.866285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261126+0000","last_change":"2023-11-27T15:23:27.211786+0000","last_active":"2023-11-27T15:24:40.261126+0000","last_peered":"2023-11-27T15:24:40.261126+0000","last_clean":"2023-11-27T15:24:40.261126+0000","last_became_active":"2023-11-27T15:23:27.211475+0000","last_became_peered":"2023-11-27T15:23:27.211475+0000","last_unstale":"2023-11-27T15:24:40.261126+0000","last_undegraded":"2023-11-27T15:24:40.261126+0000","last_fullsized":"2023-11-27T15:24:40.261126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:58:15.676400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.263816+0000","last_change":"2023-11-27T15:23:27.183246+0000","last_active":"2023-11-27T15:24:40.263816+0000","last_peered":"2023-11-27T15:24:40.263816+0000","last_clean":"2023-11-27T15:24:40.263816+0000","last_became_active":"2023-11-27T15:23:27.182454+0000","last_became_peered":"2023-11-27T15:23:27.182454+0000","last_unstale":"2023-11-27T15:24:40.263816+0000","last_undegraded":"2023-11-27T15:24:40.263816+0000","last_fullsized":"2023-11-27T15:24:40.263816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:12:50.045523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260441+0000","last_change":"2023-11-27T15:23:27.196461+0000","last_active":"2023-11-27T15:24:40.260441+0000","last_peered":"2023-11-27T15:24:40.260441+0000","last_clean":"2023-11-27T15:24:40.260441+0000","last_became_active":"2023-11-27T15:23:27.196122+0000","last_became_peered":"2023-11-27T15:23:27.196122+0000","last_unstale":"2023-11-27T15:24:40.260441+0000","last_undegraded":"2023-11-27T15:24:40.260441+0000","last_fullsized":"2023-11-27T15:24:40.260441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:31:30.005963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259166+0000","last_change":"2023-11-27T15:23:31.649582+0000","last_active":"2023-11-27T15:24:40.259166+0000","last_peered":"2023-11-27T15:24:40.259166+0000","last_clean":"2023-11-27T15:24:40.259166+0000","last_became_active":"2023-11-27T15:23:31.649172+0000","last_became_peered":"2023-11-27T15:23:31.649172+0000","last_unstale":"2023-11-27T15:24:40.259166+0000","last_undegraded":"2023-11-27T15:24:40.259166+0000","last_fullsized":"2023-11-27T15:24:40.259166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:18:30.987836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.264701+0000","last_change":"2023-11-27T15:23:26.609457+0000","last_active":"2023-11-27T15:24:40.264701+0000","last_peered":"2023-11-27T15:24:40.264701+0000","last_clean":"2023-11-27T15:24:40.264701+0000","last_became_active":"2023-11-27T15:23:26.609144+0000","last_became_peered":"2023-11-27T15:23:26.609144+0000","last_unstale":"2023-11-27T15:24:40.264701+0000","last_undegraded":"2023-11-27T15:24:40.264701+0000","last_fullsized":"2023-11-27T15:24:40.264701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:20:10.079237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.270347+0000","last_change":"2023-11-27T15:23:27.204423+0000","last_active":"2023-11-27T15:24:40.270347+0000","last_peered":"2023-11-27T15:24:40.270347+0000","last_clean":"2023-11-27T15:24:40.270347+0000","last_became_active":"2023-11-27T15:23:27.204104+0000","last_became_peered":"2023-11-27T15:23:27.204104+0000","last_unstale":"2023-11-27T15:24:40.270347+0000","last_undegraded":"2023-11-27T15:24:40.270347+0000","last_fullsized":"2023-11-27T15:24:40.270347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:36:02.362505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.319676+0000","last_change":"2023-11-27T15:23:31.655641+0000","last_active":"2023-11-27T15:24:40.319676+0000","last_peered":"2023-11-27T15:24:40.319676+0000","last_clean":"2023-11-27T15:24:40.319676+0000","last_became_active":"2023-11-27T15:23:31.654991+0000","last_became_peered":"2023-11-27T15:23:31.654991+0000","last_unstale":"2023-11-27T15:24:40.319676+0000","last_undegraded":"2023-11-27T15:24:40.319676+0000","last_fullsized":"2023-11-27T15:24:40.319676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:07:47.588692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.270644+0000","last_change":"2023-11-27T15:23:26.620780+0000","last_active":"2023-11-27T15:24:40.270644+0000","last_peered":"2023-11-27T15:24:40.270644+0000","last_clean":"2023-11-27T15:24:40.270644+0000","last_became_active":"2023-11-27T15:23:26.620438+0000","last_became_peered":"2023-11-27T15:23:26.620438+0000","last_unstale":"2023-11-27T15:24:40.270644+0000","last_undegraded":"2023-11-27T15:24:40.270644+0000","last_fullsized":"2023-11-27T15:24:40.270644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:00:45.116977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.258580+0000","last_change":"2023-11-27T15:23:27.195581+0000","last_active":"2023-11-27T15:24:40.258580+0000","last_peered":"2023-11-27T15:24:40.258580+0000","last_clean":"2023-11-27T15:24:40.258580+0000","last_became_active":"2023-11-27T15:23:27.195260+0000","last_became_peered":"2023-11-27T15:23:27.195260+0000","last_unstale":"2023-11-27T15:24:40.258580+0000","last_undegraded":"2023-11-27T15:24:40.258580+0000","last_fullsized":"2023-11-27T15:24:40.258580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:05:39.695701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320045+0000","last_change":"2023-11-27T15:23:31.655464+0000","last_active":"2023-11-27T15:24:40.320045+0000","last_peered":"2023-11-27T15:24:40.320045+0000","last_clean":"2023-11-27T15:24:40.320045+0000","last_became_active":"2023-11-27T15:23:31.654952+0000","last_became_peered":"2023-11-27T15:23:31.654952+0000","last_unstale":"2023-11-27T15:24:40.320045+0000","last_undegraded":"2023-11-27T15:24:40.320045+0000","last_fullsized":"2023-11-27T15:24:40.320045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:09:29.902445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261205+0000","last_change":"2023-11-27T15:23:26.606593+0000","last_active":"2023-11-27T15:24:40.261205+0000","last_peered":"2023-11-27T15:24:40.261205+0000","last_clean":"2023-11-27T15:24:40.261205+0000","last_became_active":"2023-11-27T15:23:26.605719+0000","last_became_peered":"2023-11-27T15:23:26.605719+0000","last_unstale":"2023-11-27T15:24:40.261205+0000","last_undegraded":"2023-11-27T15:24:40.261205+0000","last_fullsized":"2023-11-27T15:24:40.261205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:10.147295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257102+0000","last_change":"2023-11-27T15:23:27.192629+0000","last_active":"2023-11-27T15:24:40.257102+0000","last_peered":"2023-11-27T15:24:40.257102+0000","last_clean":"2023-11-27T15:24:40.257102+0000","last_became_active":"2023-11-27T15:23:27.192327+0000","last_became_peered":"2023-11-27T15:23:27.192327+0000","last_unstale":"2023-11-27T15:24:40.257102+0000","last_undegraded":"2023-11-27T15:24:40.257102+0000","last_fullsized":"2023-11-27T15:24:40.257102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:09:08.862576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.319918+0000","last_change":"2023-11-27T15:23:31.648296+0000","last_active":"2023-11-27T15:24:40.319918+0000","last_peered":"2023-11-27T15:24:40.319918+0000","last_clean":"2023-11-27T15:24:40.319918+0000","last_became_active":"2023-11-27T15:23:31.647756+0000","last_became_peered":"2023-11-27T15:23:31.647756+0000","last_unstale":"2023-11-27T15:24:40.319918+0000","last_undegraded":"2023-11-27T15:24:40.319918+0000","last_fullsized":"2023-11-27T15:24:40.319918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:44:42.109495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261456+0000","last_change":"2023-11-27T15:23:26.604061+0000","last_active":"2023-11-27T15:24:40.261456+0000","last_peered":"2023-11-27T15:24:40.261456+0000","last_clean":"2023-11-27T15:24:40.261456+0000","last_became_active":"2023-11-27T15:23:26.603725+0000","last_became_peered":"2023-11-27T15:23:26.603725+0000","last_unstale":"2023-11-27T15:24:40.261456+0000","last_undegraded":"2023-11-27T15:24:40.261456+0000","last_fullsized":"2023-11-27T15:24:40.261456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:04:54.440045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256896+0000","last_change":"2023-11-27T15:23:27.189389+0000","last_active":"2023-11-27T15:24:40.256896+0000","last_peered":"2023-11-27T15:24:40.256896+0000","last_clean":"2023-11-27T15:24:40.256896+0000","last_became_active":"2023-11-27T15:23:27.189072+0000","last_became_peered":"2023-11-27T15:23:27.189072+0000","last_unstale":"2023-11-27T15:24:40.256896+0000","last_undegraded":"2023-11-27T15:24:40.256896+0000","last_fullsized":"2023-11-27T15:24:40.256896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:48:20.053941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260214+0000","last_change":"2023-11-27T15:23:31.639042+0000","last_active":"2023-11-27T15:24:40.260214+0000","last_peered":"2023-11-27T15:24:40.260214+0000","last_clean":"2023-11-27T15:24:40.260214+0000","last_became_active":"2023-11-27T15:23:31.638611+0000","last_became_peered":"2023-11-27T15:23:31.638611+0000","last_unstale":"2023-11-27T15:24:40.260214+0000","last_undegraded":"2023-11-27T15:24:40.260214+0000","last_fullsized":"2023-11-27T15:24:40.260214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:20:34.399106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":83,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:44.323459+0000","last_change":"2023-11-27T15:23:26.602979+0000","last_active":"2023-11-27T15:24:44.323459+0000","last_peered":"2023-11-27T15:24:44.323459+0000","last_clean":"2023-11-27T15:24:44.323459+0000","last_became_active":"2023-11-27T15:23:26.602681+0000","last_became_peered":"2023-11-27T15:23:26.602681+0000","last_unstale":"2023-11-27T15:24:44.323459+0000","last_undegraded":"2023-11-27T15:24:44.323459+0000","last_fullsized":"2023-11-27T15:24:44.323459+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:43:47.264444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260526+0000","last_change":"2023-11-27T15:23:27.187294+0000","last_active":"2023-11-27T15:24:40.260526+0000","last_peered":"2023-11-27T15:24:40.260526+0000","last_clean":"2023-11-27T15:24:40.260526+0000","last_became_active":"2023-11-27T15:23:27.186445+0000","last_became_peered":"2023-11-27T15:23:27.186445+0000","last_unstale":"2023-11-27T15:24:40.260526+0000","last_undegraded":"2023-11-27T15:24:40.260526+0000","last_fullsized":"2023-11-27T15:24:40.260526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:22:27.537962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.274136+0000","last_change":"2023-11-27T15:24:40.274136+0000","last_active":"2023-11-27T15:24:40.274136+0000","last_peered":"2023-11-27T15:24:40.274136+0000","last_clean":"2023-11-27T15:24:40.274136+0000","last_became_active":"2023-11-27T15:24:40.273756+0000","last_became_peered":"2023-11-27T15:24:40.273756+0000","last_unstale":"2023-11-27T15:24:40.274136+0000","last_undegraded":"2023-11-27T15:24:40.274136+0000","last_fullsized":"2023-11-27T15:24:40.274136+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:05:43.435900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.315217+0000","last_change":"2023-11-27T15:23:26.602182+0000","last_active":"2023-11-27T15:24:40.315217+0000","last_peered":"2023-11-27T15:24:40.315217+0000","last_clean":"2023-11-27T15:24:40.315217+0000","last_became_active":"2023-11-27T15:23:26.601828+0000","last_became_peered":"2023-11-27T15:23:26.601828+0000","last_unstale":"2023-11-27T15:24:40.315217+0000","last_undegraded":"2023-11-27T15:24:40.315217+0000","last_fullsized":"2023-11-27T15:24:40.315217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:24:23.918955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256780+0000","last_change":"2023-11-27T15:23:27.204874+0000","last_active":"2023-11-27T15:24:40.256780+0000","last_peered":"2023-11-27T15:24:40.256780+0000","last_clean":"2023-11-27T15:24:40.256780+0000","last_became_active":"2023-11-27T15:23:27.204588+0000","last_became_peered":"2023-11-27T15:23:27.204588+0000","last_unstale":"2023-11-27T15:24:40.256780+0000","last_undegraded":"2023-11-27T15:24:40.256780+0000","last_fullsized":"2023-11-27T15:24:40.256780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:48:18.899473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.321364+0000","last_change":"2023-11-27T15:24:40.321364+0000","last_active":"2023-11-27T15:24:40.321364+0000","last_peered":"2023-11-27T15:24:40.321364+0000","last_clean":"2023-11-27T15:24:40.321364+0000","last_became_active":"2023-11-27T15:24:40.320919+0000","last_became_peered":"2023-11-27T15:24:40.320919+0000","last_unstale":"2023-11-27T15:24:40.321364+0000","last_undegraded":"2023-11-27T15:24:40.321364+0000","last_fullsized":"2023-11-27T15:24:40.321364+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:18:07.203277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259416+0000","last_change":"2023-11-27T15:23:26.610208+0000","last_active":"2023-11-27T15:24:40.259416+0000","last_peered":"2023-11-27T15:24:40.259416+0000","last_clean":"2023-11-27T15:24:40.259416+0000","last_became_active":"2023-11-27T15:23:26.609922+0000","last_became_peered":"2023-11-27T15:23:26.609922+0000","last_unstale":"2023-11-27T15:24:40.259416+0000","last_undegraded":"2023-11-27T15:24:40.259416+0000","last_fullsized":"2023-11-27T15:24:40.259416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:58:45.501554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268646+0000","last_change":"2023-11-27T15:23:27.201533+0000","last_active":"2023-11-27T15:24:40.268646+0000","last_peered":"2023-11-27T15:24:40.268646+0000","last_clean":"2023-11-27T15:24:40.268646+0000","last_became_active":"2023-11-27T15:23:27.200516+0000","last_became_peered":"2023-11-27T15:23:27.200516+0000","last_unstale":"2023-11-27T15:24:40.268646+0000","last_undegraded":"2023-11-27T15:24:40.268646+0000","last_fullsized":"2023-11-27T15:24:40.268646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:41:48.851813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261964+0000","last_change":"2023-11-27T15:23:31.642858+0000","last_active":"2023-11-27T15:24:40.261964+0000","last_peered":"2023-11-27T15:24:40.261964+0000","last_clean":"2023-11-27T15:24:40.261964+0000","last_became_active":"2023-11-27T15:23:31.642367+0000","last_became_peered":"2023-11-27T15:23:31.642367+0000","last_unstale":"2023-11-27T15:24:40.261964+0000","last_undegraded":"2023-11-27T15:24:40.261964+0000","last_fullsized":"2023-11-27T15:24:40.261964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:52:35.281560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256044+0000","last_change":"2023-11-27T15:23:26.602865+0000","last_active":"2023-11-27T15:24:40.256044+0000","last_peered":"2023-11-27T15:24:40.256044+0000","last_clean":"2023-11-27T15:24:40.256044+0000","last_became_active":"2023-11-27T15:23:26.602591+0000","last_became_peered":"2023-11-27T15:23:26.602591+0000","last_unstale":"2023-11-27T15:24:40.256044+0000","last_undegraded":"2023-11-27T15:24:40.256044+0000","last_fullsized":"2023-11-27T15:24:40.256044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:26:20.362045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.263574+0000","last_change":"2023-11-27T15:23:27.186551+0000","last_active":"2023-11-27T15:24:40.263574+0000","last_peered":"2023-11-27T15:24:40.263574+0000","last_clean":"2023-11-27T15:24:40.263574+0000","last_became_active":"2023-11-27T15:23:27.185713+0000","last_became_peered":"2023-11-27T15:23:27.185713+0000","last_unstale":"2023-11-27T15:24:40.263574+0000","last_undegraded":"2023-11-27T15:24:40.263574+0000","last_fullsized":"2023-11-27T15:24:40.263574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:52:53.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268719+0000","last_change":"2023-11-27T15:23:31.641039+0000","last_active":"2023-11-27T15:24:40.268719+0000","last_peered":"2023-11-27T15:24:40.268719+0000","last_clean":"2023-11-27T15:24:40.268719+0000","last_became_active":"2023-11-27T15:23:31.639234+0000","last_became_peered":"2023-11-27T15:23:31.639234+0000","last_unstale":"2023-11-27T15:24:40.268719+0000","last_undegraded":"2023-11-27T15:24:40.268719+0000","last_fullsized":"2023-11-27T15:24:40.268719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:06:56.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:48.328836+0000","last_change":"2023-11-27T15:23:26.606409+0000","last_active":"2023-11-27T15:24:48.328836+0000","last_peered":"2023-11-27T15:24:48.328836+0000","last_clean":"2023-11-27T15:24:48.328836+0000","last_became_active":"2023-11-27T15:23:26.606060+0000","last_became_peered":"2023-11-27T15:23:26.606060+0000","last_unstale":"2023-11-27T15:24:48.328836+0000","last_undegraded":"2023-11-27T15:24:48.328836+0000","last_fullsized":"2023-11-27T15:24:48.328836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:52:17.764842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256965+0000","last_change":"2023-11-27T15:23:27.180766+0000","last_active":"2023-11-27T15:24:40.256965+0000","last_peered":"2023-11-27T15:24:40.256965+0000","last_clean":"2023-11-27T15:24:40.256965+0000","last_became_active":"2023-11-27T15:23:27.180382+0000","last_became_peered":"2023-11-27T15:23:27.180382+0000","last_unstale":"2023-11-27T15:24:40.256965+0000","last_undegraded":"2023-11-27T15:24:40.256965+0000","last_fullsized":"2023-11-27T15:24:40.256965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:30:32.613171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261339+0000","last_change":"2023-11-27T15:23:31.641697+0000","last_active":"2023-11-27T15:24:40.261339+0000","last_peered":"2023-11-27T15:24:40.261339+0000","last_clean":"2023-11-27T15:24:40.261339+0000","last_became_active":"2023-11-27T15:23:31.641246+0000","last_became_peered":"2023-11-27T15:23:31.641246+0000","last_unstale":"2023-11-27T15:24:40.261339+0000","last_undegraded":"2023-11-27T15:24:40.261339+0000","last_fullsized":"2023-11-27T15:24:40.261339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:36.645857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391560+0000","last_change":"2023-11-27T15:23:26.605307+0000","last_active":"2023-11-27T15:24:40.391560+0000","last_peered":"2023-11-27T15:24:40.391560+0000","last_clean":"2023-11-27T15:24:40.391560+0000","last_became_active":"2023-11-27T15:23:26.605047+0000","last_became_peered":"2023-11-27T15:23:26.605047+0000","last_unstale":"2023-11-27T15:24:40.391560+0000","last_undegraded":"2023-11-27T15:24:40.391560+0000","last_fullsized":"2023-11-27T15:24:40.391560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:32:17.515689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.258453+0000","last_change":"2023-11-27T15:23:27.194469+0000","last_active":"2023-11-27T15:24:40.258453+0000","last_peered":"2023-11-27T15:24:40.258453+0000","last_clean":"2023-11-27T15:24:40.258453+0000","last_became_active":"2023-11-27T15:23:27.193930+0000","last_became_peered":"2023-11-27T15:23:27.193930+0000","last_unstale":"2023-11-27T15:24:40.258453+0000","last_undegraded":"2023-11-27T15:24:40.258453+0000","last_fullsized":"2023-11-27T15:24:40.258453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:59:04.654492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320418+0000","last_change":"2023-11-27T15:23:31.655787+0000","last_active":"2023-11-27T15:24:40.320418+0000","last_peered":"2023-11-27T15:24:40.320418+0000","last_clean":"2023-11-27T15:24:40.320418+0000","last_became_active":"2023-11-27T15:23:31.654847+0000","last_became_peered":"2023-11-27T15:23:31.654847+0000","last_unstale":"2023-11-27T15:24:40.320418+0000","last_undegraded":"2023-11-27T15:24:40.320418+0000","last_fullsized":"2023-11-27T15:24:40.320418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:12:47.906634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256696+0000","last_change":"2023-11-27T15:23:26.598669+0000","last_active":"2023-11-27T15:24:40.256696+0000","last_peered":"2023-11-27T15:24:40.256696+0000","last_clean":"2023-11-27T15:24:40.256696+0000","last_became_active":"2023-11-27T15:23:26.598224+0000","last_became_peered":"2023-11-27T15:23:26.598224+0000","last_unstale":"2023-11-27T15:24:40.256696+0000","last_undegraded":"2023-11-27T15:24:40.256696+0000","last_fullsized":"2023-11-27T15:24:40.256696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:26.869920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257264+0000","last_change":"2023-11-27T15:23:27.182800+0000","last_active":"2023-11-27T15:24:40.257264+0000","last_peered":"2023-11-27T15:24:40.257264+0000","last_clean":"2023-11-27T15:24:40.257264+0000","last_became_active":"2023-11-27T15:23:27.182572+0000","last_became_peered":"2023-11-27T15:23:27.182572+0000","last_unstale":"2023-11-27T15:24:40.257264+0000","last_undegraded":"2023-11-27T15:24:40.257264+0000","last_fullsized":"2023-11-27T15:24:40.257264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:23:23.268840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.265208+0000","last_change":"2023-11-27T15:23:31.645586+0000","last_active":"2023-11-27T15:24:40.265208+0000","last_peered":"2023-11-27T15:24:40.265208+0000","last_clean":"2023-11-27T15:24:40.265208+0000","last_became_active":"2023-11-27T15:23:31.644897+0000","last_became_peered":"2023-11-27T15:23:31.644897+0000","last_unstale":"2023-11-27T15:24:40.265208+0000","last_undegraded":"2023-11-27T15:24:40.265208+0000","last_fullsized":"2023-11-27T15:24:40.265208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:30:32.319019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:49.330486+0000","last_change":"2023-11-27T15:23:26.601652+0000","last_active":"2023-11-27T15:24:49.330486+0000","last_peered":"2023-11-27T15:24:49.330486+0000","last_clean":"2023-11-27T15:24:49.330486+0000","last_became_active":"2023-11-27T15:23:26.601306+0000","last_became_peered":"2023-11-27T15:23:26.601306+0000","last_unstale":"2023-11-27T15:24:49.330486+0000","last_undegraded":"2023-11-27T15:24:49.330486+0000","last_fullsized":"2023-11-27T15:24:49.330486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:30:17.858253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61422,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":112,"num_read_kb":116,"num_write":33,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260948+0000","last_change":"2023-11-27T15:23:27.186709+0000","last_active":"2023-11-27T15:24:40.260948+0000","last_peered":"2023-11-27T15:24:40.260948+0000","last_clean":"2023-11-27T15:24:40.260948+0000","last_became_active":"2023-11-27T15:23:27.186390+0000","last_became_peered":"2023-11-27T15:23:27.186390+0000","last_unstale":"2023-11-27T15:24:40.260948+0000","last_undegraded":"2023-11-27T15:24:40.260948+0000","last_fullsized":"2023-11-27T15:24:40.260948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:06:36.283058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257350+0000","last_change":"2023-11-27T15:23:31.651851+0000","last_active":"2023-11-27T15:24:40.257350+0000","last_peered":"2023-11-27T15:24:40.257350+0000","last_clean":"2023-11-27T15:24:40.257350+0000","last_became_active":"2023-11-27T15:23:31.651409+0000","last_became_peered":"2023-11-27T15:23:31.651409+0000","last_unstale":"2023-11-27T15:24:40.257350+0000","last_undegraded":"2023-11-27T15:24:40.257350+0000","last_fullsized":"2023-11-27T15:24:40.257350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:04:17.235577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259937+0000","last_change":"2023-11-27T15:23:26.602719+0000","last_active":"2023-11-27T15:24:40.259937+0000","last_peered":"2023-11-27T15:24:40.259937+0000","last_clean":"2023-11-27T15:24:40.259937+0000","last_became_active":"2023-11-27T15:23:26.602372+0000","last_became_peered":"2023-11-27T15:23:26.602372+0000","last_unstale":"2023-11-27T15:24:40.259937+0000","last_undegraded":"2023-11-27T15:24:40.259937+0000","last_fullsized":"2023-11-27T15:24:40.259937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:35:43.136903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.263099+0000","last_change":"2023-11-27T15:23:27.200479+0000","last_active":"2023-11-27T15:24:40.263099+0000","last_peered":"2023-11-27T15:24:40.263099+0000","last_clean":"2023-11-27T15:24:40.263099+0000","last_became_active":"2023-11-27T15:23:27.200155+0000","last_became_peered":"2023-11-27T15:23:27.200155+0000","last_unstale":"2023-11-27T15:24:40.263099+0000","last_undegraded":"2023-11-27T15:24:40.263099+0000","last_fullsized":"2023-11-27T15:24:40.263099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:03.091095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.392376+0000","last_change":"2023-11-27T15:24:40.392376+0000","last_active":"2023-11-27T15:24:40.392376+0000","last_peered":"2023-11-27T15:24:40.392376+0000","last_clean":"2023-11-27T15:24:40.392376+0000","last_became_active":"2023-11-27T15:24:40.391782+0000","last_became_peered":"2023-11-27T15:24:40.391782+0000","last_unstale":"2023-11-27T15:24:40.392376+0000","last_undegraded":"2023-11-27T15:24:40.392376+0000","last_fullsized":"2023-11-27T15:24:40.392376+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:12:56.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256480+0000","last_change":"2023-11-27T15:23:26.605927+0000","last_active":"2023-11-27T15:24:40.256480+0000","last_peered":"2023-11-27T15:24:40.256480+0000","last_clean":"2023-11-27T15:24:40.256480+0000","last_became_active":"2023-11-27T15:23:26.605515+0000","last_became_peered":"2023-11-27T15:23:26.605515+0000","last_unstale":"2023-11-27T15:24:40.256480+0000","last_undegraded":"2023-11-27T15:24:40.256480+0000","last_fullsized":"2023-11-27T15:24:40.256480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:22:08.421561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391225+0000","last_change":"2023-11-27T15:23:27.200237+0000","last_active":"2023-11-27T15:24:40.391225+0000","last_peered":"2023-11-27T15:24:40.391225+0000","last_clean":"2023-11-27T15:24:40.391225+0000","last_became_active":"2023-11-27T15:23:27.199952+0000","last_became_peered":"2023-11-27T15:23:27.199952+0000","last_unstale":"2023-11-27T15:24:40.391225+0000","last_undegraded":"2023-11-27T15:24:40.391225+0000","last_fullsized":"2023-11-27T15:24:40.391225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:47:19.672350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.264061+0000","last_change":"2023-11-27T15:23:31.650718+0000","last_active":"2023-11-27T15:24:40.264061+0000","last_peered":"2023-11-27T15:24:40.264061+0000","last_clean":"2023-11-27T15:24:40.264061+0000","last_became_active":"2023-11-27T15:23:31.650144+0000","last_became_peered":"2023-11-27T15:23:31.650144+0000","last_unstale":"2023-11-27T15:24:40.264061+0000","last_undegraded":"2023-11-27T15:24:40.264061+0000","last_fullsized":"2023-11-27T15:24:40.264061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:18:18.747396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":80,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:44.327430+0000","last_change":"2023-11-27T15:23:26.600962+0000","last_active":"2023-11-27T15:24:44.327430+0000","last_peered":"2023-11-27T15:24:44.327430+0000","last_clean":"2023-11-27T15:24:44.327430+0000","last_became_active":"2023-11-27T15:23:26.600628+0000","last_became_peered":"2023-11-27T15:23:26.600628+0000","last_unstale":"2023-11-27T15:24:44.327430+0000","last_undegraded":"2023-11-27T15:24:44.327430+0000","last_fullsized":"2023-11-27T15:24:44.327430+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:27:28.017653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.390863+0000","last_change":"2023-11-27T15:23:27.191939+0000","last_active":"2023-11-27T15:24:40.390863+0000","last_peered":"2023-11-27T15:24:40.390863+0000","last_clean":"2023-11-27T15:24:40.390863+0000","last_became_active":"2023-11-27T15:23:27.191634+0000","last_became_peered":"2023-11-27T15:23:27.191634+0000","last_unstale":"2023-11-27T15:24:40.390863+0000","last_undegraded":"2023-11-27T15:24:40.390863+0000","last_fullsized":"2023-11-27T15:24:40.390863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:08:48.214302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.263454+0000","last_change":"2023-11-27T15:23:31.650733+0000","last_active":"2023-11-27T15:24:40.263454+0000","last_peered":"2023-11-27T15:24:40.263454+0000","last_clean":"2023-11-27T15:24:40.263454+0000","last_became_active":"2023-11-27T15:23:31.650280+0000","last_became_peered":"2023-11-27T15:23:31.650280+0000","last_unstale":"2023-11-27T15:24:40.263454+0000","last_undegraded":"2023-11-27T15:24:40.263454+0000","last_fullsized":"2023-11-27T15:24:40.263454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:01:03.360168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.304963+0000","last_change":"2023-11-27T15:24:40.304562+0000","last_active":"2023-11-27T15:24:40.304963+0000","last_peered":"2023-11-27T15:24:40.304963+0000","last_clean":"2023-11-27T15:24:40.304963+0000","last_became_active":"2023-11-27T15:24:40.271620+0000","last_became_peered":"2023-11-27T15:24:40.271620+0000","last_unstale":"2023-11-27T15:24:40.304963+0000","last_undegraded":"2023-11-27T15:24:40.304963+0000","last_fullsized":"2023-11-27T15:24:40.304963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:32:19.491473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.321309+0000","last_change":"2023-11-27T15:24:40.320722+0000","last_active":"2023-11-27T15:24:40.321309+0000","last_peered":"2023-11-27T15:24:40.321309+0000","last_clean":"2023-11-27T15:24:40.321309+0000","last_became_active":"2023-11-27T15:24:40.320318+0000","last_became_peered":"2023-11-27T15:24:40.320318+0000","last_unstale":"2023-11-27T15:24:40.321309+0000","last_undegraded":"2023-11-27T15:24:40.321309+0000","last_fullsized":"2023-11-27T15:24:40.321309+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:16:15.476545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256313+0000","last_change":"2023-11-27T15:23:31.644073+0000","last_active":"2023-11-27T15:24:40.256313+0000","last_peered":"2023-11-27T15:24:40.256313+0000","last_clean":"2023-11-27T15:24:40.256313+0000","last_became_active":"2023-11-27T15:23:31.643689+0000","last_became_peered":"2023-11-27T15:23:31.643689+0000","last_unstale":"2023-11-27T15:24:40.256313+0000","last_undegraded":"2023-11-27T15:24:40.256313+0000","last_fullsized":"2023-11-27T15:24:40.256313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:19:47.142612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.262954+0000","last_change":"2023-11-27T15:23:26.604034+0000","last_active":"2023-11-27T15:24:40.262954+0000","last_peered":"2023-11-27T15:24:40.262954+0000","last_clean":"2023-11-27T15:24:40.262954+0000","last_became_active":"2023-11-27T15:23:26.603689+0000","last_became_peered":"2023-11-27T15:23:26.603689+0000","last_unstale":"2023-11-27T15:24:40.262954+0000","last_undegraded":"2023-11-27T15:24:40.262954+0000","last_fullsized":"2023-11-27T15:24:40.262954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:51:11.895213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260637+0000","last_change":"2023-11-27T15:23:27.192723+0000","last_active":"2023-11-27T15:24:40.260637+0000","last_peered":"2023-11-27T15:24:40.260637+0000","last_clean":"2023-11-27T15:24:40.260637+0000","last_became_active":"2023-11-27T15:23:27.192034+0000","last_became_peered":"2023-11-27T15:23:27.192034+0000","last_unstale":"2023-11-27T15:24:40.260637+0000","last_undegraded":"2023-11-27T15:24:40.260637+0000","last_fullsized":"2023-11-27T15:24:40.260637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:57:14.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260461+0000","last_change":"2023-11-27T15:23:31.652756+0000","last_active":"2023-11-27T15:24:40.260461+0000","last_peered":"2023-11-27T15:24:40.260461+0000","last_clean":"2023-11-27T15:24:40.260461+0000","last_became_active":"2023-11-27T15:23:31.652280+0000","last_became_peered":"2023-11-27T15:23:31.652280+0000","last_unstale":"2023-11-27T15:24:40.260461+0000","last_undegraded":"2023-11-27T15:24:40.260461+0000","last_fullsized":"2023-11-27T15:24:40.260461+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:40:53.743334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.264946+0000","last_change":"2023-11-27T15:23:26.609927+0000","last_active":"2023-11-27T15:24:40.264946+0000","last_peered":"2023-11-27T15:24:40.264946+0000","last_clean":"2023-11-27T15:24:40.264946+0000","last_became_active":"2023-11-27T15:23:26.608928+0000","last_became_peered":"2023-11-27T15:23:26.608928+0000","last_unstale":"2023-11-27T15:24:40.264946+0000","last_undegraded":"2023-11-27T15:24:40.264946+0000","last_fullsized":"2023-11-27T15:24:40.264946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:23:24.296172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.315649+0000","last_change":"2023-11-27T15:23:27.204619+0000","last_active":"2023-11-27T15:24:40.315649+0000","last_peered":"2023-11-27T15:24:40.315649+0000","last_clean":"2023-11-27T15:24:40.315649+0000","last_became_active":"2023-11-27T15:23:27.204298+0000","last_became_peered":"2023-11-27T15:23:27.204298+0000","last_unstale":"2023-11-27T15:24:40.315649+0000","last_undegraded":"2023-11-27T15:24:40.315649+0000","last_fullsized":"2023-11-27T15:24:40.315649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:22:34.577936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261403+0000","last_change":"2023-11-27T15:23:31.644763+0000","last_active":"2023-11-27T15:24:40.261403+0000","last_peered":"2023-11-27T15:24:40.261403+0000","last_clean":"2023-11-27T15:24:40.261403+0000","last_became_active":"2023-11-27T15:23:31.644106+0000","last_became_peered":"2023-11-27T15:23:31.644106+0000","last_unstale":"2023-11-27T15:24:40.261403+0000","last_undegraded":"2023-11-27T15:24:40.261403+0000","last_fullsized":"2023-11-27T15:24:40.261403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:05:41.326786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391024+0000","last_change":"2023-11-27T15:23:26.614703+0000","last_active":"2023-11-27T15:24:40.391024+0000","last_peered":"2023-11-27T15:24:40.391024+0000","last_clean":"2023-11-27T15:24:40.391024+0000","last_became_active":"2023-11-27T15:23:26.614324+0000","last_became_peered":"2023-11-27T15:23:26.614324+0000","last_unstale":"2023-11-27T15:24:40.391024+0000","last_undegraded":"2023-11-27T15:24:40.391024+0000","last_fullsized":"2023-11-27T15:24:40.391024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:56:54.015930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.315249+0000","last_change":"2023-11-27T15:23:27.197288+0000","last_active":"2023-11-27T15:24:40.315249+0000","last_peered":"2023-11-27T15:24:40.315249+0000","last_clean":"2023-11-27T15:24:40.315249+0000","last_became_active":"2023-11-27T15:23:27.196925+0000","last_became_peered":"2023-11-27T15:23:27.196925+0000","last_unstale":"2023-11-27T15:24:40.315249+0000","last_undegraded":"2023-11-27T15:24:40.315249+0000","last_fullsized":"2023-11-27T15:24:40.315249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:40:28.777876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.264756+0000","last_change":"2023-11-27T15:23:31.652701+0000","last_active":"2023-11-27T15:24:40.264756+0000","last_peered":"2023-11-27T15:24:40.264756+0000","last_clean":"2023-11-27T15:24:40.264756+0000","last_became_active":"2023-11-27T15:23:31.652259+0000","last_became_peered":"2023-11-27T15:23:31.652259+0000","last_unstale":"2023-11-27T15:24:40.264756+0000","last_undegraded":"2023-11-27T15:24:40.264756+0000","last_fullsized":"2023-11-27T15:24:40.264756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:14:09.426627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256267+0000","last_change":"2023-11-27T15:23:27.190103+0000","last_active":"2023-11-27T15:24:40.256267+0000","last_peered":"2023-11-27T15:24:40.256267+0000","last_clean":"2023-11-27T15:24:40.256267+0000","last_became_active":"2023-11-27T15:23:27.189803+0000","last_became_peered":"2023-11-27T15:23:27.189803+0000","last_unstale":"2023-11-27T15:24:40.256267+0000","last_undegraded":"2023-11-27T15:24:40.256267+0000","last_fullsized":"2023-11-27T15:24:40.256267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:55:45.581447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":80,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:47.332138+0000","last_change":"2023-11-27T15:23:26.606338+0000","last_active":"2023-11-27T15:24:47.332138+0000","last_peered":"2023-11-27T15:24:47.332138+0000","last_clean":"2023-11-27T15:24:47.332138+0000","last_became_active":"2023-11-27T15:23:26.601152+0000","last_became_peered":"2023-11-27T15:23:26.601152+0000","last_unstale":"2023-11-27T15:24:47.332138+0000","last_undegraded":"2023-11-27T15:24:47.332138+0000","last_fullsized":"2023-11-27T15:24:47.332138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:03:04.486749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391435+0000","last_change":"2023-11-27T15:23:31.646457+0000","last_active":"2023-11-27T15:24:40.391435+0000","last_peered":"2023-11-27T15:24:40.391435+0000","last_clean":"2023-11-27T15:24:40.391435+0000","last_became_active":"2023-11-27T15:23:31.646071+0000","last_became_peered":"2023-11-27T15:23:31.646071+0000","last_unstale":"2023-11-27T15:24:40.391435+0000","last_undegraded":"2023-11-27T15:24:40.391435+0000","last_fullsized":"2023-11-27T15:24:40.391435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:04:43.604907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268444+0000","last_change":"2023-11-27T15:23:27.201405+0000","last_active":"2023-11-27T15:24:40.268444+0000","last_peered":"2023-11-27T15:24:40.268444+0000","last_clean":"2023-11-27T15:24:40.268444+0000","last_became_active":"2023-11-27T15:23:27.200951+0000","last_became_peered":"2023-11-27T15:23:27.200951+0000","last_unstale":"2023-11-27T15:24:40.268444+0000","last_undegraded":"2023-11-27T15:24:40.268444+0000","last_fullsized":"2023-11-27T15:24:40.268444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:19:39.249483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320203+0000","last_change":"2023-11-27T15:23:26.604328+0000","last_active":"2023-11-27T15:24:40.320203+0000","last_peered":"2023-11-27T15:24:40.320203+0000","last_clean":"2023-11-27T15:24:40.320203+0000","last_became_active":"2023-11-27T15:23:26.603937+0000","last_became_peered":"2023-11-27T15:23:26.603937+0000","last_unstale":"2023-11-27T15:24:40.320203+0000","last_undegraded":"2023-11-27T15:24:40.320203+0000","last_fullsized":"2023-11-27T15:24:40.320203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:23.908983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.390673+0000","last_change":"2023-11-27T15:22:17.949235+0000","last_active":"2023-11-27T15:24:40.390673+0000","last_peered":"2023-11-27T15:24:40.390673+0000","last_clean":"2023-11-27T15:24:40.390673+0000","last_became_active":"2023-11-27T15:22:17.948822+0000","last_became_peered":"2023-11-27T15:22:17.948822+0000","last_unstale":"2023-11-27T15:24:40.390673+0000","last_undegraded":"2023-11-27T15:24:40.390673+0000","last_fullsized":"2023-11-27T15:24:40.390673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:19:02.909930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:19:02.909930+0000","last_clean_scrub_stamp":"2023-11-27T15:19:02.909930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:34:29.263598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256828+0000","last_change":"2023-11-27T15:23:31.654325+0000","last_active":"2023-11-27T15:24:40.256828+0000","last_peered":"2023-11-27T15:24:40.256828+0000","last_clean":"2023-11-27T15:24:40.256828+0000","last_became_active":"2023-11-27T15:23:31.653857+0000","last_became_peered":"2023-11-27T15:23:31.653857+0000","last_unstale":"2023-11-27T15:24:40.256828+0000","last_undegraded":"2023-11-27T15:24:40.256828+0000","last_fullsized":"2023-11-27T15:24:40.256828+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:00:23.349266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259242+0000","last_change":"2023-11-27T15:23:27.192623+0000","last_active":"2023-11-27T15:24:40.259242+0000","last_peered":"2023-11-27T15:24:40.259242+0000","last_clean":"2023-11-27T15:24:40.259242+0000","last_became_active":"2023-11-27T15:23:27.192266+0000","last_became_peered":"2023-11-27T15:23:27.192266+0000","last_unstale":"2023-11-27T15:24:40.259242+0000","last_undegraded":"2023-11-27T15:24:40.259242+0000","last_fullsized":"2023-11-27T15:24:40.259242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:30:40.703677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.390884+0000","last_change":"2023-11-27T15:23:26.603078+0000","last_active":"2023-11-27T15:24:40.390884+0000","last_peered":"2023-11-27T15:24:40.390884+0000","last_clean":"2023-11-27T15:24:40.390884+0000","last_became_active":"2023-11-27T15:23:26.602758+0000","last_became_peered":"2023-11-27T15:23:26.602758+0000","last_unstale":"2023-11-27T15:24:40.390884+0000","last_undegraded":"2023-11-27T15:24:40.390884+0000","last_fullsized":"2023-11-27T15:24:40.390884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:40:23.370790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260441+0000","last_change":"2023-11-27T15:23:31.648266+0000","last_active":"2023-11-27T15:24:40.260441+0000","last_peered":"2023-11-27T15:24:40.260441+0000","last_clean":"2023-11-27T15:24:40.260441+0000","last_became_active":"2023-11-27T15:23:31.647804+0000","last_became_peered":"2023-11-27T15:23:31.647804+0000","last_unstale":"2023-11-27T15:24:40.260441+0000","last_undegraded":"2023-11-27T15:24:40.260441+0000","last_fullsized":"2023-11-27T15:24:40.260441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:02:23.849165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261060+0000","last_change":"2023-11-27T15:23:26.615577+0000","last_active":"2023-11-27T15:24:40.261060+0000","last_peered":"2023-11-27T15:24:40.261060+0000","last_clean":"2023-11-27T15:24:40.261060+0000","last_became_active":"2023-11-27T15:23:26.615287+0000","last_became_peered":"2023-11-27T15:23:26.615287+0000","last_unstale":"2023-11-27T15:24:40.261060+0000","last_undegraded":"2023-11-27T15:24:40.261060+0000","last_fullsized":"2023-11-27T15:24:40.261060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:15:31.547687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261583+0000","last_change":"2023-11-27T15:23:27.203162+0000","last_active":"2023-11-27T15:24:40.261583+0000","last_peered":"2023-11-27T15:24:40.261583+0000","last_clean":"2023-11-27T15:24:40.261583+0000","last_became_active":"2023-11-27T15:23:27.201725+0000","last_became_peered":"2023-11-27T15:23:27.201725+0000","last_unstale":"2023-11-27T15:24:40.261583+0000","last_undegraded":"2023-11-27T15:24:40.261583+0000","last_fullsized":"2023-11-27T15:24:40.261583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:51:35.159456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268487+0000","last_change":"2023-11-27T15:23:31.653253+0000","last_active":"2023-11-27T15:24:40.268487+0000","last_peered":"2023-11-27T15:24:40.268487+0000","last_clean":"2023-11-27T15:24:40.268487+0000","last_became_active":"2023-11-27T15:23:31.652777+0000","last_became_peered":"2023-11-27T15:23:31.652777+0000","last_unstale":"2023-11-27T15:24:40.268487+0000","last_undegraded":"2023-11-27T15:24:40.268487+0000","last_fullsized":"2023-11-27T15:24:40.268487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:50:59.858365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.264228+0000","last_change":"2023-11-27T15:23:26.610599+0000","last_active":"2023-11-27T15:24:40.264228+0000","last_peered":"2023-11-27T15:24:40.264228+0000","last_clean":"2023-11-27T15:24:40.264228+0000","last_became_active":"2023-11-27T15:23:26.610205+0000","last_became_peered":"2023-11-27T15:23:26.610205+0000","last_unstale":"2023-11-27T15:24:40.264228+0000","last_undegraded":"2023-11-27T15:24:40.264228+0000","last_fullsized":"2023-11-27T15:24:40.264228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:33:15.342015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.269543+0000","last_change":"2023-11-27T15:23:27.203367+0000","last_active":"2023-11-27T15:24:40.269543+0000","last_peered":"2023-11-27T15:24:40.269543+0000","last_clean":"2023-11-27T15:24:40.269543+0000","last_became_active":"2023-11-27T15:23:27.203054+0000","last_became_peered":"2023-11-27T15:23:27.203054+0000","last_unstale":"2023-11-27T15:24:40.269543+0000","last_undegraded":"2023-11-27T15:24:40.269543+0000","last_fullsized":"2023-11-27T15:24:40.269543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:11:17.106288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.258899+0000","last_change":"2023-11-27T15:23:31.638372+0000","last_active":"2023-11-27T15:24:40.258899+0000","last_peered":"2023-11-27T15:24:40.258899+0000","last_clean":"2023-11-27T15:24:40.258899+0000","last_became_active":"2023-11-27T15:23:31.637973+0000","last_became_peered":"2023-11-27T15:23:31.637973+0000","last_unstale":"2023-11-27T15:24:40.258899+0000","last_undegraded":"2023-11-27T15:24:40.258899+0000","last_fullsized":"2023-11-27T15:24:40.258899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:03:04.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.321322+0000","last_change":"2023-11-27T15:23:26.594218+0000","last_active":"2023-11-27T15:24:40.321322+0000","last_peered":"2023-11-27T15:24:40.321322+0000","last_clean":"2023-11-27T15:24:40.321322+0000","last_became_active":"2023-11-27T15:23:26.593842+0000","last_became_peered":"2023-11-27T15:23:26.593842+0000","last_unstale":"2023-11-27T15:24:40.321322+0000","last_undegraded":"2023-11-27T15:24:40.321322+0000","last_fullsized":"2023-11-27T15:24:40.321322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:12:35.988583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.262114+0000","last_change":"2023-11-27T15:23:27.211258+0000","last_active":"2023-11-27T15:24:40.262114+0000","last_peered":"2023-11-27T15:24:40.262114+0000","last_clean":"2023-11-27T15:24:40.262114+0000","last_became_active":"2023-11-27T15:23:27.210963+0000","last_became_peered":"2023-11-27T15:23:27.210963+0000","last_unstale":"2023-11-27T15:24:40.262114+0000","last_undegraded":"2023-11-27T15:24:40.262114+0000","last_fullsized":"2023-11-27T15:24:40.262114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:46:57.760614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320998+0000","last_change":"2023-11-27T15:23:31.652938+0000","last_active":"2023-11-27T15:24:40.320998+0000","last_peered":"2023-11-27T15:24:40.320998+0000","last_clean":"2023-11-27T15:24:40.320998+0000","last_became_active":"2023-11-27T15:23:31.652504+0000","last_became_peered":"2023-11-27T15:23:31.652504+0000","last_unstale":"2023-11-27T15:24:40.320998+0000","last_undegraded":"2023-11-27T15:24:40.320998+0000","last_fullsized":"2023-11-27T15:24:40.320998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:30:17.868238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256285+0000","last_change":"2023-11-27T15:23:26.602328+0000","last_active":"2023-11-27T15:24:40.256285+0000","last_peered":"2023-11-27T15:24:40.256285+0000","last_clean":"2023-11-27T15:24:40.256285+0000","last_became_active":"2023-11-27T15:23:26.601974+0000","last_became_peered":"2023-11-27T15:23:26.601974+0000","last_unstale":"2023-11-27T15:24:40.256285+0000","last_undegraded":"2023-11-27T15:24:40.256285+0000","last_fullsized":"2023-11-27T15:24:40.256285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:25:40.773772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260086+0000","last_change":"2023-11-27T15:23:27.194577+0000","last_active":"2023-11-27T15:24:40.260086+0000","last_peered":"2023-11-27T15:24:40.260086+0000","last_clean":"2023-11-27T15:24:40.260086+0000","last_became_active":"2023-11-27T15:23:27.194186+0000","last_became_peered":"2023-11-27T15:23:27.194186+0000","last_unstale":"2023-11-27T15:24:40.260086+0000","last_undegraded":"2023-11-27T15:24:40.260086+0000","last_fullsized":"2023-11-27T15:24:40.260086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:08:52.457498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.273069+0000","last_change":"2023-11-27T15:24:40.272361+0000","last_active":"2023-11-27T15:24:40.273069+0000","last_peered":"2023-11-27T15:24:40.273069+0000","last_clean":"2023-11-27T15:24:40.273069+0000","last_became_active":"2023-11-27T15:24:40.271779+0000","last_became_peered":"2023-11-27T15:24:40.271779+0000","last_unstale":"2023-11-27T15:24:40.273069+0000","last_undegraded":"2023-11-27T15:24:40.273069+0000","last_fullsized":"2023-11-27T15:24:40.273069+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:45:31.792127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268872+0000","last_change":"2023-11-27T15:23:26.611336+0000","last_active":"2023-11-27T15:24:40.268872+0000","last_peered":"2023-11-27T15:24:40.268872+0000","last_clean":"2023-11-27T15:24:40.268872+0000","last_became_active":"2023-11-27T15:23:26.610819+0000","last_became_peered":"2023-11-27T15:23:26.610819+0000","last_unstale":"2023-11-27T15:24:40.268872+0000","last_undegraded":"2023-11-27T15:24:40.268872+0000","last_fullsized":"2023-11-27T15:24:40.268872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:56:02.723345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259915+0000","last_change":"2023-11-27T15:23:27.198336+0000","last_active":"2023-11-27T15:24:40.259915+0000","last_peered":"2023-11-27T15:24:40.259915+0000","last_clean":"2023-11-27T15:24:40.259915+0000","last_became_active":"2023-11-27T15:23:27.198062+0000","last_became_peered":"2023-11-27T15:23:27.198062+0000","last_unstale":"2023-11-27T15:24:40.259915+0000","last_undegraded":"2023-11-27T15:24:40.259915+0000","last_fullsized":"2023-11-27T15:24:40.259915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:48:59.593682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320576+0000","last_change":"2023-11-27T15:23:31.655526+0000","last_active":"2023-11-27T15:24:40.320576+0000","last_peered":"2023-11-27T15:24:40.320576+0000","last_clean":"2023-11-27T15:24:40.320576+0000","last_became_active":"2023-11-27T15:23:31.655011+0000","last_became_peered":"2023-11-27T15:23:31.655011+0000","last_unstale":"2023-11-27T15:24:40.320576+0000","last_undegraded":"2023-11-27T15:24:40.320576+0000","last_fullsized":"2023-11-27T15:24:40.320576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:51:20.671016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259314+0000","last_change":"2023-11-27T15:23:26.604536+0000","last_active":"2023-11-27T15:24:40.259314+0000","last_peered":"2023-11-27T15:24:40.259314+0000","last_clean":"2023-11-27T15:24:40.259314+0000","last_became_active":"2023-11-27T15:23:26.604239+0000","last_became_peered":"2023-11-27T15:23:26.604239+0000","last_unstale":"2023-11-27T15:24:40.259314+0000","last_undegraded":"2023-11-27T15:24:40.259314+0000","last_fullsized":"2023-11-27T15:24:40.259314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:27:04.229005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.269527+0000","last_change":"2023-11-27T15:23:27.201159+0000","last_active":"2023-11-27T15:24:40.269527+0000","last_peered":"2023-11-27T15:24:40.269527+0000","last_clean":"2023-11-27T15:24:40.269527+0000","last_became_active":"2023-11-27T15:23:27.200800+0000","last_became_peered":"2023-11-27T15:23:27.200800+0000","last_unstale":"2023-11-27T15:24:40.269527+0000","last_undegraded":"2023-11-27T15:24:40.269527+0000","last_fullsized":"2023-11-27T15:24:40.269527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:30:05.805286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.321029+0000","last_change":"2023-11-27T15:23:31.652835+0000","last_active":"2023-11-27T15:24:40.321029+0000","last_peered":"2023-11-27T15:24:40.321029+0000","last_clean":"2023-11-27T15:24:40.321029+0000","last_became_active":"2023-11-27T15:23:31.652425+0000","last_became_peered":"2023-11-27T15:23:31.652425+0000","last_unstale":"2023-11-27T15:24:40.321029+0000","last_undegraded":"2023-11-27T15:24:40.321029+0000","last_fullsized":"2023-11-27T15:24:40.321029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:58:38.875484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260732+0000","last_change":"2023-11-27T15:23:26.606303+0000","last_active":"2023-11-27T15:24:40.260732+0000","last_peered":"2023-11-27T15:24:40.260732+0000","last_clean":"2023-11-27T15:24:40.260732+0000","last_became_active":"2023-11-27T15:23:26.605827+0000","last_became_peered":"2023-11-27T15:23:26.605827+0000","last_unstale":"2023-11-27T15:24:40.260732+0000","last_undegraded":"2023-11-27T15:24:40.260732+0000","last_fullsized":"2023-11-27T15:24:40.260732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:15:07.123396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391060+0000","last_change":"2023-11-27T15:23:27.198422+0000","last_active":"2023-11-27T15:24:40.391060+0000","last_peered":"2023-11-27T15:24:40.391060+0000","last_clean":"2023-11-27T15:24:40.391060+0000","last_became_active":"2023-11-27T15:23:27.198116+0000","last_became_peered":"2023-11-27T15:23:27.198116+0000","last_unstale":"2023-11-27T15:24:40.391060+0000","last_undegraded":"2023-11-27T15:24:40.391060+0000","last_fullsized":"2023-11-27T15:24:40.391060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:52:18.469771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256291+0000","last_change":"2023-11-27T15:23:31.647798+0000","last_active":"2023-11-27T15:24:40.256291+0000","last_peered":"2023-11-27T15:24:40.256291+0000","last_clean":"2023-11-27T15:24:40.256291+0000","last_became_active":"2023-11-27T15:23:31.647087+0000","last_became_peered":"2023-11-27T15:23:31.647087+0000","last_unstale":"2023-11-27T15:24:40.256291+0000","last_undegraded":"2023-11-27T15:24:40.256291+0000","last_fullsized":"2023-11-27T15:24:40.256291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:17:43.516721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.273308+0000","last_change":"2023-11-27T15:24:40.272888+0000","last_active":"2023-11-27T15:24:40.273308+0000","last_peered":"2023-11-27T15:24:40.273308+0000","last_clean":"2023-11-27T15:24:40.273308+0000","last_became_active":"2023-11-27T15:24:40.264563+0000","last_became_peered":"2023-11-27T15:24:40.264563+0000","last_unstale":"2023-11-27T15:24:40.273308+0000","last_undegraded":"2023-11-27T15:24:40.273308+0000","last_fullsized":"2023-11-27T15:24:40.273308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:55:28.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391666+0000","last_change":"2023-11-27T15:23:27.186270+0000","last_active":"2023-11-27T15:24:40.391666+0000","last_peered":"2023-11-27T15:24:40.391666+0000","last_clean":"2023-11-27T15:24:40.391666+0000","last_became_active":"2023-11-27T15:23:27.185952+0000","last_became_peered":"2023-11-27T15:23:27.185952+0000","last_unstale":"2023-11-27T15:24:40.391666+0000","last_undegraded":"2023-11-27T15:24:40.391666+0000","last_fullsized":"2023-11-27T15:24:40.391666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:38:12.220989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.258975+0000","last_change":"2023-11-27T15:23:31.648666+0000","last_active":"2023-11-27T15:24:40.258975+0000","last_peered":"2023-11-27T15:24:40.258975+0000","last_clean":"2023-11-27T15:24:40.258975+0000","last_became_active":"2023-11-27T15:23:31.647245+0000","last_became_peered":"2023-11-27T15:23:31.647245+0000","last_unstale":"2023-11-27T15:24:40.258975+0000","last_undegraded":"2023-11-27T15:24:40.258975+0000","last_fullsized":"2023-11-27T15:24:40.258975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:33:56.207371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256961+0000","last_change":"2023-11-27T15:23:26.606886+0000","last_active":"2023-11-27T15:24:40.256961+0000","last_peered":"2023-11-27T15:24:40.256961+0000","last_clean":"2023-11-27T15:24:40.256961+0000","last_became_active":"2023-11-27T15:23:26.606556+0000","last_became_peered":"2023-11-27T15:23:26.606556+0000","last_unstale":"2023-11-27T15:24:40.256961+0000","last_undegraded":"2023-11-27T15:24:40.256961+0000","last_fullsized":"2023-11-27T15:24:40.256961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:45:16.168185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391373+0000","last_change":"2023-11-27T15:23:27.194532+0000","last_active":"2023-11-27T15:24:40.391373+0000","last_peered":"2023-11-27T15:24:40.391373+0000","last_clean":"2023-11-27T15:24:40.391373+0000","last_became_active":"2023-11-27T15:23:27.194202+0000","last_became_peered":"2023-11-27T15:23:27.194202+0000","last_unstale":"2023-11-27T15:24:40.391373+0000","last_undegraded":"2023-11-27T15:24:40.391373+0000","last_fullsized":"2023-11-27T15:24:40.391373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:56:54.160848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.255949+0000","last_change":"2023-11-27T15:23:31.638466+0000","last_active":"2023-11-27T15:24:40.255949+0000","last_peered":"2023-11-27T15:24:40.255949+0000","last_clean":"2023-11-27T15:24:40.255949+0000","last_became_active":"2023-11-27T15:23:31.638103+0000","last_became_peered":"2023-11-27T15:23:31.638103+0000","last_unstale":"2023-11-27T15:24:40.255949+0000","last_undegraded":"2023-11-27T15:24:40.255949+0000","last_fullsized":"2023-11-27T15:24:40.255949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:06:34.847983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.472903+0000","last_change":"2023-11-27T15:24:40.472903+0000","last_active":"2023-11-27T15:24:40.472903+0000","last_peered":"2023-11-27T15:24:40.472903+0000","last_clean":"2023-11-27T15:24:40.472903+0000","last_became_active":"2023-11-27T15:24:40.321194+0000","last_became_peered":"2023-11-27T15:24:40.321194+0000","last_unstale":"2023-11-27T15:24:40.472903+0000","last_undegraded":"2023-11-27T15:24:40.472903+0000","last_fullsized":"2023-11-27T15:24:40.472903+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:04:18.954319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391695+0000","last_change":"2023-11-27T15:23:27.185677+0000","last_active":"2023-11-27T15:24:40.391695+0000","last_peered":"2023-11-27T15:24:40.391695+0000","last_clean":"2023-11-27T15:24:40.391695+0000","last_became_active":"2023-11-27T15:23:27.185321+0000","last_became_peered":"2023-11-27T15:23:27.185321+0000","last_unstale":"2023-11-27T15:24:40.391695+0000","last_undegraded":"2023-11-27T15:24:40.391695+0000","last_fullsized":"2023-11-27T15:24:40.391695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:58:02.569108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260276+0000","last_change":"2023-11-27T15:23:27.195847+0000","last_active":"2023-11-27T15:24:40.260276+0000","last_peered":"2023-11-27T15:24:40.260276+0000","last_clean":"2023-11-27T15:24:40.260276+0000","last_became_active":"2023-11-27T15:23:27.195421+0000","last_became_peered":"2023-11-27T15:23:27.195421+0000","last_unstale":"2023-11-27T15:24:40.260276+0000","last_undegraded":"2023-11-27T15:24:40.260276+0000","last_fullsized":"2023-11-27T15:24:40.260276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:32:18.774313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259311+0000","last_change":"2023-11-27T15:23:27.181352+0000","last_active":"2023-11-27T15:24:40.259311+0000","last_peered":"2023-11-27T15:24:40.259311+0000","last_clean":"2023-11-27T15:24:40.259311+0000","last_became_active":"2023-11-27T15:23:27.181042+0000","last_became_peered":"2023-11-27T15:23:27.181042+0000","last_unstale":"2023-11-27T15:24:40.259311+0000","last_undegraded":"2023-11-27T15:24:40.259311+0000","last_fullsized":"2023-11-27T15:24:40.259311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:20:09.382676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259118+0000","last_change":"2023-11-27T15:23:27.199915+0000","last_active":"2023-11-27T15:24:40.259118+0000","last_peered":"2023-11-27T15:24:40.259118+0000","last_clean":"2023-11-27T15:24:40.259118+0000","last_became_active":"2023-11-27T15:23:27.199595+0000","last_became_peered":"2023-11-27T15:23:27.199595+0000","last_unstale":"2023-11-27T15:24:40.259118+0000","last_undegraded":"2023-11-27T15:24:40.259118+0000","last_fullsized":"2023-11-27T15:24:40.259118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:20:53.989286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.262679+0000","last_change":"2023-11-27T15:23:27.210258+0000","last_active":"2023-11-27T15:24:40.262679+0000","last_peered":"2023-11-27T15:24:40.262679+0000","last_clean":"2023-11-27T15:24:40.262679+0000","last_became_active":"2023-11-27T15:23:27.208028+0000","last_became_peered":"2023-11-27T15:23:27.208028+0000","last_unstale":"2023-11-27T15:24:40.262679+0000","last_undegraded":"2023-11-27T15:24:40.262679+0000","last_fullsized":"2023-11-27T15:24:40.262679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:19:21.249912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.277411+0000","last_change":"2023-11-27T15:24:40.277411+0000","last_active":"2023-11-27T15:24:40.277411+0000","last_peered":"2023-11-27T15:24:40.277411+0000","last_clean":"2023-11-27T15:24:40.277411+0000","last_became_active":"2023-11-27T15:24:40.276642+0000","last_became_peered":"2023-11-27T15:24:40.276642+0000","last_unstale":"2023-11-27T15:24:40.277411+0000","last_undegraded":"2023-11-27T15:24:40.277411+0000","last_fullsized":"2023-11-27T15:24:40.277411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:22:44.632562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260814+0000","last_change":"2023-11-27T15:23:27.185855+0000","last_active":"2023-11-27T15:24:40.260814+0000","last_peered":"2023-11-27T15:24:40.260814+0000","last_clean":"2023-11-27T15:24:40.260814+0000","last_became_active":"2023-11-27T15:23:27.185535+0000","last_became_peered":"2023-11-27T15:23:27.185535+0000","last_unstale":"2023-11-27T15:24:40.260814+0000","last_undegraded":"2023-11-27T15:24:40.260814+0000","last_fullsized":"2023-11-27T15:24:40.260814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:38:04.393821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.262269+0000","last_change":"2023-11-27T15:23:27.190163+0000","last_active":"2023-11-27T15:24:40.262269+0000","last_peered":"2023-11-27T15:24:40.262269+0000","last_clean":"2023-11-27T15:24:40.262269+0000","last_became_active":"2023-11-27T15:23:27.188729+0000","last_became_peered":"2023-11-27T15:23:27.188729+0000","last_unstale":"2023-11-27T15:24:40.262269+0000","last_undegraded":"2023-11-27T15:24:40.262269+0000","last_fullsized":"2023-11-27T15:24:40.262269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:36:31.808113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.269343+0000","last_change":"2023-11-27T15:23:27.203904+0000","last_active":"2023-11-27T15:24:40.269343+0000","last_peered":"2023-11-27T15:24:40.269343+0000","last_clean":"2023-11-27T15:24:40.269343+0000","last_became_active":"2023-11-27T15:23:27.203578+0000","last_became_peered":"2023-11-27T15:23:27.203578+0000","last_unstale":"2023-11-27T15:24:40.269343+0000","last_undegraded":"2023-11-27T15:24:40.269343+0000","last_fullsized":"2023-11-27T15:24:40.269343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:05:42.402028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256156+0000","last_change":"2023-11-27T15:23:27.206791+0000","last_active":"2023-11-27T15:24:40.256156+0000","last_peered":"2023-11-27T15:24:40.256156+0000","last_clean":"2023-11-27T15:24:40.256156+0000","last_became_active":"2023-11-27T15:23:27.206529+0000","last_became_peered":"2023-11-27T15:23:27.206529+0000","last_unstale":"2023-11-27T15:24:40.256156+0000","last_undegraded":"2023-11-27T15:24:40.256156+0000","last_fullsized":"2023-11-27T15:24:40.256156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:25:25.559405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.269123+0000","last_change":"2023-11-27T15:23:27.197562+0000","last_active":"2023-11-27T15:24:40.269123+0000","last_peered":"2023-11-27T15:24:40.269123+0000","last_clean":"2023-11-27T15:24:40.269123+0000","last_became_active":"2023-11-27T15:23:27.197233+0000","last_became_peered":"2023-11-27T15:23:27.197233+0000","last_unstale":"2023-11-27T15:24:40.269123+0000","last_undegraded":"2023-11-27T15:24:40.269123+0000","last_fullsized":"2023-11-27T15:24:40.269123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:21:03.470847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256739+0000","last_change":"2023-11-27T15:23:27.191281+0000","last_active":"2023-11-27T15:24:40.256739+0000","last_peered":"2023-11-27T15:24:40.256739+0000","last_clean":"2023-11-27T15:24:40.256739+0000","last_became_active":"2023-11-27T15:23:27.190973+0000","last_became_peered":"2023-11-27T15:23:27.190973+0000","last_unstale":"2023-11-27T15:24:40.256739+0000","last_undegraded":"2023-11-27T15:24:40.256739+0000","last_fullsized":"2023-11-27T15:24:40.256739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:53:32.343841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260209+0000","last_change":"2023-11-27T15:23:27.199251+0000","last_active":"2023-11-27T15:24:40.260209+0000","last_peered":"2023-11-27T15:24:40.260209+0000","last_clean":"2023-11-27T15:24:40.260209+0000","last_became_active":"2023-11-27T15:23:27.198926+0000","last_became_peered":"2023-11-27T15:23:27.198926+0000","last_unstale":"2023-11-27T15:24:40.260209+0000","last_undegraded":"2023-11-27T15:24:40.260209+0000","last_fullsized":"2023-11-27T15:24:40.260209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:01:23.484571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261579+0000","last_change":"2023-11-27T15:23:27.192227+0000","last_active":"2023-11-27T15:24:40.261579+0000","last_peered":"2023-11-27T15:24:40.261579+0000","last_clean":"2023-11-27T15:24:40.261579+0000","last_became_active":"2023-11-27T15:23:27.191852+0000","last_became_peered":"2023-11-27T15:23:27.191852+0000","last_unstale":"2023-11-27T15:24:40.261579+0000","last_undegraded":"2023-11-27T15:24:40.261579+0000","last_fullsized":"2023-11-27T15:24:40.261579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:02:22.487861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65716,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":327,"num_read_kb":236,"num_write":155,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6180,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93738972,"statfs":{"total":95995035648,"available":95988707328,"internally_reserved":0,"allocated":1011712,"data_stored":490981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.48499999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85099999999999998,"5min":0.87,"15min":0.87},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.57699999999999996}]},{"osd":1,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.90200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64800000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83899999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85999999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.85999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77100000000000002,"15min":0.77000000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97299999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70899999999999996,"15min":0.71799999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.77200000000000002,"5min":1.087,"15min":1.087},"last":0.68000000000000005}]},{"osd":5,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73599999999999999,"15min":0.74299999999999999},"min":{"1min":0.46400000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.98099999999999998,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.501,"5min":0.435,"15min":0.435},"max":{"1min":1.1180000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.92700000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.71699999999999997,"15min":0.73099999999999998},"min":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79200000000000004,"5min":1.073,"15min":1.073},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.85599999999999998,"15min":0.85999999999999999},"min":{"1min":0.66400000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.032,"5min":1.651,"15min":1.651},"last":0.90900000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.88700000000000001,"15min":0.89200000000000002},"min":{"1min":0.61799999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0489999999999999,"5min":1.673,"15min":1.673},"last":1.03},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80000000000000004,"15min":0.80600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.014,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.72199999999999998}]},{"osd":8,"last update":"Mon Nov 27 15:24:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.996}]},{"osd":9,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.077,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.90600000000000003,"15min":0.90000000000000002},"min":{"1min":0.81799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1319999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.0609999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85599999999999998,"15min":0.86199999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96399999999999997,"5min":1.268,"15min":1.268},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":0.93899999999999995},"min":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0880000000000001,"5min":1.762,"15min":1.762},"last":1.0469999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645347,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7160,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93737992,"statfs":{"total":95995035648,"available":95987703808,"internally_reserved":0,"allocated":1687552,"data_stored":1139570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.629},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.56299999999999994}]},{"osd":1,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45300000000000001,"15min":0.45200000000000001},"min":{"1min":0.32400000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.622,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72399999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90200000000000002,"5min":1.014,"15min":1.014},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80000000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78000000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79800000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79700000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.40500000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98799999999999999,"5min":1.167,"15min":1.167},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.35299999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79600000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.81200000000000006}]},{"osd":5,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95399999999999996,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72999999999999998,"15min":0.72799999999999998},"min":{"1min":0.54200000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96599999999999997,"5min":1,"15min":1},"last":0.91000000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.76100000000000001},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.99399999999999999,"5min":1.125,"15min":1.125},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.3999999999999999,"5min":0.92400000000000004,"15min":0.84399999999999997},"min":{"1min":0.53700000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":13.009,"5min":13.009,"15min":13.009},"last":0.97299999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65500000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0760000000000001,"5min":1.091,"15min":1.091},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.89000000000000001}]},{"osd":9,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.405,"5min":0.94999999999999996,"15min":0.874},"min":{"1min":0.57799999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":12.989000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.86299999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.78300000000000003},"min":{"1min":0.60799999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0309999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.87},{"interface":"front","average":{"1min":0.871,"5min":0.80800000000000005,"15min":0.79800000000000004},"min":{"1min":0.72299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.95099999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":1024000,"data_stored":495201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.56100000000000005,"15min":0.57599999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64900000000000002,"5min":1.054,"15min":1.054},"last":0.62},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.75800000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68000000000000005,"15min":0.69099999999999995},"min":{"1min":0.44400000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.0029999999999999,"5min":1.046,"15min":1.046},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.044,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.81200000000000006}]},{"osd":2,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.079,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.53100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97199999999999998,"5min":1.294,"15min":1.294},"last":0.68999999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.73699999999999999,"15min":0.755},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.3520000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.123},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.66000000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.82399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.55800000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.94599999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.88800000000000001,"15min":0.90400000000000003},"min":{"1min":0.64800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3129999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.85899999999999999},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.099}]},{"osd":6,"last update":"Mon Nov 27 15:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.84299999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.94799999999999995,"15min":0.97199999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.429,"5min":2.069,"15min":2.069},"last":1.169},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.95599999999999996,"15min":0.98399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.702,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.028}]},{"osd":8,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.63900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1990000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.155},{"interface":"front","average":{"1min":0.872,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.478,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.2130000000000001}]},{"osd":10,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.88700000000000001,"15min":0.90800000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0389999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.94899999999999995,"15min":0.95099999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":1.2729999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.141}]}]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6496,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738656,"statfs":{"total":95995035648,"available":95988383744,"internally_reserved":0,"allocated":1007616,"data_stored":490582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.44800000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.85499999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.495,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.85699999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.65200000000000002,"15min":0.63200000000000001},"min":{"1min":0.56499999999999995,"5min":0.315,"15min":0.315},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.63100000000000001,"15min":0.60999999999999999},"min":{"1min":0.52000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.57099999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.36899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.40200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.60699999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.46200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93400000000000005,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.70299999999999996,"15min":0.67600000000000005},"min":{"1min":0.505,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98399999999999999,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.89700000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.67300000000000004,"15min":0.65500000000000003},"min":{"1min":0.502,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.69499999999999995,"15min":0.67800000000000005},"min":{"1min":0.59199999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.68500000000000005}]},{"osd":5,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.48099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0029999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.45100000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.75900000000000001,"15min":0.745},"min":{"1min":0.60099999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.083,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.52500000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.02,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.79900000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.753,"15min":0.72899999999999998},"min":{"1min":0.55100000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.996,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.79400000000000004,"15min":0.78000000000000003},"min":{"1min":0.67000000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.55200000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.73699999999999999,"15min":0.72199999999999998},"min":{"1min":0.63200000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.75},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.59399999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.117,"5min":1.119,"15min":1.119},"last":0.81499999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.79800000000000004,"15min":0.77900000000000003},"min":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.123,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.84099999999999997,"15min":0.82499999999999996},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.76300000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.84099999999999997}]}]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7152,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738000,"statfs":{"total":95995035648,"available":95987712000,"internally_reserved":0,"allocated":1613824,"data_stored":1087227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.35799999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46400000000000002,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.255,"15min":0.255},"max":{"1min":0.69099999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66200000000000003,"5min":0.754,"15min":0.754},"last":0.45100000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66500000000000004,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91300000000000003,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89700000000000002,"5min":1.03,"15min":1.03},"last":0.61299999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.58999999999999997},"min":{"1min":0.39400000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95799999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86799999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.59299999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.35699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.33500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79100000000000004,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53300000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.45600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88500000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93200000000000005,"5min":1.018,"15min":1.018},"last":0.72199999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.72699999999999998,"15min":0.73199999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.77800000000000002},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.876,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":0.749}]},{"osd":8,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97099999999999997,"5min":6.1050000000000004,"15min":6.1050000000000004},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82399999999999995,"15min":0.78800000000000003},"min":{"1min":0.51400000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.016,"5min":6.0330000000000004,"15min":6.0330000000000004},"last":0.873}]},{"osd":9,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.54600000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98299999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81299999999999994}]},{"osd":10,"last update":"Mon Nov 27 15:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0089999999999999,"5min":1.171,"15min":1.171},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.754,"15min":0.75900000000000001},"min":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94799999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.77400000000000002}]},{"osd":11,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.86099999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6424,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738728,"statfs":{"total":95995035648,"available":95988457472,"internally_reserved":0,"allocated":999424,"data_stored":486281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.56299999999999994,"15min":0.51100000000000001},"min":{"1min":0.34799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.3330000000000002,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":4.2240000000000002,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":0.88200000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.80800000000000005,"15min":0.753},"min":{"1min":0.55700000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.8700000000000001,"5min":4.8700000000000001,"15min":4.8700000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.71099999999999997,"15min":0.65700000000000003},"min":{"1min":0.54500000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.72199999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":0.78200000000000003,"15min":0.71099999999999997},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.274,"5min":4.274,"15min":4.274},"last":1.026},{"interface":"front","average":{"1min":1.0880000000000001,"5min":0.84599999999999997,"15min":0.78000000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":4.9470000000000001,"5min":4.9470000000000001,"15min":4.9470000000000001},"last":0.746}]},{"osd":3,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.73499999999999999,"15min":0.67700000000000005},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":5.0830000000000002,"5min":5.0830000000000002,"15min":5.0830000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.69299999999999995,"15min":0.66300000000000003},"min":{"1min":0.36199999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.8380000000000001,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":0.58799999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.873,"15min":0.82399999999999995},"min":{"1min":0.58799999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.9119999999999999,"5min":4.9119999999999999,"15min":4.9119999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":1.077,"5min":0.82599999999999996,"15min":0.77000000000000002},"min":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.8769999999999998,"5min":3.8769999999999998,"15min":3.8769999999999998},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.83299999999999996,"15min":0.79800000000000004},"min":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.9329999999999998,"5min":3.9329999999999998,"15min":3.9329999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.871,"15min":0.82299999999999995},"min":{"1min":0.60999999999999999,"5min":0.497,"15min":0.497},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":1.147}]},{"osd":7,"last update":"Mon Nov 27 15:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82199999999999995,"15min":0.81699999999999995},"min":{"1min":0.55400000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.254,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.83799999999999997,"15min":0.82699999999999996},"min":{"1min":0.60799999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0940000000000001,"5min":1.175,"15min":1.175},"last":0.78000000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":3.5070000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.6619999999999999,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":1.1100000000000001}]},{"osd":9,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.57499999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":0.93799999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.90600000000000003}]},{"osd":10,"last update":"Mon Nov 27 15:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.95699999999999996,"15min":0.95299999999999996},"min":{"1min":0.77800000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.93600000000000005,"15min":0.93000000000000005},"min":{"1min":0.71499999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.60799999999999998}]},{"osd":11,"last update":"Mon Nov 27 15:24:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":4.4089999999999998,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.80400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7580,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737572,"statfs":{"total":95995035648,"available":95987273728,"internally_reserved":0,"allocated":1069056,"data_stored":548195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.52200000000000002},"min":{"1min":0.36299999999999999,"5min":0.307,"15min":0.30599999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.55700000000000005},"min":{"1min":0.42999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.0589999999999999,"5min":6.2839999999999998,"15min":6.2839999999999998},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Nov 27 15:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.57899999999999996,"15min":0.52900000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.92900000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.749},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59599999999999997,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.77100000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58499999999999996,"15min":0.56499999999999995},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0669999999999999,"5min":6.3179999999999996,"15min":6.3179999999999996},"last":0.83999999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46899999999999997,"15min":0.42099999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.5,"15min":0.45800000000000002},"min":{"1min":0.308,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.68300000000000005,"15min":0.63300000000000001},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0069999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.51100000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.89900000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0880000000000001,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.52800000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93500000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.79400000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.372,"5min":0.316,"15min":0.316},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.71799999999999997,"15min":0.68999999999999995},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.94899999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81499999999999995,"15min":0.81100000000000005},"min":{"1min":0.55600000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.0489999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69399999999999995,"15min":0.68200000000000005},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94399999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.76400000000000001}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.86699999999999999,"15min":0.879},"min":{"1min":0.51600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1020000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.91500000000000004,"15min":0.92500000000000004},"min":{"1min":0.59599999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.997}]},{"osd":10,"last update":"Mon Nov 27 15:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89600000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.86599999999999999,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.046,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.95899999999999996}]},{"osd":11,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.98099999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7468,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737684,"statfs":{"total":95995035648,"available":95987388416,"internally_reserved":0,"allocated":1019904,"data_stored":487338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.67600000000000005,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.34799999999999998,"15min":0.33800000000000002},"max":{"1min":6.6909999999999998,"5min":6.6909999999999998,"15min":6.6909999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63400000000000001,"15min":0.56599999999999995},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.74199999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.59299999999999997,"15min":0.55000000000000004},"min":{"1min":0.49299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89000000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.629,"15min":0.58599999999999997},"min":{"1min":0.57599999999999996,"5min":0.376,"15min":0.376},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.39700000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67400000000000004,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.78000000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.71199999999999997,"15min":0.68500000000000005},"min":{"1min":0.55700000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.499},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.68700000000000006,"15min":0.67500000000000004},"min":{"1min":0.58299999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.41799999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.68700000000000006,"15min":0.64400000000000002},"min":{"1min":0.37,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.67200000000000004,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.70499999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:24:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.77200000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.57799999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.1899999999999999,"5min":0.89900000000000002,"15min":0.86099999999999999},"min":{"1min":0.65600000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":6.7220000000000004,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.627}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.89600000000000002,"15min":0.91100000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":0.874},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.032,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.94399999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.89100000000000001,"15min":0.89400000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.93200000000000005,"15min":0.93799999999999994},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.92700000000000005}]},{"osd":11,"last update":"Mon Nov 27 15:24:40 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":0.89800000000000002,"15min":0.83599999999999997},"min":{"1min":0.57499999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.71,"5min":6.71,"15min":6.71},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.05,"5min":0.92000000000000004,"15min":0.89800000000000002},"min":{"1min":0.68600000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.89400000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7656,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737496,"statfs":{"total":95995035648,"available":95987195904,"internally_reserved":0,"allocated":1081344,"data_stored":548450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.45600000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.77700000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.51300000000000001},"min":{"1min":0.46800000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78700000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Mon Nov 27 15:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.59399999999999997,"15min":0.51800000000000002},"min":{"1min":0.42199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.133,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84899999999999998,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.83599999999999997}]},{"osd":3,"last update":"Mon Nov 27 15:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67400000000000004,"15min":0.81200000000000006},"min":{"1min":0.47599999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77500000000000002,"5min":7.1189999999999998,"15min":7.1189999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.61599999999999999,"15min":0.57399999999999995},"min":{"1min":0.51400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64600000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.47999999999999998,"15min":0.45600000000000002},"min":{"1min":0.35999999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.505},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.53100000000000003,"15min":0.49299999999999999},"min":{"1min":0.39500000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.435}]},{"osd":6,"last update":"Mon Nov 27 15:23:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.65600000000000003,"15min":0.63200000000000001},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.65900000000000003,"15min":0.63900000000000001},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.88500000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.752},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.78800000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.53300000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.65900000000000003,"15min":0.627},"min":{"1min":0.64100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70199999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.72099999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.77900000000000003,"15min":0.75900000000000001},"min":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.747,"15min":0.72799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.76700000000000002}]},{"osd":11,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.81699999999999995}]}]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6756,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738396,"statfs":{"total":95995035648,"available":95988117504,"internally_reserved":0,"allocated":1011712,"data_stored":486475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48199999999999998,"15min":0.42899999999999999},"min":{"1min":0.35099999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.49399999999999999,"15min":0.45000000000000001},"min":{"1min":0.32300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1759999999999999,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.65700000000000003}]},{"osd":1,"last update":"Mon Nov 27 15:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.63400000000000001,"15min":0.58399999999999996},"min":{"1min":0.54000000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68500000000000005,"15min":0.62},"min":{"1min":0.54000000000000004,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.69399999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65400000000000003,"15min":0.627},"min":{"1min":0.505,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67300000000000004,"15min":0.64100000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.73599999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64600000000000002,"15min":0.61199999999999999},"min":{"1min":0.503,"5min":0.443,"15min":0.443},"max":{"1min":0.80800000000000005,"5min":1.034,"15min":1.034},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69399999999999995,"15min":0.64900000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88200000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68300000000000005,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88900000000000001,"5min":1.21,"15min":1.21},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.93600000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.54800000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.63700000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:24:10 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.89500000000000002,"15min":0.83199999999999996},"min":{"1min":0.48599999999999999,"5min":0.442,"15min":0.442},"max":{"1min":8.4710000000000001,"5min":8.4710000000000001,"15min":8.4710000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80400000000000005},"min":{"1min":0.57999999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0209999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.81100000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:23:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.81399999999999995},"min":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.84699999999999998,"15min":0.84999999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.80500000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:23:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.83399999999999996,"15min":0.85099999999999998},"min":{"1min":0.54000000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96399999999999997,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.93700000000000006,"15min":0.95199999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.028,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.874}]},{"osd":10,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.92300000000000004,"15min":0.93600000000000005},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.0349999999999999,"5min":1.462,"15min":1.462},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.85299999999999998,"15min":0.86599999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":0.95699999999999996,"5min":1.234,"15min":1.234},"last":0.85999999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85599999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888127,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6624,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738528,"statfs":{"total":95995035648,"available":95988252672,"internally_reserved":0,"allocated":1007616,"data_stored":490942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.58399999999999996}]},{"osd":1,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.375},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.70899999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.502},{"interface":"front","average":{"1min":0.498,"5min":0.436,"15min":0.39600000000000002},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.53300000000000003}]},{"osd":2,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72199999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61099999999999999,"15min":0.59599999999999997},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.62,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":0.85499999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.58699999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":1.329,"15min":1.329},"last":0.57099999999999995}]},{"osd":5,"last update":"Mon Nov 27 15:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.60799999999999998},"min":{"1min":0.51200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.68200000000000005,"15min":0.628},"min":{"1min":0.52300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93899999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:23:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.46800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97599999999999998,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.44500000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.84699999999999998,"5min":1.288,"15min":1.288},"last":0.66900000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.42899999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92000000000000004,"5min":1.171,"15min":1.171},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70499999999999996,"15min":0.69999999999999996},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.022,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68799999999999994}]},{"osd":8,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.52700000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97999999999999998,"5min":1.038,"15min":1.038},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.60299999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.97099999999999997,"5min":1.107,"15min":1.107},"last":0.871}]},{"osd":9,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1599999999999999,"5min":1.222,"15min":1.222},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.84099999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.96899999999999997,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.751}]},{"osd":10,"last update":"Mon Nov 27 15:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.79500000000000004,"15min":0.78300000000000003},"min":{"1min":0.54300000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.014,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75700000000000001,"15min":0.747},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1339999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.83799999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.93899999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":1622016,"data_stored":1082473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58599999999999997,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79200000000000004,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.41799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70199999999999996,"5min":0.873,"15min":0.873},"last":0.57099999999999995}]},{"osd":1,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.753}]},{"osd":2,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42899999999999999,"15min":0.40300000000000002},"min":{"1min":0.34000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59799999999999998,"5min":0.752,"15min":0.752},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.44900000000000001,"15min":0.42799999999999999},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65700000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.66400000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.53300000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.92400000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.64200000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64200000000000002,"15min":0.621},"min":{"1min":0.39300000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":2.097,"15min":2.097},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.623,"15min":0.56799999999999995},"min":{"1min":0.45100000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92400000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.79800000000000004}]},{"osd":6,"last update":"Mon Nov 27 15:23:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90100000000000002,"5min":1.004,"15min":1.004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91500000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.71699999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.47199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.86299999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69899999999999995,"15min":0.67500000000000004},"min":{"1min":0.55900000000000005,"5min":0.434,"15min":0.434},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.83599999999999997}]},{"osd":8,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.67000000000000004},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69599999999999995,"15min":0.68300000000000005},"min":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95899999999999996,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.76900000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.012,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77600000000000002,"15min":0.76600000000000001},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86899999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79200000000000004,"15min":0.78500000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.98999999999999999,"5min":1.026,"15min":1.026},"last":0.879},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81499999999999995,"15min":0.81100000000000005},"min":{"1min":0.68200000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:24:52.376 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-11-27T15:24:53.235 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:53.784 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:53 smithi008 ceph-mon[128788]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:53.784 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:53 smithi008 ceph-mon[128788]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 57 B/s, 0 objects/s recovering 2023-11-27T15:24:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:53 smithi060 ceph-mon[131518]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:53 smithi060 ceph-mon[131518]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 57 B/s, 0 objects/s recovering 2023-11-27T15:24:53.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:53 smithi029 ceph-mon[131166]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:53.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:53 smithi029 ceph-mon[131166]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 57 B/s, 0 objects/s recovering 2023-11-27T15:24:54.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.754+0000 7f932f099700 1 -- 172.21.15.8:0/1715563933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9328100ef0 msgr2=0x7f9328104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:54.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.754+0000 7f932f099700 1 --2- 172.21.15.8:0/1715563933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9328100ef0 0x7f9328104ec0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f93140099c0 tx=0x7f9314009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.755+0000 7f932f099700 1 -- 172.21.15.8:0/1715563933 shutdown_connections 2023-11-27T15:24:54.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.755+0000 7f932f099700 1 --2- 172.21.15.8:0/1715563933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93281055f0 0x7f932810d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.755+0000 7f932f099700 1 --2- 172.21.15.8:0/1715563933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9328100550 0x7f9328100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.755+0000 7f932f099700 1 --2- 172.21.15.8:0/1715563933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9328100ef0 0x7f9328104ec0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.755+0000 7f932f099700 1 -- 172.21.15.8:0/1715563933 >> 172.21.15.8:0/1715563933 conn(0x7f93280fbdf0 msgr2=0x7f93280fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:54.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.755+0000 7f932f099700 1 -- 172.21.15.8:0/1715563933 shutdown_connections 2023-11-27T15:24:54.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.755+0000 7f932f099700 1 -- 172.21.15.8:0/1715563933 wait complete. 2023-11-27T15:24:54.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.756+0000 7f932f099700 1 Processor -- start 2023-11-27T15:24:54.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.756+0000 7f932f099700 1 -- start start 2023-11-27T15:24:54.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.756+0000 7f932f099700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9328100550 0x7f9328198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:54.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.756+0000 7f932f099700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9328100ef0 0x7f9328198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:54.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.756+0000 7f932f099700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93281055f0 0x7f932819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.756+0000 7f932f099700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932819d5e0 con 0x7f9328100550 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.756+0000 7f932f099700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932819d750 con 0x7f9328100ef0 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.756+0000 7f932f099700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932819d910 con 0x7f93281055f0 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f932ce35700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9328100550 0x7f9328198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f9327fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9328100ef0 0x7f9328198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f932ce35700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9328100550 0x7f9328198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45262/0 (socket says 172.21.15.8:45262) 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f932ce35700 1 -- 172.21.15.8:0/704216077 learned_addr learned my addr 172.21.15.8:0/704216077 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f932d636700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93281055f0 0x7f932819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f932ce35700 1 -- 172.21.15.8:0/704216077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93281055f0 msgr2=0x7f932819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f932ce35700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93281055f0 0x7f932819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f932ce35700 1 -- 172.21.15.8:0/704216077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9328100ef0 msgr2=0x7f9328198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f932ce35700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9328100ef0 0x7f9328198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f932ce35700 1 -- 172.21.15.8:0/704216077 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93140095f0 con 0x7f9328100550 2023-11-27T15:24:54.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f9327fff700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9328100ef0 0x7f9328198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:54.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f932ce35700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9328100550 0x7f9328198620 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f931c00e9a0 tx=0x7f931c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:54.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.757+0000 7f9325ffb700 1 -- 172.21.15.8:0/704216077 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c009790 con 0x7f9328100550 2023-11-27T15:24:54.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.758+0000 7f9325ffb700 1 -- 172.21.15.8:0/704216077 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f931c00cbd0 con 0x7f9328100550 2023-11-27T15:24:54.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.758+0000 7f932f099700 1 -- 172.21.15.8:0/704216077 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9328110320 con 0x7f9328100550 2023-11-27T15:24:54.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.758+0000 7f9325ffb700 1 -- 172.21.15.8:0/704216077 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c018680 con 0x7f9328100550 2023-11-27T15:24:54.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.758+0000 7f932f099700 1 -- 172.21.15.8:0/704216077 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9328110190 con 0x7f9328100550 2023-11-27T15:24:54.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.759+0000 7f932f099700 1 -- 172.21.15.8:0/704216077 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93281104b0 con 0x7f9328100550 2023-11-27T15:24:54.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.761+0000 7f9325ffb700 1 -- 172.21.15.8:0/704216077 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f931c00cd40 con 0x7f9328100550 2023-11-27T15:24:54.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.762+0000 7f9325ffb700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f93180a11e0 0x7f93180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:54.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.762+0000 7f9325ffb700 1 -- 172.21.15.8:0/704216077 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f931c014070 con 0x7f9328100550 2023-11-27T15:24:54.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.762+0000 7f9327fff700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f93180a11e0 0x7f93180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:54.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.763+0000 7f9327fff700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f93180a11e0 0x7f93180a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9314009fd0 tx=0x7f931401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:54.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.766+0000 7f9325ffb700 1 -- 172.21.15.8:0/704216077 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f931c051730 con 0x7f9328100550 2023-11-27T15:24:54.972 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.969+0000 7f932f099700 1 -- 172.21.15.8:0/704216077 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9328073d30 con 0x7f93180a11e0 2023-11-27T15:24:54.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.982+0000 7f9325ffb700 1 -- 172.21.15.8:0/704216077 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426537 (secure 0 0 0) 0x7f9328073d30 con 0x7f93180a11e0 2023-11-27T15:24:54.993 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.987+0000 7f932f099700 1 -- 172.21.15.8:0/704216077 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f93180a11e0 msgr2=0x7f93180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.987+0000 7f932f099700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f93180a11e0 0x7f93180a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9314009fd0 tx=0x7f931401a040 comp rx=0 tx=0).stop 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.987+0000 7f932f099700 1 -- 172.21.15.8:0/704216077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9328100550 msgr2=0x7f9328198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.987+0000 7f932f099700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9328100550 0x7f9328198620 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f931c00e9a0 tx=0x7f931c00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.987+0000 7f932f099700 1 -- 172.21.15.8:0/704216077 shutdown_connections 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.987+0000 7f932f099700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f93180a11e0 0x7f93180a3690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.987+0000 7f932f099700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93281055f0 0x7f932819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.988+0000 7f932f099700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9328100550 0x7f9328198620 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.988+0000 7f932f099700 1 --2- 172.21.15.8:0/704216077 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9328100ef0 0x7f9328198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.988+0000 7f932f099700 1 -- 172.21.15.8:0/704216077 >> 172.21.15.8:0/704216077 conn(0x7f93280fbdf0 msgr2=0x7f93280fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:54.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.988+0000 7f932f099700 1 -- 172.21.15.8:0/704216077 shutdown_connections 2023-11-27T15:24:54.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:54.988+0000 7f932f099700 1 -- 172.21.15.8:0/704216077 wait complete. 2023-11-27T15:24:54.998 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2023-11-27T15:24:55.435 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2023-11-27T15:24:54.349369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656330,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":468,"num_read_kb":350,"num_write":338,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181430,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":83168,"kb_used_data":13824,"kb_used_omap":0,"kb_used_meta":69248,"kb_avail":1124858656,"statfs":{"total":1151940427776,"available":1151855263744,"internally_reserved":0,"allocated":14155776,"data_stored":7833715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70909952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":34,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003339"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261233+0000","last_change":"2023-11-27T15:23:27.211699+0000","last_active":"2023-11-27T15:24:40.261233+0000","last_peered":"2023-11-27T15:24:40.261233+0000","last_clean":"2023-11-27T15:24:40.261233+0000","last_became_active":"2023-11-27T15:23:27.211372+0000","last_became_peered":"2023-11-27T15:23:27.211372+0000","last_unstale":"2023-11-27T15:24:40.261233+0000","last_undegraded":"2023-11-27T15:24:40.261233+0000","last_fullsized":"2023-11-27T15:24:40.261233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:04:28.645927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.262233+0000","last_change":"2023-11-27T15:23:27.210567+0000","last_active":"2023-11-27T15:24:40.262233+0000","last_peered":"2023-11-27T15:24:40.262233+0000","last_clean":"2023-11-27T15:24:40.262233+0000","last_became_active":"2023-11-27T15:23:27.208340+0000","last_became_peered":"2023-11-27T15:23:27.208340+0000","last_unstale":"2023-11-27T15:24:40.262233+0000","last_undegraded":"2023-11-27T15:24:40.262233+0000","last_fullsized":"2023-11-27T15:24:40.262233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:55:01.081997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257911+0000","last_change":"2023-11-27T15:23:27.196040+0000","last_active":"2023-11-27T15:24:40.257911+0000","last_peered":"2023-11-27T15:24:40.257911+0000","last_clean":"2023-11-27T15:24:40.257911+0000","last_became_active":"2023-11-27T15:23:27.195195+0000","last_became_peered":"2023-11-27T15:23:27.195195+0000","last_unstale":"2023-11-27T15:24:40.257911+0000","last_undegraded":"2023-11-27T15:24:40.257911+0000","last_fullsized":"2023-11-27T15:24:40.257911+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:07:01.884552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320800+0000","last_change":"2023-11-27T15:23:27.190096+0000","last_active":"2023-11-27T15:24:40.320800+0000","last_peered":"2023-11-27T15:24:40.320800+0000","last_clean":"2023-11-27T15:24:40.320800+0000","last_became_active":"2023-11-27T15:23:27.189768+0000","last_became_peered":"2023-11-27T15:23:27.189768+0000","last_unstale":"2023-11-27T15:24:40.320800+0000","last_undegraded":"2023-11-27T15:24:40.320800+0000","last_fullsized":"2023-11-27T15:24:40.320800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:00:55.203158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259735+0000","last_change":"2023-11-27T15:23:27.200974+0000","last_active":"2023-11-27T15:24:40.259735+0000","last_peered":"2023-11-27T15:24:40.259735+0000","last_clean":"2023-11-27T15:24:40.259735+0000","last_became_active":"2023-11-27T15:23:27.200647+0000","last_became_peered":"2023-11-27T15:23:27.200647+0000","last_unstale":"2023-11-27T15:24:40.259735+0000","last_undegraded":"2023-11-27T15:24:40.259735+0000","last_fullsized":"2023-11-27T15:24:40.259735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:05:54.959172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.315298+0000","last_change":"2023-11-27T15:23:27.188417+0000","last_active":"2023-11-27T15:24:40.315298+0000","last_peered":"2023-11-27T15:24:40.315298+0000","last_clean":"2023-11-27T15:24:40.315298+0000","last_became_active":"2023-11-27T15:23:27.188079+0000","last_became_peered":"2023-11-27T15:23:27.188079+0000","last_unstale":"2023-11-27T15:24:40.315298+0000","last_undegraded":"2023-11-27T15:24:40.315298+0000","last_fullsized":"2023-11-27T15:24:40.315298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:22:04.529440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.390841+0000","last_change":"2023-11-27T15:23:27.192410+0000","last_active":"2023-11-27T15:24:40.390841+0000","last_peered":"2023-11-27T15:24:40.390841+0000","last_clean":"2023-11-27T15:24:40.390841+0000","last_became_active":"2023-11-27T15:23:27.192026+0000","last_became_peered":"2023-11-27T15:23:27.192026+0000","last_unstale":"2023-11-27T15:24:40.390841+0000","last_undegraded":"2023-11-27T15:24:40.390841+0000","last_fullsized":"2023-11-27T15:24:40.390841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:58:54.291338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268554+0000","last_change":"2023-11-27T15:23:27.205778+0000","last_active":"2023-11-27T15:24:40.268554+0000","last_peered":"2023-11-27T15:24:40.268554+0000","last_clean":"2023-11-27T15:24:40.268554+0000","last_became_active":"2023-11-27T15:23:27.205467+0000","last_became_peered":"2023-11-27T15:23:27.205467+0000","last_unstale":"2023-11-27T15:24:40.268554+0000","last_undegraded":"2023-11-27T15:24:40.268554+0000","last_fullsized":"2023-11-27T15:24:40.268554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:58:16.116452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257004+0000","last_change":"2023-11-27T15:23:27.187920+0000","last_active":"2023-11-27T15:24:40.257004+0000","last_peered":"2023-11-27T15:24:40.257004+0000","last_clean":"2023-11-27T15:24:40.257004+0000","last_became_active":"2023-11-27T15:23:27.186989+0000","last_became_peered":"2023-11-27T15:23:27.186989+0000","last_unstale":"2023-11-27T15:24:40.257004+0000","last_undegraded":"2023-11-27T15:24:40.257004+0000","last_fullsized":"2023-11-27T15:24:40.257004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:25:30.816437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.315866+0000","last_change":"2023-11-27T15:23:27.181330+0000","last_active":"2023-11-27T15:24:40.315866+0000","last_peered":"2023-11-27T15:24:40.315866+0000","last_clean":"2023-11-27T15:24:40.315866+0000","last_became_active":"2023-11-27T15:23:27.180986+0000","last_became_peered":"2023-11-27T15:23:27.180986+0000","last_unstale":"2023-11-27T15:24:40.315866+0000","last_undegraded":"2023-11-27T15:24:40.315866+0000","last_fullsized":"2023-11-27T15:24:40.315866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:00:04.601663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391622+0000","last_change":"2023-11-27T15:23:27.190673+0000","last_active":"2023-11-27T15:24:40.391622+0000","last_peered":"2023-11-27T15:24:40.391622+0000","last_clean":"2023-11-27T15:24:40.391622+0000","last_became_active":"2023-11-27T15:23:27.186256+0000","last_became_peered":"2023-11-27T15:23:27.186256+0000","last_unstale":"2023-11-27T15:24:40.391622+0000","last_undegraded":"2023-11-27T15:24:40.391622+0000","last_fullsized":"2023-11-27T15:24:40.391622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:42:50.554823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.317407+0000","last_change":"2023-11-27T15:24:40.316671+0000","last_active":"2023-11-27T15:24:40.317407+0000","last_peered":"2023-11-27T15:24:40.317407+0000","last_clean":"2023-11-27T15:24:40.317407+0000","last_became_active":"2023-11-27T15:24:40.316245+0000","last_became_peered":"2023-11-27T15:24:40.316245+0000","last_unstale":"2023-11-27T15:24:40.317407+0000","last_undegraded":"2023-11-27T15:24:40.317407+0000","last_fullsized":"2023-11-27T15:24:40.317407+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:21:28.469719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260086+0000","last_change":"2023-11-27T15:23:27.190146+0000","last_active":"2023-11-27T15:24:40.260086+0000","last_peered":"2023-11-27T15:24:40.260086+0000","last_clean":"2023-11-27T15:24:40.260086+0000","last_became_active":"2023-11-27T15:23:27.189133+0000","last_became_peered":"2023-11-27T15:23:27.189133+0000","last_unstale":"2023-11-27T15:24:40.260086+0000","last_undegraded":"2023-11-27T15:24:40.260086+0000","last_fullsized":"2023-11-27T15:24:40.260086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:11:57.964507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.258139+0000","last_change":"2023-11-27T15:23:27.207461+0000","last_active":"2023-11-27T15:24:40.258139+0000","last_peered":"2023-11-27T15:24:40.258139+0000","last_clean":"2023-11-27T15:24:40.258139+0000","last_became_active":"2023-11-27T15:23:27.206837+0000","last_became_peered":"2023-11-27T15:23:27.206837+0000","last_unstale":"2023-11-27T15:24:40.258139+0000","last_undegraded":"2023-11-27T15:24:40.258139+0000","last_fullsized":"2023-11-27T15:24:40.258139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:36:53.427531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257998+0000","last_change":"2023-11-27T15:23:27.211206+0000","last_active":"2023-11-27T15:24:40.257998+0000","last_peered":"2023-11-27T15:24:40.257998+0000","last_clean":"2023-11-27T15:24:40.257998+0000","last_became_active":"2023-11-27T15:23:27.209933+0000","last_became_peered":"2023-11-27T15:23:27.209933+0000","last_unstale":"2023-11-27T15:24:40.257998+0000","last_undegraded":"2023-11-27T15:24:40.257998+0000","last_fullsized":"2023-11-27T15:24:40.257998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:01:16.466116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256415+0000","last_change":"2023-11-27T15:23:27.188863+0000","last_active":"2023-11-27T15:24:40.256415+0000","last_peered":"2023-11-27T15:24:40.256415+0000","last_clean":"2023-11-27T15:24:40.256415+0000","last_became_active":"2023-11-27T15:23:27.188534+0000","last_became_peered":"2023-11-27T15:23:27.188534+0000","last_unstale":"2023-11-27T15:24:40.256415+0000","last_undegraded":"2023-11-27T15:24:40.256415+0000","last_fullsized":"2023-11-27T15:24:40.256415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:29:05.866285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261126+0000","last_change":"2023-11-27T15:23:27.211786+0000","last_active":"2023-11-27T15:24:40.261126+0000","last_peered":"2023-11-27T15:24:40.261126+0000","last_clean":"2023-11-27T15:24:40.261126+0000","last_became_active":"2023-11-27T15:23:27.211475+0000","last_became_peered":"2023-11-27T15:23:27.211475+0000","last_unstale":"2023-11-27T15:24:40.261126+0000","last_undegraded":"2023-11-27T15:24:40.261126+0000","last_fullsized":"2023-11-27T15:24:40.261126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:58:15.676400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.263816+0000","last_change":"2023-11-27T15:23:27.183246+0000","last_active":"2023-11-27T15:24:40.263816+0000","last_peered":"2023-11-27T15:24:40.263816+0000","last_clean":"2023-11-27T15:24:40.263816+0000","last_became_active":"2023-11-27T15:23:27.182454+0000","last_became_peered":"2023-11-27T15:23:27.182454+0000","last_unstale":"2023-11-27T15:24:40.263816+0000","last_undegraded":"2023-11-27T15:24:40.263816+0000","last_fullsized":"2023-11-27T15:24:40.263816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:12:50.045523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260441+0000","last_change":"2023-11-27T15:23:27.196461+0000","last_active":"2023-11-27T15:24:40.260441+0000","last_peered":"2023-11-27T15:24:40.260441+0000","last_clean":"2023-11-27T15:24:40.260441+0000","last_became_active":"2023-11-27T15:23:27.196122+0000","last_became_peered":"2023-11-27T15:23:27.196122+0000","last_unstale":"2023-11-27T15:24:40.260441+0000","last_undegraded":"2023-11-27T15:24:40.260441+0000","last_fullsized":"2023-11-27T15:24:40.260441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:31:30.005963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259166+0000","last_change":"2023-11-27T15:23:31.649582+0000","last_active":"2023-11-27T15:24:40.259166+0000","last_peered":"2023-11-27T15:24:40.259166+0000","last_clean":"2023-11-27T15:24:40.259166+0000","last_became_active":"2023-11-27T15:23:31.649172+0000","last_became_peered":"2023-11-27T15:23:31.649172+0000","last_unstale":"2023-11-27T15:24:40.259166+0000","last_undegraded":"2023-11-27T15:24:40.259166+0000","last_fullsized":"2023-11-27T15:24:40.259166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:18:30.987836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.264701+0000","last_change":"2023-11-27T15:23:26.609457+0000","last_active":"2023-11-27T15:24:40.264701+0000","last_peered":"2023-11-27T15:24:40.264701+0000","last_clean":"2023-11-27T15:24:40.264701+0000","last_became_active":"2023-11-27T15:23:26.609144+0000","last_became_peered":"2023-11-27T15:23:26.609144+0000","last_unstale":"2023-11-27T15:24:40.264701+0000","last_undegraded":"2023-11-27T15:24:40.264701+0000","last_fullsized":"2023-11-27T15:24:40.264701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:20:10.079237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.270347+0000","last_change":"2023-11-27T15:23:27.204423+0000","last_active":"2023-11-27T15:24:40.270347+0000","last_peered":"2023-11-27T15:24:40.270347+0000","last_clean":"2023-11-27T15:24:40.270347+0000","last_became_active":"2023-11-27T15:23:27.204104+0000","last_became_peered":"2023-11-27T15:23:27.204104+0000","last_unstale":"2023-11-27T15:24:40.270347+0000","last_undegraded":"2023-11-27T15:24:40.270347+0000","last_fullsized":"2023-11-27T15:24:40.270347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:36:02.362505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.319676+0000","last_change":"2023-11-27T15:23:31.655641+0000","last_active":"2023-11-27T15:24:40.319676+0000","last_peered":"2023-11-27T15:24:40.319676+0000","last_clean":"2023-11-27T15:24:40.319676+0000","last_became_active":"2023-11-27T15:23:31.654991+0000","last_became_peered":"2023-11-27T15:23:31.654991+0000","last_unstale":"2023-11-27T15:24:40.319676+0000","last_undegraded":"2023-11-27T15:24:40.319676+0000","last_fullsized":"2023-11-27T15:24:40.319676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:07:47.588692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.270644+0000","last_change":"2023-11-27T15:23:26.620780+0000","last_active":"2023-11-27T15:24:40.270644+0000","last_peered":"2023-11-27T15:24:40.270644+0000","last_clean":"2023-11-27T15:24:40.270644+0000","last_became_active":"2023-11-27T15:23:26.620438+0000","last_became_peered":"2023-11-27T15:23:26.620438+0000","last_unstale":"2023-11-27T15:24:40.270644+0000","last_undegraded":"2023-11-27T15:24:40.270644+0000","last_fullsized":"2023-11-27T15:24:40.270644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:00:45.116977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.258580+0000","last_change":"2023-11-27T15:23:27.195581+0000","last_active":"2023-11-27T15:24:40.258580+0000","last_peered":"2023-11-27T15:24:40.258580+0000","last_clean":"2023-11-27T15:24:40.258580+0000","last_became_active":"2023-11-27T15:23:27.195260+0000","last_became_peered":"2023-11-27T15:23:27.195260+0000","last_unstale":"2023-11-27T15:24:40.258580+0000","last_undegraded":"2023-11-27T15:24:40.258580+0000","last_fullsized":"2023-11-27T15:24:40.258580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:05:39.695701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320045+0000","last_change":"2023-11-27T15:23:31.655464+0000","last_active":"2023-11-27T15:24:40.320045+0000","last_peered":"2023-11-27T15:24:40.320045+0000","last_clean":"2023-11-27T15:24:40.320045+0000","last_became_active":"2023-11-27T15:23:31.654952+0000","last_became_peered":"2023-11-27T15:23:31.654952+0000","last_unstale":"2023-11-27T15:24:40.320045+0000","last_undegraded":"2023-11-27T15:24:40.320045+0000","last_fullsized":"2023-11-27T15:24:40.320045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:09:29.902445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261205+0000","last_change":"2023-11-27T15:23:26.606593+0000","last_active":"2023-11-27T15:24:40.261205+0000","last_peered":"2023-11-27T15:24:40.261205+0000","last_clean":"2023-11-27T15:24:40.261205+0000","last_became_active":"2023-11-27T15:23:26.605719+0000","last_became_peered":"2023-11-27T15:23:26.605719+0000","last_unstale":"2023-11-27T15:24:40.261205+0000","last_undegraded":"2023-11-27T15:24:40.261205+0000","last_fullsized":"2023-11-27T15:24:40.261205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:10.147295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257102+0000","last_change":"2023-11-27T15:23:27.192629+0000","last_active":"2023-11-27T15:24:40.257102+0000","last_peered":"2023-11-27T15:24:40.257102+0000","last_clean":"2023-11-27T15:24:40.257102+0000","last_became_active":"2023-11-27T15:23:27.192327+0000","last_became_peered":"2023-11-27T15:23:27.192327+0000","last_unstale":"2023-11-27T15:24:40.257102+0000","last_undegraded":"2023-11-27T15:24:40.257102+0000","last_fullsized":"2023-11-27T15:24:40.257102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:09:08.862576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.319918+0000","last_change":"2023-11-27T15:23:31.648296+0000","last_active":"2023-11-27T15:24:40.319918+0000","last_peered":"2023-11-27T15:24:40.319918+0000","last_clean":"2023-11-27T15:24:40.319918+0000","last_became_active":"2023-11-27T15:23:31.647756+0000","last_became_peered":"2023-11-27T15:23:31.647756+0000","last_unstale":"2023-11-27T15:24:40.319918+0000","last_undegraded":"2023-11-27T15:24:40.319918+0000","last_fullsized":"2023-11-27T15:24:40.319918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:44:42.109495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261456+0000","last_change":"2023-11-27T15:23:26.604061+0000","last_active":"2023-11-27T15:24:40.261456+0000","last_peered":"2023-11-27T15:24:40.261456+0000","last_clean":"2023-11-27T15:24:40.261456+0000","last_became_active":"2023-11-27T15:23:26.603725+0000","last_became_peered":"2023-11-27T15:23:26.603725+0000","last_unstale":"2023-11-27T15:24:40.261456+0000","last_undegraded":"2023-11-27T15:24:40.261456+0000","last_fullsized":"2023-11-27T15:24:40.261456+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:04:54.440045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256896+0000","last_change":"2023-11-27T15:23:27.189389+0000","last_active":"2023-11-27T15:24:40.256896+0000","last_peered":"2023-11-27T15:24:40.256896+0000","last_clean":"2023-11-27T15:24:40.256896+0000","last_became_active":"2023-11-27T15:23:27.189072+0000","last_became_peered":"2023-11-27T15:23:27.189072+0000","last_unstale":"2023-11-27T15:24:40.256896+0000","last_undegraded":"2023-11-27T15:24:40.256896+0000","last_fullsized":"2023-11-27T15:24:40.256896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:48:20.053941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260214+0000","last_change":"2023-11-27T15:23:31.639042+0000","last_active":"2023-11-27T15:24:40.260214+0000","last_peered":"2023-11-27T15:24:40.260214+0000","last_clean":"2023-11-27T15:24:40.260214+0000","last_became_active":"2023-11-27T15:23:31.638611+0000","last_became_peered":"2023-11-27T15:23:31.638611+0000","last_unstale":"2023-11-27T15:24:40.260214+0000","last_undegraded":"2023-11-27T15:24:40.260214+0000","last_fullsized":"2023-11-27T15:24:40.260214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:20:34.399106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:49.331281+0000","last_change":"2023-11-27T15:23:26.602979+0000","last_active":"2023-11-27T15:24:49.331281+0000","last_peered":"2023-11-27T15:24:49.331281+0000","last_clean":"2023-11-27T15:24:49.331281+0000","last_became_active":"2023-11-27T15:23:26.602681+0000","last_became_peered":"2023-11-27T15:23:26.602681+0000","last_unstale":"2023-11-27T15:24:49.331281+0000","last_undegraded":"2023-11-27T15:24:49.331281+0000","last_fullsized":"2023-11-27T15:24:49.331281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:43:47.264444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260526+0000","last_change":"2023-11-27T15:23:27.187294+0000","last_active":"2023-11-27T15:24:40.260526+0000","last_peered":"2023-11-27T15:24:40.260526+0000","last_clean":"2023-11-27T15:24:40.260526+0000","last_became_active":"2023-11-27T15:23:27.186445+0000","last_became_peered":"2023-11-27T15:23:27.186445+0000","last_unstale":"2023-11-27T15:24:40.260526+0000","last_undegraded":"2023-11-27T15:24:40.260526+0000","last_fullsized":"2023-11-27T15:24:40.260526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:22:27.537962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.274136+0000","last_change":"2023-11-27T15:24:40.274136+0000","last_active":"2023-11-27T15:24:40.274136+0000","last_peered":"2023-11-27T15:24:40.274136+0000","last_clean":"2023-11-27T15:24:40.274136+0000","last_became_active":"2023-11-27T15:24:40.273756+0000","last_became_peered":"2023-11-27T15:24:40.273756+0000","last_unstale":"2023-11-27T15:24:40.274136+0000","last_undegraded":"2023-11-27T15:24:40.274136+0000","last_fullsized":"2023-11-27T15:24:40.274136+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:05:43.435900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.315217+0000","last_change":"2023-11-27T15:23:26.602182+0000","last_active":"2023-11-27T15:24:40.315217+0000","last_peered":"2023-11-27T15:24:40.315217+0000","last_clean":"2023-11-27T15:24:40.315217+0000","last_became_active":"2023-11-27T15:23:26.601828+0000","last_became_peered":"2023-11-27T15:23:26.601828+0000","last_unstale":"2023-11-27T15:24:40.315217+0000","last_undegraded":"2023-11-27T15:24:40.315217+0000","last_fullsized":"2023-11-27T15:24:40.315217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:24:23.918955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256780+0000","last_change":"2023-11-27T15:23:27.204874+0000","last_active":"2023-11-27T15:24:40.256780+0000","last_peered":"2023-11-27T15:24:40.256780+0000","last_clean":"2023-11-27T15:24:40.256780+0000","last_became_active":"2023-11-27T15:23:27.204588+0000","last_became_peered":"2023-11-27T15:23:27.204588+0000","last_unstale":"2023-11-27T15:24:40.256780+0000","last_undegraded":"2023-11-27T15:24:40.256780+0000","last_fullsized":"2023-11-27T15:24:40.256780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:48:18.899473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.321364+0000","last_change":"2023-11-27T15:24:40.321364+0000","last_active":"2023-11-27T15:24:40.321364+0000","last_peered":"2023-11-27T15:24:40.321364+0000","last_clean":"2023-11-27T15:24:40.321364+0000","last_became_active":"2023-11-27T15:24:40.320919+0000","last_became_peered":"2023-11-27T15:24:40.320919+0000","last_unstale":"2023-11-27T15:24:40.321364+0000","last_undegraded":"2023-11-27T15:24:40.321364+0000","last_fullsized":"2023-11-27T15:24:40.321364+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:18:07.203277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,10,0],"acting":[11,4,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259416+0000","last_change":"2023-11-27T15:23:26.610208+0000","last_active":"2023-11-27T15:24:40.259416+0000","last_peered":"2023-11-27T15:24:40.259416+0000","last_clean":"2023-11-27T15:24:40.259416+0000","last_became_active":"2023-11-27T15:23:26.609922+0000","last_became_peered":"2023-11-27T15:23:26.609922+0000","last_unstale":"2023-11-27T15:24:40.259416+0000","last_undegraded":"2023-11-27T15:24:40.259416+0000","last_fullsized":"2023-11-27T15:24:40.259416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:58:45.501554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268646+0000","last_change":"2023-11-27T15:23:27.201533+0000","last_active":"2023-11-27T15:24:40.268646+0000","last_peered":"2023-11-27T15:24:40.268646+0000","last_clean":"2023-11-27T15:24:40.268646+0000","last_became_active":"2023-11-27T15:23:27.200516+0000","last_became_peered":"2023-11-27T15:23:27.200516+0000","last_unstale":"2023-11-27T15:24:40.268646+0000","last_undegraded":"2023-11-27T15:24:40.268646+0000","last_fullsized":"2023-11-27T15:24:40.268646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:41:48.851813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261964+0000","last_change":"2023-11-27T15:23:31.642858+0000","last_active":"2023-11-27T15:24:40.261964+0000","last_peered":"2023-11-27T15:24:40.261964+0000","last_clean":"2023-11-27T15:24:40.261964+0000","last_became_active":"2023-11-27T15:23:31.642367+0000","last_became_peered":"2023-11-27T15:23:31.642367+0000","last_unstale":"2023-11-27T15:24:40.261964+0000","last_undegraded":"2023-11-27T15:24:40.261964+0000","last_fullsized":"2023-11-27T15:24:40.261964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:52:35.281560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256044+0000","last_change":"2023-11-27T15:23:26.602865+0000","last_active":"2023-11-27T15:24:40.256044+0000","last_peered":"2023-11-27T15:24:40.256044+0000","last_clean":"2023-11-27T15:24:40.256044+0000","last_became_active":"2023-11-27T15:23:26.602591+0000","last_became_peered":"2023-11-27T15:23:26.602591+0000","last_unstale":"2023-11-27T15:24:40.256044+0000","last_undegraded":"2023-11-27T15:24:40.256044+0000","last_fullsized":"2023-11-27T15:24:40.256044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:26:20.362045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.263574+0000","last_change":"2023-11-27T15:23:27.186551+0000","last_active":"2023-11-27T15:24:40.263574+0000","last_peered":"2023-11-27T15:24:40.263574+0000","last_clean":"2023-11-27T15:24:40.263574+0000","last_became_active":"2023-11-27T15:23:27.185713+0000","last_became_peered":"2023-11-27T15:23:27.185713+0000","last_unstale":"2023-11-27T15:24:40.263574+0000","last_undegraded":"2023-11-27T15:24:40.263574+0000","last_fullsized":"2023-11-27T15:24:40.263574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:52:53.751342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268719+0000","last_change":"2023-11-27T15:23:31.641039+0000","last_active":"2023-11-27T15:24:40.268719+0000","last_peered":"2023-11-27T15:24:40.268719+0000","last_clean":"2023-11-27T15:24:40.268719+0000","last_became_active":"2023-11-27T15:23:31.639234+0000","last_became_peered":"2023-11-27T15:23:31.639234+0000","last_unstale":"2023-11-27T15:24:40.268719+0000","last_undegraded":"2023-11-27T15:24:40.268719+0000","last_fullsized":"2023-11-27T15:24:40.268719+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:06:56.397390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:48.328836+0000","last_change":"2023-11-27T15:23:26.606409+0000","last_active":"2023-11-27T15:24:48.328836+0000","last_peered":"2023-11-27T15:24:48.328836+0000","last_clean":"2023-11-27T15:24:48.328836+0000","last_became_active":"2023-11-27T15:23:26.606060+0000","last_became_peered":"2023-11-27T15:23:26.606060+0000","last_unstale":"2023-11-27T15:24:48.328836+0000","last_undegraded":"2023-11-27T15:24:48.328836+0000","last_fullsized":"2023-11-27T15:24:48.328836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:52:17.764842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256965+0000","last_change":"2023-11-27T15:23:27.180766+0000","last_active":"2023-11-27T15:24:40.256965+0000","last_peered":"2023-11-27T15:24:40.256965+0000","last_clean":"2023-11-27T15:24:40.256965+0000","last_became_active":"2023-11-27T15:23:27.180382+0000","last_became_peered":"2023-11-27T15:23:27.180382+0000","last_unstale":"2023-11-27T15:24:40.256965+0000","last_undegraded":"2023-11-27T15:24:40.256965+0000","last_fullsized":"2023-11-27T15:24:40.256965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:30:32.613171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261339+0000","last_change":"2023-11-27T15:23:31.641697+0000","last_active":"2023-11-27T15:24:40.261339+0000","last_peered":"2023-11-27T15:24:40.261339+0000","last_clean":"2023-11-27T15:24:40.261339+0000","last_became_active":"2023-11-27T15:23:31.641246+0000","last_became_peered":"2023-11-27T15:23:31.641246+0000","last_unstale":"2023-11-27T15:24:40.261339+0000","last_undegraded":"2023-11-27T15:24:40.261339+0000","last_fullsized":"2023-11-27T15:24:40.261339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:36.645857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391560+0000","last_change":"2023-11-27T15:23:26.605307+0000","last_active":"2023-11-27T15:24:40.391560+0000","last_peered":"2023-11-27T15:24:40.391560+0000","last_clean":"2023-11-27T15:24:40.391560+0000","last_became_active":"2023-11-27T15:23:26.605047+0000","last_became_peered":"2023-11-27T15:23:26.605047+0000","last_unstale":"2023-11-27T15:24:40.391560+0000","last_undegraded":"2023-11-27T15:24:40.391560+0000","last_fullsized":"2023-11-27T15:24:40.391560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:32:17.515689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.258453+0000","last_change":"2023-11-27T15:23:27.194469+0000","last_active":"2023-11-27T15:24:40.258453+0000","last_peered":"2023-11-27T15:24:40.258453+0000","last_clean":"2023-11-27T15:24:40.258453+0000","last_became_active":"2023-11-27T15:23:27.193930+0000","last_became_peered":"2023-11-27T15:23:27.193930+0000","last_unstale":"2023-11-27T15:24:40.258453+0000","last_undegraded":"2023-11-27T15:24:40.258453+0000","last_fullsized":"2023-11-27T15:24:40.258453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:59:04.654492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320418+0000","last_change":"2023-11-27T15:23:31.655787+0000","last_active":"2023-11-27T15:24:40.320418+0000","last_peered":"2023-11-27T15:24:40.320418+0000","last_clean":"2023-11-27T15:24:40.320418+0000","last_became_active":"2023-11-27T15:23:31.654847+0000","last_became_peered":"2023-11-27T15:23:31.654847+0000","last_unstale":"2023-11-27T15:24:40.320418+0000","last_undegraded":"2023-11-27T15:24:40.320418+0000","last_fullsized":"2023-11-27T15:24:40.320418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:12:47.906634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256696+0000","last_change":"2023-11-27T15:23:26.598669+0000","last_active":"2023-11-27T15:24:40.256696+0000","last_peered":"2023-11-27T15:24:40.256696+0000","last_clean":"2023-11-27T15:24:40.256696+0000","last_became_active":"2023-11-27T15:23:26.598224+0000","last_became_peered":"2023-11-27T15:23:26.598224+0000","last_unstale":"2023-11-27T15:24:40.256696+0000","last_undegraded":"2023-11-27T15:24:40.256696+0000","last_fullsized":"2023-11-27T15:24:40.256696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:26.869920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257264+0000","last_change":"2023-11-27T15:23:27.182800+0000","last_active":"2023-11-27T15:24:40.257264+0000","last_peered":"2023-11-27T15:24:40.257264+0000","last_clean":"2023-11-27T15:24:40.257264+0000","last_became_active":"2023-11-27T15:23:27.182572+0000","last_became_peered":"2023-11-27T15:23:27.182572+0000","last_unstale":"2023-11-27T15:24:40.257264+0000","last_undegraded":"2023-11-27T15:24:40.257264+0000","last_fullsized":"2023-11-27T15:24:40.257264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:23:23.268840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.265208+0000","last_change":"2023-11-27T15:23:31.645586+0000","last_active":"2023-11-27T15:24:40.265208+0000","last_peered":"2023-11-27T15:24:40.265208+0000","last_clean":"2023-11-27T15:24:40.265208+0000","last_became_active":"2023-11-27T15:23:31.644897+0000","last_became_peered":"2023-11-27T15:23:31.644897+0000","last_unstale":"2023-11-27T15:24:40.265208+0000","last_undegraded":"2023-11-27T15:24:40.265208+0000","last_fullsized":"2023-11-27T15:24:40.265208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:30:32.319019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":170,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:49.330486+0000","last_change":"2023-11-27T15:23:26.601652+0000","last_active":"2023-11-27T15:24:49.330486+0000","last_peered":"2023-11-27T15:24:49.330486+0000","last_clean":"2023-11-27T15:24:49.330486+0000","last_became_active":"2023-11-27T15:23:26.601306+0000","last_became_peered":"2023-11-27T15:23:26.601306+0000","last_unstale":"2023-11-27T15:24:49.330486+0000","last_undegraded":"2023-11-27T15:24:49.330486+0000","last_fullsized":"2023-11-27T15:24:49.330486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:30:17.858253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61422,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":112,"num_read_kb":116,"num_write":33,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260948+0000","last_change":"2023-11-27T15:23:27.186709+0000","last_active":"2023-11-27T15:24:40.260948+0000","last_peered":"2023-11-27T15:24:40.260948+0000","last_clean":"2023-11-27T15:24:40.260948+0000","last_became_active":"2023-11-27T15:23:27.186390+0000","last_became_peered":"2023-11-27T15:23:27.186390+0000","last_unstale":"2023-11-27T15:24:40.260948+0000","last_undegraded":"2023-11-27T15:24:40.260948+0000","last_fullsized":"2023-11-27T15:24:40.260948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:06:36.283058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.257350+0000","last_change":"2023-11-27T15:23:31.651851+0000","last_active":"2023-11-27T15:24:40.257350+0000","last_peered":"2023-11-27T15:24:40.257350+0000","last_clean":"2023-11-27T15:24:40.257350+0000","last_became_active":"2023-11-27T15:23:31.651409+0000","last_became_peered":"2023-11-27T15:23:31.651409+0000","last_unstale":"2023-11-27T15:24:40.257350+0000","last_undegraded":"2023-11-27T15:24:40.257350+0000","last_fullsized":"2023-11-27T15:24:40.257350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:04:17.235577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259937+0000","last_change":"2023-11-27T15:23:26.602719+0000","last_active":"2023-11-27T15:24:40.259937+0000","last_peered":"2023-11-27T15:24:40.259937+0000","last_clean":"2023-11-27T15:24:40.259937+0000","last_became_active":"2023-11-27T15:23:26.602372+0000","last_became_peered":"2023-11-27T15:23:26.602372+0000","last_unstale":"2023-11-27T15:24:40.259937+0000","last_undegraded":"2023-11-27T15:24:40.259937+0000","last_fullsized":"2023-11-27T15:24:40.259937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:35:43.136903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.263099+0000","last_change":"2023-11-27T15:23:27.200479+0000","last_active":"2023-11-27T15:24:40.263099+0000","last_peered":"2023-11-27T15:24:40.263099+0000","last_clean":"2023-11-27T15:24:40.263099+0000","last_became_active":"2023-11-27T15:23:27.200155+0000","last_became_peered":"2023-11-27T15:23:27.200155+0000","last_unstale":"2023-11-27T15:24:40.263099+0000","last_undegraded":"2023-11-27T15:24:40.263099+0000","last_fullsized":"2023-11-27T15:24:40.263099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:57:03.091095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.392376+0000","last_change":"2023-11-27T15:24:40.392376+0000","last_active":"2023-11-27T15:24:40.392376+0000","last_peered":"2023-11-27T15:24:40.392376+0000","last_clean":"2023-11-27T15:24:40.392376+0000","last_became_active":"2023-11-27T15:24:40.391782+0000","last_became_peered":"2023-11-27T15:24:40.391782+0000","last_unstale":"2023-11-27T15:24:40.392376+0000","last_undegraded":"2023-11-27T15:24:40.392376+0000","last_fullsized":"2023-11-27T15:24:40.392376+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:12:56.963372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256480+0000","last_change":"2023-11-27T15:23:26.605927+0000","last_active":"2023-11-27T15:24:40.256480+0000","last_peered":"2023-11-27T15:24:40.256480+0000","last_clean":"2023-11-27T15:24:40.256480+0000","last_became_active":"2023-11-27T15:23:26.605515+0000","last_became_peered":"2023-11-27T15:23:26.605515+0000","last_unstale":"2023-11-27T15:24:40.256480+0000","last_undegraded":"2023-11-27T15:24:40.256480+0000","last_fullsized":"2023-11-27T15:24:40.256480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:22:08.421561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391225+0000","last_change":"2023-11-27T15:23:27.200237+0000","last_active":"2023-11-27T15:24:40.391225+0000","last_peered":"2023-11-27T15:24:40.391225+0000","last_clean":"2023-11-27T15:24:40.391225+0000","last_became_active":"2023-11-27T15:23:27.199952+0000","last_became_peered":"2023-11-27T15:23:27.199952+0000","last_unstale":"2023-11-27T15:24:40.391225+0000","last_undegraded":"2023-11-27T15:24:40.391225+0000","last_fullsized":"2023-11-27T15:24:40.391225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:47:19.672350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.264061+0000","last_change":"2023-11-27T15:23:31.650718+0000","last_active":"2023-11-27T15:24:40.264061+0000","last_peered":"2023-11-27T15:24:40.264061+0000","last_clean":"2023-11-27T15:24:40.264061+0000","last_became_active":"2023-11-27T15:23:31.650144+0000","last_became_peered":"2023-11-27T15:23:31.650144+0000","last_unstale":"2023-11-27T15:24:40.264061+0000","last_undegraded":"2023-11-27T15:24:40.264061+0000","last_fullsized":"2023-11-27T15:24:40.264061+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:18:18.747396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'4","reported_seq":85,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:49.336290+0000","last_change":"2023-11-27T15:23:26.600962+0000","last_active":"2023-11-27T15:24:49.336290+0000","last_peered":"2023-11-27T15:24:49.336290+0000","last_clean":"2023-11-27T15:24:49.336290+0000","last_became_active":"2023-11-27T15:23:26.600628+0000","last_became_peered":"2023-11-27T15:23:26.600628+0000","last_unstale":"2023-11-27T15:24:49.336290+0000","last_undegraded":"2023-11-27T15:24:49.336290+0000","last_fullsized":"2023-11-27T15:24:49.336290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:27:28.017653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.390863+0000","last_change":"2023-11-27T15:23:27.191939+0000","last_active":"2023-11-27T15:24:40.390863+0000","last_peered":"2023-11-27T15:24:40.390863+0000","last_clean":"2023-11-27T15:24:40.390863+0000","last_became_active":"2023-11-27T15:23:27.191634+0000","last_became_peered":"2023-11-27T15:23:27.191634+0000","last_unstale":"2023-11-27T15:24:40.390863+0000","last_undegraded":"2023-11-27T15:24:40.390863+0000","last_fullsized":"2023-11-27T15:24:40.390863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:08:48.214302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.263454+0000","last_change":"2023-11-27T15:23:31.650733+0000","last_active":"2023-11-27T15:24:40.263454+0000","last_peered":"2023-11-27T15:24:40.263454+0000","last_clean":"2023-11-27T15:24:40.263454+0000","last_became_active":"2023-11-27T15:23:31.650280+0000","last_became_peered":"2023-11-27T15:23:31.650280+0000","last_unstale":"2023-11-27T15:24:40.263454+0000","last_undegraded":"2023-11-27T15:24:40.263454+0000","last_fullsized":"2023-11-27T15:24:40.263454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:01:03.360168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.304963+0000","last_change":"2023-11-27T15:24:40.304562+0000","last_active":"2023-11-27T15:24:40.304963+0000","last_peered":"2023-11-27T15:24:40.304963+0000","last_clean":"2023-11-27T15:24:40.304963+0000","last_became_active":"2023-11-27T15:24:40.271620+0000","last_became_peered":"2023-11-27T15:24:40.271620+0000","last_unstale":"2023-11-27T15:24:40.304963+0000","last_undegraded":"2023-11-27T15:24:40.304963+0000","last_fullsized":"2023-11-27T15:24:40.304963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:32:19.491473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.321309+0000","last_change":"2023-11-27T15:24:40.320722+0000","last_active":"2023-11-27T15:24:40.321309+0000","last_peered":"2023-11-27T15:24:40.321309+0000","last_clean":"2023-11-27T15:24:40.321309+0000","last_became_active":"2023-11-27T15:24:40.320318+0000","last_became_peered":"2023-11-27T15:24:40.320318+0000","last_unstale":"2023-11-27T15:24:40.321309+0000","last_undegraded":"2023-11-27T15:24:40.321309+0000","last_fullsized":"2023-11-27T15:24:40.321309+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:16:15.476545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256313+0000","last_change":"2023-11-27T15:23:31.644073+0000","last_active":"2023-11-27T15:24:40.256313+0000","last_peered":"2023-11-27T15:24:40.256313+0000","last_clean":"2023-11-27T15:24:40.256313+0000","last_became_active":"2023-11-27T15:23:31.643689+0000","last_became_peered":"2023-11-27T15:23:31.643689+0000","last_unstale":"2023-11-27T15:24:40.256313+0000","last_undegraded":"2023-11-27T15:24:40.256313+0000","last_fullsized":"2023-11-27T15:24:40.256313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:19:47.142612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.262954+0000","last_change":"2023-11-27T15:23:26.604034+0000","last_active":"2023-11-27T15:24:40.262954+0000","last_peered":"2023-11-27T15:24:40.262954+0000","last_clean":"2023-11-27T15:24:40.262954+0000","last_became_active":"2023-11-27T15:23:26.603689+0000","last_became_peered":"2023-11-27T15:23:26.603689+0000","last_unstale":"2023-11-27T15:24:40.262954+0000","last_undegraded":"2023-11-27T15:24:40.262954+0000","last_fullsized":"2023-11-27T15:24:40.262954+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:51:11.895213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260637+0000","last_change":"2023-11-27T15:23:27.192723+0000","last_active":"2023-11-27T15:24:40.260637+0000","last_peered":"2023-11-27T15:24:40.260637+0000","last_clean":"2023-11-27T15:24:40.260637+0000","last_became_active":"2023-11-27T15:23:27.192034+0000","last_became_peered":"2023-11-27T15:23:27.192034+0000","last_unstale":"2023-11-27T15:24:40.260637+0000","last_undegraded":"2023-11-27T15:24:40.260637+0000","last_fullsized":"2023-11-27T15:24:40.260637+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:57:14.566261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260461+0000","last_change":"2023-11-27T15:23:31.652756+0000","last_active":"2023-11-27T15:24:40.260461+0000","last_peered":"2023-11-27T15:24:40.260461+0000","last_clean":"2023-11-27T15:24:40.260461+0000","last_became_active":"2023-11-27T15:23:31.652280+0000","last_became_peered":"2023-11-27T15:23:31.652280+0000","last_unstale":"2023-11-27T15:24:40.260461+0000","last_undegraded":"2023-11-27T15:24:40.260461+0000","last_fullsized":"2023-11-27T15:24:40.260461+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:40:53.743334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.264946+0000","last_change":"2023-11-27T15:23:26.609927+0000","last_active":"2023-11-27T15:24:40.264946+0000","last_peered":"2023-11-27T15:24:40.264946+0000","last_clean":"2023-11-27T15:24:40.264946+0000","last_became_active":"2023-11-27T15:23:26.608928+0000","last_became_peered":"2023-11-27T15:23:26.608928+0000","last_unstale":"2023-11-27T15:24:40.264946+0000","last_undegraded":"2023-11-27T15:24:40.264946+0000","last_fullsized":"2023-11-27T15:24:40.264946+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:23:24.296172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.315649+0000","last_change":"2023-11-27T15:23:27.204619+0000","last_active":"2023-11-27T15:24:40.315649+0000","last_peered":"2023-11-27T15:24:40.315649+0000","last_clean":"2023-11-27T15:24:40.315649+0000","last_became_active":"2023-11-27T15:23:27.204298+0000","last_became_peered":"2023-11-27T15:23:27.204298+0000","last_unstale":"2023-11-27T15:24:40.315649+0000","last_undegraded":"2023-11-27T15:24:40.315649+0000","last_fullsized":"2023-11-27T15:24:40.315649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:22:34.577936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261403+0000","last_change":"2023-11-27T15:23:31.644763+0000","last_active":"2023-11-27T15:24:40.261403+0000","last_peered":"2023-11-27T15:24:40.261403+0000","last_clean":"2023-11-27T15:24:40.261403+0000","last_became_active":"2023-11-27T15:23:31.644106+0000","last_became_peered":"2023-11-27T15:23:31.644106+0000","last_unstale":"2023-11-27T15:24:40.261403+0000","last_undegraded":"2023-11-27T15:24:40.261403+0000","last_fullsized":"2023-11-27T15:24:40.261403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:05:41.326786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391024+0000","last_change":"2023-11-27T15:23:26.614703+0000","last_active":"2023-11-27T15:24:40.391024+0000","last_peered":"2023-11-27T15:24:40.391024+0000","last_clean":"2023-11-27T15:24:40.391024+0000","last_became_active":"2023-11-27T15:23:26.614324+0000","last_became_peered":"2023-11-27T15:23:26.614324+0000","last_unstale":"2023-11-27T15:24:40.391024+0000","last_undegraded":"2023-11-27T15:24:40.391024+0000","last_fullsized":"2023-11-27T15:24:40.391024+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:56:54.015930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.315249+0000","last_change":"2023-11-27T15:23:27.197288+0000","last_active":"2023-11-27T15:24:40.315249+0000","last_peered":"2023-11-27T15:24:40.315249+0000","last_clean":"2023-11-27T15:24:40.315249+0000","last_became_active":"2023-11-27T15:23:27.196925+0000","last_became_peered":"2023-11-27T15:23:27.196925+0000","last_unstale":"2023-11-27T15:24:40.315249+0000","last_undegraded":"2023-11-27T15:24:40.315249+0000","last_fullsized":"2023-11-27T15:24:40.315249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:40:28.777876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.264756+0000","last_change":"2023-11-27T15:23:31.652701+0000","last_active":"2023-11-27T15:24:40.264756+0000","last_peered":"2023-11-27T15:24:40.264756+0000","last_clean":"2023-11-27T15:24:40.264756+0000","last_became_active":"2023-11-27T15:23:31.652259+0000","last_became_peered":"2023-11-27T15:23:31.652259+0000","last_unstale":"2023-11-27T15:24:40.264756+0000","last_undegraded":"2023-11-27T15:24:40.264756+0000","last_fullsized":"2023-11-27T15:24:40.264756+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:14:09.426627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256267+0000","last_change":"2023-11-27T15:23:27.190103+0000","last_active":"2023-11-27T15:24:40.256267+0000","last_peered":"2023-11-27T15:24:40.256267+0000","last_clean":"2023-11-27T15:24:40.256267+0000","last_became_active":"2023-11-27T15:23:27.189803+0000","last_became_peered":"2023-11-27T15:23:27.189803+0000","last_unstale":"2023-11-27T15:24:40.256267+0000","last_undegraded":"2023-11-27T15:24:40.256267+0000","last_fullsized":"2023-11-27T15:24:40.256267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:55:45.581447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":85,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:52.340499+0000","last_change":"2023-11-27T15:23:26.606338+0000","last_active":"2023-11-27T15:24:52.340499+0000","last_peered":"2023-11-27T15:24:52.340499+0000","last_clean":"2023-11-27T15:24:52.340499+0000","last_became_active":"2023-11-27T15:23:26.601152+0000","last_became_peered":"2023-11-27T15:23:26.601152+0000","last_unstale":"2023-11-27T15:24:52.340499+0000","last_undegraded":"2023-11-27T15:24:52.340499+0000","last_fullsized":"2023-11-27T15:24:52.340499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:03:04.486749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391435+0000","last_change":"2023-11-27T15:23:31.646457+0000","last_active":"2023-11-27T15:24:40.391435+0000","last_peered":"2023-11-27T15:24:40.391435+0000","last_clean":"2023-11-27T15:24:40.391435+0000","last_became_active":"2023-11-27T15:23:31.646071+0000","last_became_peered":"2023-11-27T15:23:31.646071+0000","last_unstale":"2023-11-27T15:24:40.391435+0000","last_undegraded":"2023-11-27T15:24:40.391435+0000","last_fullsized":"2023-11-27T15:24:40.391435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:04:43.604907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268444+0000","last_change":"2023-11-27T15:23:27.201405+0000","last_active":"2023-11-27T15:24:40.268444+0000","last_peered":"2023-11-27T15:24:40.268444+0000","last_clean":"2023-11-27T15:24:40.268444+0000","last_became_active":"2023-11-27T15:23:27.200951+0000","last_became_peered":"2023-11-27T15:23:27.200951+0000","last_unstale":"2023-11-27T15:24:40.268444+0000","last_undegraded":"2023-11-27T15:24:40.268444+0000","last_fullsized":"2023-11-27T15:24:40.268444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:19:39.249483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320203+0000","last_change":"2023-11-27T15:23:26.604328+0000","last_active":"2023-11-27T15:24:40.320203+0000","last_peered":"2023-11-27T15:24:40.320203+0000","last_clean":"2023-11-27T15:24:40.320203+0000","last_became_active":"2023-11-27T15:23:26.603937+0000","last_became_peered":"2023-11-27T15:23:26.603937+0000","last_unstale":"2023-11-27T15:24:40.320203+0000","last_undegraded":"2023-11-27T15:24:40.320203+0000","last_fullsized":"2023-11-27T15:24:40.320203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:03:23.908983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.390673+0000","last_change":"2023-11-27T15:22:17.949235+0000","last_active":"2023-11-27T15:24:40.390673+0000","last_peered":"2023-11-27T15:24:40.390673+0000","last_clean":"2023-11-27T15:24:40.390673+0000","last_became_active":"2023-11-27T15:22:17.948822+0000","last_became_peered":"2023-11-27T15:22:17.948822+0000","last_unstale":"2023-11-27T15:24:40.390673+0000","last_undegraded":"2023-11-27T15:24:40.390673+0000","last_fullsized":"2023-11-27T15:24:40.390673+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:19:02.909930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:19:02.909930+0000","last_clean_scrub_stamp":"2023-11-27T15:19:02.909930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:34:29.263598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256828+0000","last_change":"2023-11-27T15:23:31.654325+0000","last_active":"2023-11-27T15:24:40.256828+0000","last_peered":"2023-11-27T15:24:40.256828+0000","last_clean":"2023-11-27T15:24:40.256828+0000","last_became_active":"2023-11-27T15:23:31.653857+0000","last_became_peered":"2023-11-27T15:23:31.653857+0000","last_unstale":"2023-11-27T15:24:40.256828+0000","last_undegraded":"2023-11-27T15:24:40.256828+0000","last_fullsized":"2023-11-27T15:24:40.256828+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:00:23.349266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259242+0000","last_change":"2023-11-27T15:23:27.192623+0000","last_active":"2023-11-27T15:24:40.259242+0000","last_peered":"2023-11-27T15:24:40.259242+0000","last_clean":"2023-11-27T15:24:40.259242+0000","last_became_active":"2023-11-27T15:23:27.192266+0000","last_became_peered":"2023-11-27T15:23:27.192266+0000","last_unstale":"2023-11-27T15:24:40.259242+0000","last_undegraded":"2023-11-27T15:24:40.259242+0000","last_fullsized":"2023-11-27T15:24:40.259242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:30:40.703677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.390884+0000","last_change":"2023-11-27T15:23:26.603078+0000","last_active":"2023-11-27T15:24:40.390884+0000","last_peered":"2023-11-27T15:24:40.390884+0000","last_clean":"2023-11-27T15:24:40.390884+0000","last_became_active":"2023-11-27T15:23:26.602758+0000","last_became_peered":"2023-11-27T15:23:26.602758+0000","last_unstale":"2023-11-27T15:24:40.390884+0000","last_undegraded":"2023-11-27T15:24:40.390884+0000","last_fullsized":"2023-11-27T15:24:40.390884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:40:23.370790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260441+0000","last_change":"2023-11-27T15:23:31.648266+0000","last_active":"2023-11-27T15:24:40.260441+0000","last_peered":"2023-11-27T15:24:40.260441+0000","last_clean":"2023-11-27T15:24:40.260441+0000","last_became_active":"2023-11-27T15:23:31.647804+0000","last_became_peered":"2023-11-27T15:23:31.647804+0000","last_unstale":"2023-11-27T15:24:40.260441+0000","last_undegraded":"2023-11-27T15:24:40.260441+0000","last_fullsized":"2023-11-27T15:24:40.260441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:02:23.849165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261060+0000","last_change":"2023-11-27T15:23:26.615577+0000","last_active":"2023-11-27T15:24:40.261060+0000","last_peered":"2023-11-27T15:24:40.261060+0000","last_clean":"2023-11-27T15:24:40.261060+0000","last_became_active":"2023-11-27T15:23:26.615287+0000","last_became_peered":"2023-11-27T15:23:26.615287+0000","last_unstale":"2023-11-27T15:24:40.261060+0000","last_undegraded":"2023-11-27T15:24:40.261060+0000","last_fullsized":"2023-11-27T15:24:40.261060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:15:31.547687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261583+0000","last_change":"2023-11-27T15:23:27.203162+0000","last_active":"2023-11-27T15:24:40.261583+0000","last_peered":"2023-11-27T15:24:40.261583+0000","last_clean":"2023-11-27T15:24:40.261583+0000","last_became_active":"2023-11-27T15:23:27.201725+0000","last_became_peered":"2023-11-27T15:23:27.201725+0000","last_unstale":"2023-11-27T15:24:40.261583+0000","last_undegraded":"2023-11-27T15:24:40.261583+0000","last_fullsized":"2023-11-27T15:24:40.261583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:51:35.159456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268487+0000","last_change":"2023-11-27T15:23:31.653253+0000","last_active":"2023-11-27T15:24:40.268487+0000","last_peered":"2023-11-27T15:24:40.268487+0000","last_clean":"2023-11-27T15:24:40.268487+0000","last_became_active":"2023-11-27T15:23:31.652777+0000","last_became_peered":"2023-11-27T15:23:31.652777+0000","last_unstale":"2023-11-27T15:24:40.268487+0000","last_undegraded":"2023-11-27T15:24:40.268487+0000","last_fullsized":"2023-11-27T15:24:40.268487+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:50:59.858365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.264228+0000","last_change":"2023-11-27T15:23:26.610599+0000","last_active":"2023-11-27T15:24:40.264228+0000","last_peered":"2023-11-27T15:24:40.264228+0000","last_clean":"2023-11-27T15:24:40.264228+0000","last_became_active":"2023-11-27T15:23:26.610205+0000","last_became_peered":"2023-11-27T15:23:26.610205+0000","last_unstale":"2023-11-27T15:24:40.264228+0000","last_undegraded":"2023-11-27T15:24:40.264228+0000","last_fullsized":"2023-11-27T15:24:40.264228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:33:15.342015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.269543+0000","last_change":"2023-11-27T15:23:27.203367+0000","last_active":"2023-11-27T15:24:40.269543+0000","last_peered":"2023-11-27T15:24:40.269543+0000","last_clean":"2023-11-27T15:24:40.269543+0000","last_became_active":"2023-11-27T15:23:27.203054+0000","last_became_peered":"2023-11-27T15:23:27.203054+0000","last_unstale":"2023-11-27T15:24:40.269543+0000","last_undegraded":"2023-11-27T15:24:40.269543+0000","last_fullsized":"2023-11-27T15:24:40.269543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:11:17.106288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.258899+0000","last_change":"2023-11-27T15:23:31.638372+0000","last_active":"2023-11-27T15:24:40.258899+0000","last_peered":"2023-11-27T15:24:40.258899+0000","last_clean":"2023-11-27T15:24:40.258899+0000","last_became_active":"2023-11-27T15:23:31.637973+0000","last_became_peered":"2023-11-27T15:23:31.637973+0000","last_unstale":"2023-11-27T15:24:40.258899+0000","last_undegraded":"2023-11-27T15:24:40.258899+0000","last_fullsized":"2023-11-27T15:24:40.258899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:03:04.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.321322+0000","last_change":"2023-11-27T15:23:26.594218+0000","last_active":"2023-11-27T15:24:40.321322+0000","last_peered":"2023-11-27T15:24:40.321322+0000","last_clean":"2023-11-27T15:24:40.321322+0000","last_became_active":"2023-11-27T15:23:26.593842+0000","last_became_peered":"2023-11-27T15:23:26.593842+0000","last_unstale":"2023-11-27T15:24:40.321322+0000","last_undegraded":"2023-11-27T15:24:40.321322+0000","last_fullsized":"2023-11-27T15:24:40.321322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:12:35.988583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.262114+0000","last_change":"2023-11-27T15:23:27.211258+0000","last_active":"2023-11-27T15:24:40.262114+0000","last_peered":"2023-11-27T15:24:40.262114+0000","last_clean":"2023-11-27T15:24:40.262114+0000","last_became_active":"2023-11-27T15:23:27.210963+0000","last_became_peered":"2023-11-27T15:23:27.210963+0000","last_unstale":"2023-11-27T15:24:40.262114+0000","last_undegraded":"2023-11-27T15:24:40.262114+0000","last_fullsized":"2023-11-27T15:24:40.262114+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:46:57.760614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320998+0000","last_change":"2023-11-27T15:23:31.652938+0000","last_active":"2023-11-27T15:24:40.320998+0000","last_peered":"2023-11-27T15:24:40.320998+0000","last_clean":"2023-11-27T15:24:40.320998+0000","last_became_active":"2023-11-27T15:23:31.652504+0000","last_became_peered":"2023-11-27T15:23:31.652504+0000","last_unstale":"2023-11-27T15:24:40.320998+0000","last_undegraded":"2023-11-27T15:24:40.320998+0000","last_fullsized":"2023-11-27T15:24:40.320998+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:30:17.868238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256285+0000","last_change":"2023-11-27T15:23:26.602328+0000","last_active":"2023-11-27T15:24:40.256285+0000","last_peered":"2023-11-27T15:24:40.256285+0000","last_clean":"2023-11-27T15:24:40.256285+0000","last_became_active":"2023-11-27T15:23:26.601974+0000","last_became_peered":"2023-11-27T15:23:26.601974+0000","last_unstale":"2023-11-27T15:24:40.256285+0000","last_undegraded":"2023-11-27T15:24:40.256285+0000","last_fullsized":"2023-11-27T15:24:40.256285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:25:40.773772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260086+0000","last_change":"2023-11-27T15:23:27.194577+0000","last_active":"2023-11-27T15:24:40.260086+0000","last_peered":"2023-11-27T15:24:40.260086+0000","last_clean":"2023-11-27T15:24:40.260086+0000","last_became_active":"2023-11-27T15:23:27.194186+0000","last_became_peered":"2023-11-27T15:23:27.194186+0000","last_unstale":"2023-11-27T15:24:40.260086+0000","last_undegraded":"2023-11-27T15:24:40.260086+0000","last_fullsized":"2023-11-27T15:24:40.260086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:08:52.457498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.273069+0000","last_change":"2023-11-27T15:24:40.272361+0000","last_active":"2023-11-27T15:24:40.273069+0000","last_peered":"2023-11-27T15:24:40.273069+0000","last_clean":"2023-11-27T15:24:40.273069+0000","last_became_active":"2023-11-27T15:24:40.271779+0000","last_became_peered":"2023-11-27T15:24:40.271779+0000","last_unstale":"2023-11-27T15:24:40.273069+0000","last_undegraded":"2023-11-27T15:24:40.273069+0000","last_fullsized":"2023-11-27T15:24:40.273069+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:45:31.792127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.268872+0000","last_change":"2023-11-27T15:23:26.611336+0000","last_active":"2023-11-27T15:24:40.268872+0000","last_peered":"2023-11-27T15:24:40.268872+0000","last_clean":"2023-11-27T15:24:40.268872+0000","last_became_active":"2023-11-27T15:23:26.610819+0000","last_became_peered":"2023-11-27T15:23:26.610819+0000","last_unstale":"2023-11-27T15:24:40.268872+0000","last_undegraded":"2023-11-27T15:24:40.268872+0000","last_fullsized":"2023-11-27T15:24:40.268872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:56:02.723345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259915+0000","last_change":"2023-11-27T15:23:27.198336+0000","last_active":"2023-11-27T15:24:40.259915+0000","last_peered":"2023-11-27T15:24:40.259915+0000","last_clean":"2023-11-27T15:24:40.259915+0000","last_became_active":"2023-11-27T15:23:27.198062+0000","last_became_peered":"2023-11-27T15:23:27.198062+0000","last_unstale":"2023-11-27T15:24:40.259915+0000","last_undegraded":"2023-11-27T15:24:40.259915+0000","last_fullsized":"2023-11-27T15:24:40.259915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:48:59.593682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.320576+0000","last_change":"2023-11-27T15:23:31.655526+0000","last_active":"2023-11-27T15:24:40.320576+0000","last_peered":"2023-11-27T15:24:40.320576+0000","last_clean":"2023-11-27T15:24:40.320576+0000","last_became_active":"2023-11-27T15:23:31.655011+0000","last_became_peered":"2023-11-27T15:23:31.655011+0000","last_unstale":"2023-11-27T15:24:40.320576+0000","last_undegraded":"2023-11-27T15:24:40.320576+0000","last_fullsized":"2023-11-27T15:24:40.320576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:51:20.671016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259314+0000","last_change":"2023-11-27T15:23:26.604536+0000","last_active":"2023-11-27T15:24:40.259314+0000","last_peered":"2023-11-27T15:24:40.259314+0000","last_clean":"2023-11-27T15:24:40.259314+0000","last_became_active":"2023-11-27T15:23:26.604239+0000","last_became_peered":"2023-11-27T15:23:26.604239+0000","last_unstale":"2023-11-27T15:24:40.259314+0000","last_undegraded":"2023-11-27T15:24:40.259314+0000","last_fullsized":"2023-11-27T15:24:40.259314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:27:04.229005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.269527+0000","last_change":"2023-11-27T15:23:27.201159+0000","last_active":"2023-11-27T15:24:40.269527+0000","last_peered":"2023-11-27T15:24:40.269527+0000","last_clean":"2023-11-27T15:24:40.269527+0000","last_became_active":"2023-11-27T15:23:27.200800+0000","last_became_peered":"2023-11-27T15:23:27.200800+0000","last_unstale":"2023-11-27T15:24:40.269527+0000","last_undegraded":"2023-11-27T15:24:40.269527+0000","last_fullsized":"2023-11-27T15:24:40.269527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:30:05.805286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.321029+0000","last_change":"2023-11-27T15:23:31.652835+0000","last_active":"2023-11-27T15:24:40.321029+0000","last_peered":"2023-11-27T15:24:40.321029+0000","last_clean":"2023-11-27T15:24:40.321029+0000","last_became_active":"2023-11-27T15:23:31.652425+0000","last_became_peered":"2023-11-27T15:23:31.652425+0000","last_unstale":"2023-11-27T15:24:40.321029+0000","last_undegraded":"2023-11-27T15:24:40.321029+0000","last_fullsized":"2023-11-27T15:24:40.321029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:58:38.875484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260732+0000","last_change":"2023-11-27T15:23:26.606303+0000","last_active":"2023-11-27T15:24:40.260732+0000","last_peered":"2023-11-27T15:24:40.260732+0000","last_clean":"2023-11-27T15:24:40.260732+0000","last_became_active":"2023-11-27T15:23:26.605827+0000","last_became_peered":"2023-11-27T15:23:26.605827+0000","last_unstale":"2023-11-27T15:24:40.260732+0000","last_undegraded":"2023-11-27T15:24:40.260732+0000","last_fullsized":"2023-11-27T15:24:40.260732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:15:07.123396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391060+0000","last_change":"2023-11-27T15:23:27.198422+0000","last_active":"2023-11-27T15:24:40.391060+0000","last_peered":"2023-11-27T15:24:40.391060+0000","last_clean":"2023-11-27T15:24:40.391060+0000","last_became_active":"2023-11-27T15:23:27.198116+0000","last_became_peered":"2023-11-27T15:23:27.198116+0000","last_unstale":"2023-11-27T15:24:40.391060+0000","last_undegraded":"2023-11-27T15:24:40.391060+0000","last_fullsized":"2023-11-27T15:24:40.391060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:52:18.469771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256291+0000","last_change":"2023-11-27T15:23:31.647798+0000","last_active":"2023-11-27T15:24:40.256291+0000","last_peered":"2023-11-27T15:24:40.256291+0000","last_clean":"2023-11-27T15:24:40.256291+0000","last_became_active":"2023-11-27T15:23:31.647087+0000","last_became_peered":"2023-11-27T15:23:31.647087+0000","last_unstale":"2023-11-27T15:24:40.256291+0000","last_undegraded":"2023-11-27T15:24:40.256291+0000","last_fullsized":"2023-11-27T15:24:40.256291+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:17:43.516721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.273308+0000","last_change":"2023-11-27T15:24:40.272888+0000","last_active":"2023-11-27T15:24:40.273308+0000","last_peered":"2023-11-27T15:24:40.273308+0000","last_clean":"2023-11-27T15:24:40.273308+0000","last_became_active":"2023-11-27T15:24:40.264563+0000","last_became_peered":"2023-11-27T15:24:40.264563+0000","last_unstale":"2023-11-27T15:24:40.273308+0000","last_undegraded":"2023-11-27T15:24:40.273308+0000","last_fullsized":"2023-11-27T15:24:40.273308+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:55:28.056798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391666+0000","last_change":"2023-11-27T15:23:27.186270+0000","last_active":"2023-11-27T15:24:40.391666+0000","last_peered":"2023-11-27T15:24:40.391666+0000","last_clean":"2023-11-27T15:24:40.391666+0000","last_became_active":"2023-11-27T15:23:27.185952+0000","last_became_peered":"2023-11-27T15:23:27.185952+0000","last_unstale":"2023-11-27T15:24:40.391666+0000","last_undegraded":"2023-11-27T15:24:40.391666+0000","last_fullsized":"2023-11-27T15:24:40.391666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:38:12.220989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.258975+0000","last_change":"2023-11-27T15:23:31.648666+0000","last_active":"2023-11-27T15:24:40.258975+0000","last_peered":"2023-11-27T15:24:40.258975+0000","last_clean":"2023-11-27T15:24:40.258975+0000","last_became_active":"2023-11-27T15:23:31.647245+0000","last_became_peered":"2023-11-27T15:23:31.647245+0000","last_unstale":"2023-11-27T15:24:40.258975+0000","last_undegraded":"2023-11-27T15:24:40.258975+0000","last_fullsized":"2023-11-27T15:24:40.258975+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:33:56.207371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256961+0000","last_change":"2023-11-27T15:23:26.606886+0000","last_active":"2023-11-27T15:24:40.256961+0000","last_peered":"2023-11-27T15:24:40.256961+0000","last_clean":"2023-11-27T15:24:40.256961+0000","last_became_active":"2023-11-27T15:23:26.606556+0000","last_became_peered":"2023-11-27T15:23:26.606556+0000","last_unstale":"2023-11-27T15:24:40.256961+0000","last_undegraded":"2023-11-27T15:24:40.256961+0000","last_fullsized":"2023-11-27T15:24:40.256961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:45:16.168185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391373+0000","last_change":"2023-11-27T15:23:27.194532+0000","last_active":"2023-11-27T15:24:40.391373+0000","last_peered":"2023-11-27T15:24:40.391373+0000","last_clean":"2023-11-27T15:24:40.391373+0000","last_became_active":"2023-11-27T15:23:27.194202+0000","last_became_peered":"2023-11-27T15:23:27.194202+0000","last_unstale":"2023-11-27T15:24:40.391373+0000","last_undegraded":"2023-11-27T15:24:40.391373+0000","last_fullsized":"2023-11-27T15:24:40.391373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:56:54.160848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.255949+0000","last_change":"2023-11-27T15:23:31.638466+0000","last_active":"2023-11-27T15:24:40.255949+0000","last_peered":"2023-11-27T15:24:40.255949+0000","last_clean":"2023-11-27T15:24:40.255949+0000","last_became_active":"2023-11-27T15:23:31.638103+0000","last_became_peered":"2023-11-27T15:23:31.638103+0000","last_unstale":"2023-11-27T15:24:40.255949+0000","last_undegraded":"2023-11-27T15:24:40.255949+0000","last_fullsized":"2023-11-27T15:24:40.255949+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:30.612668+0000","last_clean_scrub_stamp":"2023-11-27T15:23:30.612668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:06:34.847983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.472903+0000","last_change":"2023-11-27T15:24:40.472903+0000","last_active":"2023-11-27T15:24:40.472903+0000","last_peered":"2023-11-27T15:24:40.472903+0000","last_clean":"2023-11-27T15:24:40.472903+0000","last_became_active":"2023-11-27T15:24:40.321194+0000","last_became_peered":"2023-11-27T15:24:40.321194+0000","last_unstale":"2023-11-27T15:24:40.472903+0000","last_undegraded":"2023-11-27T15:24:40.472903+0000","last_fullsized":"2023-11-27T15:24:40.472903+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:25.560328+0000","last_clean_scrub_stamp":"2023-11-27T15:23:25.560328+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:04:18.954319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.391695+0000","last_change":"2023-11-27T15:23:27.185677+0000","last_active":"2023-11-27T15:24:40.391695+0000","last_peered":"2023-11-27T15:24:40.391695+0000","last_clean":"2023-11-27T15:24:40.391695+0000","last_became_active":"2023-11-27T15:23:27.185321+0000","last_became_peered":"2023-11-27T15:23:27.185321+0000","last_unstale":"2023-11-27T15:24:40.391695+0000","last_undegraded":"2023-11-27T15:24:40.391695+0000","last_fullsized":"2023-11-27T15:24:40.391695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:58:02.569108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260276+0000","last_change":"2023-11-27T15:23:27.195847+0000","last_active":"2023-11-27T15:24:40.260276+0000","last_peered":"2023-11-27T15:24:40.260276+0000","last_clean":"2023-11-27T15:24:40.260276+0000","last_became_active":"2023-11-27T15:23:27.195421+0000","last_became_peered":"2023-11-27T15:23:27.195421+0000","last_unstale":"2023-11-27T15:24:40.260276+0000","last_undegraded":"2023-11-27T15:24:40.260276+0000","last_fullsized":"2023-11-27T15:24:40.260276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:32:18.774313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259311+0000","last_change":"2023-11-27T15:23:27.181352+0000","last_active":"2023-11-27T15:24:40.259311+0000","last_peered":"2023-11-27T15:24:40.259311+0000","last_clean":"2023-11-27T15:24:40.259311+0000","last_became_active":"2023-11-27T15:23:27.181042+0000","last_became_peered":"2023-11-27T15:23:27.181042+0000","last_unstale":"2023-11-27T15:24:40.259311+0000","last_undegraded":"2023-11-27T15:24:40.259311+0000","last_fullsized":"2023-11-27T15:24:40.259311+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:20:09.382676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.259118+0000","last_change":"2023-11-27T15:23:27.199915+0000","last_active":"2023-11-27T15:24:40.259118+0000","last_peered":"2023-11-27T15:24:40.259118+0000","last_clean":"2023-11-27T15:24:40.259118+0000","last_became_active":"2023-11-27T15:23:27.199595+0000","last_became_peered":"2023-11-27T15:23:27.199595+0000","last_unstale":"2023-11-27T15:24:40.259118+0000","last_undegraded":"2023-11-27T15:24:40.259118+0000","last_fullsized":"2023-11-27T15:24:40.259118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:20:53.989286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.262679+0000","last_change":"2023-11-27T15:23:27.210258+0000","last_active":"2023-11-27T15:24:40.262679+0000","last_peered":"2023-11-27T15:24:40.262679+0000","last_clean":"2023-11-27T15:24:40.262679+0000","last_became_active":"2023-11-27T15:23:27.208028+0000","last_became_peered":"2023-11-27T15:23:27.208028+0000","last_unstale":"2023-11-27T15:24:40.262679+0000","last_undegraded":"2023-11-27T15:24:40.262679+0000","last_fullsized":"2023-11-27T15:24:40.262679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:19:21.249912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.277411+0000","last_change":"2023-11-27T15:24:40.277411+0000","last_active":"2023-11-27T15:24:40.277411+0000","last_peered":"2023-11-27T15:24:40.277411+0000","last_clean":"2023-11-27T15:24:40.277411+0000","last_became_active":"2023-11-27T15:24:40.276642+0000","last_became_peered":"2023-11-27T15:24:40.276642+0000","last_unstale":"2023-11-27T15:24:40.277411+0000","last_undegraded":"2023-11-27T15:24:40.277411+0000","last_fullsized":"2023-11-27T15:24:40.277411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:22:44.632562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260814+0000","last_change":"2023-11-27T15:23:27.185855+0000","last_active":"2023-11-27T15:24:40.260814+0000","last_peered":"2023-11-27T15:24:40.260814+0000","last_clean":"2023-11-27T15:24:40.260814+0000","last_became_active":"2023-11-27T15:23:27.185535+0000","last_became_peered":"2023-11-27T15:23:27.185535+0000","last_unstale":"2023-11-27T15:24:40.260814+0000","last_undegraded":"2023-11-27T15:24:40.260814+0000","last_fullsized":"2023-11-27T15:24:40.260814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:38:04.393821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.262269+0000","last_change":"2023-11-27T15:23:27.190163+0000","last_active":"2023-11-27T15:24:40.262269+0000","last_peered":"2023-11-27T15:24:40.262269+0000","last_clean":"2023-11-27T15:24:40.262269+0000","last_became_active":"2023-11-27T15:23:27.188729+0000","last_became_peered":"2023-11-27T15:23:27.188729+0000","last_unstale":"2023-11-27T15:24:40.262269+0000","last_undegraded":"2023-11-27T15:24:40.262269+0000","last_fullsized":"2023-11-27T15:24:40.262269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:36:31.808113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.269343+0000","last_change":"2023-11-27T15:23:27.203904+0000","last_active":"2023-11-27T15:24:40.269343+0000","last_peered":"2023-11-27T15:24:40.269343+0000","last_clean":"2023-11-27T15:24:40.269343+0000","last_became_active":"2023-11-27T15:23:27.203578+0000","last_became_peered":"2023-11-27T15:23:27.203578+0000","last_unstale":"2023-11-27T15:24:40.269343+0000","last_undegraded":"2023-11-27T15:24:40.269343+0000","last_fullsized":"2023-11-27T15:24:40.269343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:05:42.402028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256156+0000","last_change":"2023-11-27T15:23:27.206791+0000","last_active":"2023-11-27T15:24:40.256156+0000","last_peered":"2023-11-27T15:24:40.256156+0000","last_clean":"2023-11-27T15:24:40.256156+0000","last_became_active":"2023-11-27T15:23:27.206529+0000","last_became_peered":"2023-11-27T15:23:27.206529+0000","last_unstale":"2023-11-27T15:24:40.256156+0000","last_undegraded":"2023-11-27T15:24:40.256156+0000","last_fullsized":"2023-11-27T15:24:40.256156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:25:25.559405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.269123+0000","last_change":"2023-11-27T15:23:27.197562+0000","last_active":"2023-11-27T15:24:40.269123+0000","last_peered":"2023-11-27T15:24:40.269123+0000","last_clean":"2023-11-27T15:24:40.269123+0000","last_became_active":"2023-11-27T15:23:27.197233+0000","last_became_peered":"2023-11-27T15:23:27.197233+0000","last_unstale":"2023-11-27T15:24:40.269123+0000","last_undegraded":"2023-11-27T15:24:40.269123+0000","last_fullsized":"2023-11-27T15:24:40.269123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:21:03.470847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.256739+0000","last_change":"2023-11-27T15:23:27.191281+0000","last_active":"2023-11-27T15:24:40.256739+0000","last_peered":"2023-11-27T15:24:40.256739+0000","last_clean":"2023-11-27T15:24:40.256739+0000","last_became_active":"2023-11-27T15:23:27.190973+0000","last_became_peered":"2023-11-27T15:23:27.190973+0000","last_unstale":"2023-11-27T15:24:40.256739+0000","last_undegraded":"2023-11-27T15:24:40.256739+0000","last_fullsized":"2023-11-27T15:24:40.256739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:53:32.343841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.260209+0000","last_change":"2023-11-27T15:23:27.199251+0000","last_active":"2023-11-27T15:24:40.260209+0000","last_peered":"2023-11-27T15:24:40.260209+0000","last_clean":"2023-11-27T15:24:40.260209+0000","last_became_active":"2023-11-27T15:23:27.198926+0000","last_became_peered":"2023-11-27T15:23:27.198926+0000","last_unstale":"2023-11-27T15:24:40.260209+0000","last_undegraded":"2023-11-27T15:24:40.260209+0000","last_fullsized":"2023-11-27T15:24:40.260209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:01:23.484571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:24:40.261579+0000","last_change":"2023-11-27T15:23:27.192227+0000","last_active":"2023-11-27T15:24:40.261579+0000","last_peered":"2023-11-27T15:24:40.261579+0000","last_clean":"2023-11-27T15:24:40.261579+0000","last_became_active":"2023-11-27T15:23:27.191852+0000","last_became_peered":"2023-11-27T15:23:27.191852+0000","last_unstale":"2023-11-27T15:24:40.261579+0000","last_undegraded":"2023-11-27T15:24:40.261579+0000","last_fullsized":"2023-11-27T15:24:40.261579+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:23:26.584718+0000","last_clean_scrub_stamp":"2023-11-27T15:23:26.584718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:02:22.487861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65716,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":342,"num_read_kb":241,"num_write":155,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":694,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":197148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6180,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93738972,"statfs":{"total":95995035648,"available":95988707328,"internally_reserved":0,"allocated":1011712,"data_stored":490981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.63500000000000001,"15min":0.64500000000000002},"min":{"1min":0.48499999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85099999999999998,"5min":0.87,"15min":0.87},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78900000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59599999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.90200000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61899999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83899999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.501},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59099999999999997,"15min":0.57599999999999996},"min":{"1min":0.40000000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85999999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.70699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77100000000000002,"15min":0.77000000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97299999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70899999999999996,"15min":0.71799999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.77200000000000002,"5min":1.087,"15min":1.087},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73599999999999999,"15min":0.74299999999999999},"min":{"1min":0.46400000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.98099999999999998,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.748},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75600000000000001,"15min":0.754},"min":{"1min":0.501,"5min":0.435,"15min":0.435},"max":{"1min":1.1180000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.88300000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.71699999999999997,"15min":0.73099999999999998},"min":{"1min":0.42999999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.79200000000000004,"5min":1.073,"15min":1.073},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.85599999999999998,"15min":0.85999999999999999},"min":{"1min":0.66400000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.032,"5min":1.651,"15min":1.651},"last":0.96499999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.88700000000000001,"15min":0.89200000000000002},"min":{"1min":0.61799999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0489999999999999,"5min":1.673,"15min":1.673},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80000000000000004,"15min":0.80600000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.014,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.79900000000000004}]},{"osd":8,"last update":"Mon Nov 27 15:24:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.92400000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.077,"5min":1.7729999999999999,"15min":1.7729999999999999},"last":1.004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.90600000000000003,"15min":0.90000000000000002},"min":{"1min":0.81799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1319999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.0349999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.85599999999999998,"15min":0.86199999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96399999999999997,"5min":1.268,"15min":1.268},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":0.93899999999999995},"min":{"1min":0.54100000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0880000000000001,"5min":1.762,"15min":1.762},"last":0.86699999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645347,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7160,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93737992,"statfs":{"total":95995035648,"available":95987703808,"internally_reserved":0,"allocated":1687552,"data_stored":1139570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.629},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.56299999999999994}]},{"osd":1,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45300000000000001,"15min":0.45200000000000001},"min":{"1min":0.32400000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.622,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.499,"15min":0.5},"min":{"1min":0.30099999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.72399999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.90200000000000002,"5min":1.014,"15min":1.014},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.51500000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80000000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.78000000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.65000000000000002,"15min":0.66200000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79800000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.79700000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.59499999999999997},"min":{"1min":0.40500000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98799999999999999,"5min":1.167,"15min":1.167},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.35299999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.79600000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.81200000000000006}]},{"osd":5,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95399999999999996,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72999999999999998,"15min":0.72799999999999998},"min":{"1min":0.54200000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96599999999999997,"5min":1,"15min":1},"last":0.91000000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.755,"15min":0.76100000000000001},"min":{"1min":0.48899999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.99399999999999999,"5min":1.125,"15min":1.125},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.3999999999999999,"5min":0.92400000000000004,"15min":0.84399999999999997},"min":{"1min":0.53700000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":13.009,"5min":13.009,"15min":13.009},"last":0.97299999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65500000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0760000000000001,"5min":1.091,"15min":1.091},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.89000000000000001}]},{"osd":9,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.85099999999999998,"15min":0.85499999999999998},"min":{"1min":0.58099999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.405,"5min":0.94999999999999996,"15min":0.874},"min":{"1min":0.57799999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":12.989000000000001,"5min":12.989000000000001,"15min":12.989000000000001},"last":0.86299999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.78300000000000003},"min":{"1min":0.60799999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0309999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.87},{"interface":"front","average":{"1min":0.871,"5min":0.80800000000000005,"15min":0.79800000000000004},"min":{"1min":0.72299999999999998,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.95099999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":1024000,"data_stored":495201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.56100000000000005,"15min":0.57599999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64900000000000002,"5min":1.054,"15min":1.054},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.34999999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.68000000000000005,"15min":0.69099999999999995},"min":{"1min":0.44400000000000001,"5min":0.315,"15min":0.315},"max":{"1min":1.0029999999999999,"5min":1.046,"15min":1.046},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.044,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.66900000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.40999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.079,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.53100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97199999999999998,"5min":1.294,"15min":1.294},"last":0.61899999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.73699999999999999,"15min":0.755},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.3520000000000001,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.69699999999999995,"15min":0.70499999999999996},"min":{"1min":0.376,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.53800000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.82399999999999995},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.875,"15min":0.877},"min":{"1min":0.55800000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.9870000000000001,"5min":2.9870000000000001,"15min":2.9870000000000001},"last":0.73899999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.88800000000000001,"15min":0.90400000000000003},"min":{"1min":0.64800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3129999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.85899999999999999},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:22:53 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.84299999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.94799999999999995,"15min":0.97199999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.429,"5min":2.069,"15min":2.069},"last":0.875},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.95599999999999996,"15min":0.98399999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.702,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.67800000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.63900000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1990000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.872,"5min":1.0049999999999999,"15min":1.0269999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.478,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":0.79700000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.88700000000000001,"15min":0.90800000000000003},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.0389999999999999,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.94899999999999995,"15min":0.95099999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.948,"5min":2.948,"15min":2.948},"last":0.90400000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.64000000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070509,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6496,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738656,"statfs":{"total":95995035648,"available":95988383744,"internally_reserved":0,"allocated":1007616,"data_stored":490582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.60199999999999998},"min":{"1min":0.44800000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.85499999999999998,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58499999999999996,"15min":0.57399999999999995},"min":{"1min":0.495,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.66300000000000003}]},{"osd":1,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.65200000000000002,"15min":0.63200000000000001},"min":{"1min":0.56499999999999995,"5min":0.315,"15min":0.315},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.63100000000000001,"15min":0.60999999999999999},"min":{"1min":0.52000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.88600000000000001,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.87}]},{"osd":2,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.36899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.49099999999999999,"15min":0.48999999999999999},"min":{"1min":0.40200000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.69599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.46200000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93400000000000005,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.70299999999999996,"15min":0.67600000000000005},"min":{"1min":0.505,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98399999999999999,"5min":3.3940000000000001,"15min":3.3940000000000001},"last":0.63800000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.67300000000000004,"15min":0.65500000000000003},"min":{"1min":0.502,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.69499999999999995,"15min":0.67800000000000005},"min":{"1min":0.59199999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.79200000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.48099999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0029999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.75900000000000001,"15min":0.745},"min":{"1min":0.60099999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.083,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.52500000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.02,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.71299999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.753,"15min":0.72899999999999998},"min":{"1min":0.55100000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.996,"5min":3.4020000000000001,"15min":3.4020000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.79400000000000004,"15min":0.78000000000000003},"min":{"1min":0.67000000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.98999999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.73699999999999999,"15min":0.72199999999999998},"min":{"1min":0.63200000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.59399999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.117,"5min":1.119,"15min":1.119},"last":0.96199999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.79800000000000004,"15min":0.77900000000000003},"min":{"1min":0.59999999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.123,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.84099999999999997,"15min":0.82499999999999996},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":1.012}]},{"osd":11,"last update":"Mon Nov 27 15:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.82699999999999996}]}]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7152,"kb_used_data":1576,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738000,"statfs":{"total":95995035648,"available":95987712000,"internally_reserved":0,"allocated":1613824,"data_stored":1087227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:23:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.53500000000000003,"15min":0.52500000000000002},"min":{"1min":0.34100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.35799999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46400000000000002,"15min":0.46200000000000002},"min":{"1min":0.309,"5min":0.255,"15min":0.255},"max":{"1min":0.69099999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.66200000000000003,"5min":0.754,"15min":0.754},"last":0.64800000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66500000000000004,"15min":0.63600000000000001},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.91300000000000003,"5min":6.0720000000000001,"15min":6.0720000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.61599999999999999,"15min":0.60399999999999998},"min":{"1min":0.42999999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89700000000000002,"5min":1.03,"15min":1.03},"last":0.89200000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.58999999999999997},"min":{"1min":0.39400000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95799999999999996,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86799999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84999999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56899999999999995,"15min":0.56599999999999995},"min":{"1min":0.35699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79700000000000004,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.745},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.33500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79100000000000004,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68400000000000005,"15min":0.68100000000000005},"min":{"1min":0.45600000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88500000000000001,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93200000000000005,"5min":1.018,"15min":1.018},"last":1.052}]},{"osd":6,"last update":"Mon Nov 27 15:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.72699999999999998,"15min":0.73199999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.80200000000000005,"15min":0.77800000000000002},"min":{"1min":0.51500000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.876,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":0.86799999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79800000000000004,"15min":0.76500000000000001},"min":{"1min":0.48699999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97099999999999997,"5min":6.1050000000000004,"15min":6.1050000000000004},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82399999999999995,"15min":0.78800000000000003},"min":{"1min":0.51400000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.016,"5min":6.0330000000000004,"15min":6.0330000000000004},"last":1.0880000000000001}]},{"osd":9,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.83099999999999996,"15min":0.84299999999999997},"min":{"1min":0.54600000000000004,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98299999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.94199999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0089999999999999,"5min":1.171,"15min":1.171},"last":1.022},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.754,"15min":0.75900000000000001},"min":{"1min":0.54500000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94799999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.98599999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":1.1299999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462966,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6424,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738728,"statfs":{"total":95995035648,"available":95988457472,"internally_reserved":0,"allocated":999424,"data_stored":486281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.56299999999999994,"15min":0.51100000000000001},"min":{"1min":0.34799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":3.3330000000000002,"5min":3.3330000000000002,"15min":3.3330000000000002},"last":0.621},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.57899999999999996,"15min":0.55500000000000005},"min":{"1min":0.33100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":4.2240000000000002,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":0.39100000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.80800000000000005,"15min":0.753},"min":{"1min":0.55700000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.8700000000000001,"5min":4.8700000000000001,"15min":4.8700000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.71099999999999997,"15min":0.65700000000000003},"min":{"1min":0.54500000000000004,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":0.78200000000000003,"15min":0.71099999999999997},"min":{"1min":0.51600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":4.274,"5min":4.274,"15min":4.274},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.0880000000000001,"5min":0.84599999999999997,"15min":0.78000000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":4.9470000000000001,"5min":4.9470000000000001,"15min":4.9470000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.73499999999999999,"15min":0.67700000000000005},"min":{"1min":0.47599999999999998,"5min":0.376,"15min":0.376},"max":{"1min":5.0830000000000002,"5min":5.0830000000000002,"15min":5.0830000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.69299999999999995,"15min":0.66300000000000003},"min":{"1min":0.36199999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":3.8380000000000001,"5min":3.8380000000000001,"15min":3.8380000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.873,"15min":0.82399999999999995},"min":{"1min":0.58799999999999997,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":4.9119999999999999,"5min":4.9119999999999999,"15min":4.9119999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.077,"5min":0.82599999999999996,"15min":0.77000000000000002},"min":{"1min":0.48599999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.8769999999999998,"5min":3.8769999999999998,"15min":3.8769999999999998},"last":0.70799999999999996}]},{"osd":5,"last update":"Mon Nov 27 15:24:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.83299999999999996,"15min":0.79800000000000004},"min":{"1min":0.47899999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":3.9329999999999998,"5min":3.9329999999999998,"15min":3.9329999999999998},"last":0.747},{"interface":"front","average":{"1min":1.1240000000000001,"5min":0.871,"15min":0.82299999999999995},"min":{"1min":0.60999999999999999,"5min":0.497,"15min":0.497},"max":{"1min":4.3090000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":0.84399999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:24:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82199999999999995,"15min":0.81699999999999995},"min":{"1min":0.55400000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.254,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.83799999999999997,"15min":0.82699999999999996},"min":{"1min":0.60799999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0940000000000001,"5min":1.175,"15min":1.175},"last":0.51700000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.52900000000000003,"5min":0.5,"15min":0.5},"max":{"1min":3.5070000000000001,"5min":3.5070000000000001,"15min":3.5070000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.90700000000000003,"15min":0.89100000000000001},"min":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":3.6619999999999999,"5min":3.6619999999999999,"15min":3.6619999999999999},"last":0.68899999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95299999999999996,"15min":0.95699999999999996},"min":{"1min":0.57499999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93200000000000005,"15min":0.93799999999999994},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.86799999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:24:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.95699999999999996,"15min":0.95299999999999996},"min":{"1min":0.77800000000000002,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.93600000000000005,"15min":0.93000000000000005},"min":{"1min":0.71499999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":0.88900000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:24:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":4.4089999999999998,"5min":4.4089999999999998,"15min":4.4089999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.90700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7580,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737572,"statfs":{"total":95995035648,"available":95987273728,"internally_reserved":0,"allocated":1069056,"data_stored":548195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.52200000000000002},"min":{"1min":0.36299999999999999,"5min":0.307,"15min":0.30599999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.55700000000000005},"min":{"1min":0.42999999999999999,"5min":0.314,"15min":0.314},"max":{"1min":1.0589999999999999,"5min":6.2839999999999998,"15min":6.2839999999999998},"last":0.51400000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.57899999999999996,"15min":0.52900000000000003},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.92900000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59599999999999997,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.749}]},{"osd":3,"last update":"Mon Nov 27 15:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58499999999999996,"15min":0.56499999999999995},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0669999999999999,"5min":6.3179999999999996,"15min":6.3179999999999996},"last":0.63400000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.46899999999999997,"15min":0.42099999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.5,"15min":0.45800000000000002},"min":{"1min":0.308,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.68300000000000005,"15min":0.63300000000000001},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0069999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.51100000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.95999999999999996,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.94799999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.503,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0880000000000001,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.877},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.52800000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.93500000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70099999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:24:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.372,"5min":0.316,"15min":0.316},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.71799999999999997,"15min":0.68999999999999995},"min":{"1min":0.47099999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.98899999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81499999999999995,"15min":0.81100000000000005},"min":{"1min":0.55600000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.0489999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.034},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69399999999999995,"15min":0.68200000000000005},"min":{"1min":0.501,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94399999999999995,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.89300000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.86699999999999999,"15min":0.879},"min":{"1min":0.51600000000000001,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1020000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.91500000000000004,"15min":0.92500000000000004},"min":{"1min":0.59599999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.012}]},{"osd":10,"last update":"Mon Nov 27 15:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89600000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.86599999999999999,"15min":0.874},"min":{"1min":0.52900000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.046,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.90700000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.96899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7468,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737684,"statfs":{"total":95995035648,"available":95987388416,"internally_reserved":0,"allocated":1019904,"data_stored":487338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.67600000000000005,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.34799999999999998,"15min":0.33800000000000002},"max":{"1min":6.6909999999999998,"5min":6.6909999999999998,"15min":6.6909999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.63400000000000001,"15min":0.56599999999999995},"min":{"1min":0.41899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Nov 27 15:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.59299999999999997,"15min":0.55000000000000004},"min":{"1min":0.49299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.89000000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.629,"15min":0.58599999999999997},"min":{"1min":0.57599999999999996,"5min":0.376,"15min":0.376},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.90900000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.39400000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67400000000000004,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.50900000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.71199999999999997,"15min":0.68500000000000005},"min":{"1min":0.55700000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.68700000000000006,"15min":0.67500000000000004},"min":{"1min":0.58299999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.65100000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:24:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.68700000000000006,"15min":0.64400000000000002},"min":{"1min":0.37,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.67200000000000004,"15min":0.64000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.48699999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:24:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.874}]},{"osd":8,"last update":"Mon Nov 27 15:24:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.57799999999999996,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":0.89900000000000002,"15min":0.86099999999999999},"min":{"1min":0.65600000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":6.7220000000000004,"5min":6.7220000000000004,"15min":6.7220000000000004},"last":0.78800000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.89600000000000002,"15min":0.91100000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.85299999999999998,"15min":0.874},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.032,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.59599999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.89100000000000001,"15min":0.89400000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.93200000000000005,"15min":0.93799999999999994},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.71499999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:24:40 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":0.89800000000000002,"15min":0.83599999999999997},"min":{"1min":0.57499999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":6.71,"5min":6.71,"15min":6.71},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.05,"5min":0.92000000000000004,"15min":0.89800000000000002},"min":{"1min":0.68600000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9179999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":0.77500000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7656,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737496,"statfs":{"total":95995035648,"available":95987195904,"internally_reserved":0,"allocated":1081344,"data_stored":548450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.53400000000000003},"min":{"1min":0.45600000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.77700000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.53400000000000003,"15min":0.51300000000000001},"min":{"1min":0.46800000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.78700000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51400000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:24:14 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.59399999999999997,"15min":0.51800000000000002},"min":{"1min":0.42199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.133,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58599999999999997,"15min":0.51200000000000001},"min":{"1min":0.38,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84899999999999998,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.48799999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67400000000000004,"15min":0.81200000000000006},"min":{"1min":0.47599999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.77500000000000002,"5min":7.1189999999999998,"15min":7.1189999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.61599999999999999,"15min":0.57399999999999995},"min":{"1min":0.51400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57899999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.59799999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.47999999999999998,"15min":0.45600000000000002},"min":{"1min":0.35999999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.53100000000000003,"15min":0.49299999999999999},"min":{"1min":0.39500000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42399999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.65800000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.69199999999999995,"15min":0.65000000000000002},"min":{"1min":0.629,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98499999999999999,"5min":1.02,"15min":1.02},"last":0.65300000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.749}]},{"osd":8,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.53300000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.65900000000000003,"15min":0.627},"min":{"1min":0.64100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66600000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.54000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.68799999999999994}]},{"osd":10,"last update":"Mon Nov 27 15:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.77900000000000003,"15min":0.75900000000000001},"min":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.627},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.747,"15min":0.72799999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.71499999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.76000000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6756,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738396,"statfs":{"total":95995035648,"available":95988117504,"internally_reserved":0,"allocated":1011712,"data_stored":486475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.48199999999999998,"15min":0.42899999999999999},"min":{"1min":0.35099999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.49399999999999999,"15min":0.45000000000000001},"min":{"1min":0.32300000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.1759999999999999,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.48899999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.63400000000000001,"15min":0.58399999999999996},"min":{"1min":0.54000000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.68500000000000005,"15min":0.62},"min":{"1min":0.54000000000000004,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.64400000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65400000000000003,"15min":0.627},"min":{"1min":0.505,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.67300000000000004,"15min":0.64100000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.752}]},{"osd":4,"last update":"Mon Nov 27 15:24:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64600000000000002,"15min":0.61199999999999999},"min":{"1min":0.503,"5min":0.443,"15min":0.443},"max":{"1min":0.80800000000000005,"5min":1.034,"15min":1.034},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69399999999999995,"15min":0.64900000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88200000000000001,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.70999999999999996}]},{"osd":5,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68300000000000005,"15min":0.64900000000000002},"min":{"1min":0.45700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88900000000000001,"5min":1.21,"15min":1.21},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.47499999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91600000000000004,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.69699999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.45500000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.54800000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.56999999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:24:10 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":0.89500000000000002,"15min":0.83199999999999996},"min":{"1min":0.48599999999999999,"5min":0.442,"15min":0.442},"max":{"1min":8.4710000000000001,"5min":8.4710000000000001,"15min":8.4710000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80800000000000005,"15min":0.80400000000000005},"min":{"1min":0.57999999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0209999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.58499999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:23:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.80300000000000005,"15min":0.81399999999999995},"min":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.84699999999999998,"15min":0.84999999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.80500000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:23:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.83399999999999996,"15min":0.85099999999999998},"min":{"1min":0.54000000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96399999999999997,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.93700000000000006,"15min":0.95199999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.028,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.73399999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.92300000000000004,"15min":0.93600000000000005},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.0349999999999999,"5min":1.462,"15min":1.462},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.85299999999999998,"15min":0.86599999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":0.95699999999999996,"5min":1.234,"15min":1.234},"last":0.56399999999999995}]},{"osd":11,"last update":"Mon Nov 27 15:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80400000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888128,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6624,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738528,"statfs":{"total":95995035648,"available":95988252672,"internally_reserved":0,"allocated":1007616,"data_stored":490942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.54900000000000004,"15min":0.52100000000000002},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.52300000000000002,"15min":0.49199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.58399999999999996}]},{"osd":1,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.41099999999999998,"15min":0.375},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.70899999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.436,"15min":0.39600000000000002},"min":{"1min":0.29999999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.35099999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72199999999999998,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61099999999999999,"15min":0.59599999999999997},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.69199999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:23:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.62,"15min":0.57299999999999995},"min":{"1min":0.432,"5min":0.375,"15min":0.375},"max":{"1min":0.85499999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60599999999999998,"15min":0.58699999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":1.329,"15min":1.329},"last":0.502}]},{"osd":5,"last update":"Mon Nov 27 15:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.60799999999999998},"min":{"1min":0.51200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.68200000000000005,"15min":0.628},"min":{"1min":0.52300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93899999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.56799999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.69699999999999995},"min":{"1min":0.55600000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.96999999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69499999999999995,"15min":0.69199999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0489999999999999,"5min":1.288,"15min":1.288},"last":0.67700000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:24:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.42899999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92000000000000004,"5min":1.171,"15min":1.171},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70499999999999996,"15min":0.69999999999999996},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.022,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73399999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.52700000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97999999999999998,"5min":1.038,"15min":1.038},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.60299999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.97099999999999997,"5min":1.107,"15min":1.107},"last":0.71999999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1599999999999999,"5min":1.222,"15min":1.222},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.84099999999999997,"15min":0.84299999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.96899999999999997,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.755}]},{"osd":10,"last update":"Mon Nov 27 15:24:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.79500000000000004,"15min":0.78300000000000003},"min":{"1min":0.54300000000000004,"5min":0.432,"15min":0.432},"max":{"1min":1.014,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.75700000000000001,"15min":0.747},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1339999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.70799999999999996}]},{"osd":11,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.76900000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":1622016,"data_stored":1082473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58599999999999997,"15min":0.61199999999999999},"min":{"1min":0.379,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.79200000000000004,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.41799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70199999999999996,"5min":0.873,"15min":0.873},"last":0.52400000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.56699999999999995,"15min":0.52900000000000003},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80600000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56999999999999995,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.71899999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42899999999999999,"15min":0.40300000000000002},"min":{"1min":0.34000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.59799999999999998,"5min":0.752,"15min":0.752},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.44900000000000001,"15min":0.42799999999999999},"min":{"1min":0.314,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65700000000000003,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.53300000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.92400000000000004,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64200000000000002,"15min":0.621},"min":{"1min":0.39300000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88400000000000001,"5min":2.097,"15min":2.097},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.623,"15min":0.56799999999999995},"min":{"1min":0.45100000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92400000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73199999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:24:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65900000000000003,"15min":0.65300000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.872,"5min":1.004,"15min":1.004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68600000000000005,"15min":0.67100000000000004},"min":{"1min":0.53000000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90200000000000002,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.61699999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.47199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.86299999999999999,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.69899999999999995,"15min":0.67500000000000004},"min":{"1min":0.55900000000000005,"5min":0.434,"15min":0.434},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.81899999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.67000000000000004},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.625},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69599999999999995,"15min":0.68300000000000005},"min":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95899999999999996,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.77100000000000002,"15min":0.76400000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.012,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77600000000000002,"15min":0.76600000000000001},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.74299999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:24:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79200000000000004,"15min":0.78500000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.98999999999999999,"5min":1.026,"15min":1.026},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.81499999999999995,"15min":0.81100000000000005},"min":{"1min":0.68200000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:24:55.443 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-11-27T15:24:55.443 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-27T15:24:55.443 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-11-27T15:24:55.444 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-11-27T15:24:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:55 smithi060 ceph-mon[131518]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2023-11-27T15:24:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:55 smithi008 ceph-mon[128788]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2023-11-27T15:24:55.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:55 smithi029 ceph-mon[131166]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 48 B/s, 0 objects/s recovering 2023-11-27T15:24:56.307 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:24:56.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:56 smithi060 ceph-mon[131518]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:56.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:56 smithi008 ceph-mon[128788]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:56.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:56 smithi029 ceph-mon[131166]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:24:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:57 smithi060 ceph-mon[131518]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-11-27T15:24:57.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.827+0000 7ff95c649700 1 -- 172.21.15.8:0/1545727537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff954100540 msgr2=0x7ff954100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:57.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.827+0000 7ff95c649700 1 --2- 172.21.15.8:0/1545727537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff954100540 0x7ff954100910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff9500099c0 tx=0x7ff950009cd0 comp rx=0 tx=0).stop 2023-11-27T15:24:57.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.827+0000 7ff95c649700 1 -- 172.21.15.8:0/1545727537 shutdown_connections 2023-11-27T15:24:57.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.827+0000 7ff95c649700 1 --2- 172.21.15.8:0/1545727537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff9541055e0 0x7ff95410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:57.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.827+0000 7ff95c649700 1 --2- 172.21.15.8:0/1545727537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff954100540 0x7ff954100910 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:57.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.827+0000 7ff95c649700 1 --2- 172.21.15.8:0/1545727537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff954100ee0 0x7ff954104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:57.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.827+0000 7ff95c649700 1 -- 172.21.15.8:0/1545727537 >> 172.21.15.8:0/1545727537 conn(0x7ff9540fbdc0 msgr2=0x7ff9540fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:57.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.827+0000 7ff95c649700 1 -- 172.21.15.8:0/1545727537 shutdown_connections 2023-11-27T15:24:57.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.827+0000 7ff95c649700 1 -- 172.21.15.8:0/1545727537 wait complete. 2023-11-27T15:24:57.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.828+0000 7ff95c649700 1 Processor -- start 2023-11-27T15:24:57.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.829+0000 7ff95c649700 1 -- start start 2023-11-27T15:24:57.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.829+0000 7ff95c649700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff954100540 0x7ff954072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:57.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.829+0000 7ff95c649700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff954100ee0 0x7ff954073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:57.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.829+0000 7ff95c649700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9541055e0 0x7ff9540738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:57.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.829+0000 7ff95c649700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9541a4850 con 0x7ff9541055e0 2023-11-27T15:24:57.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.829+0000 7ff95c649700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9541a49c0 con 0x7ff954100ee0 2023-11-27T15:24:57.833 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.829+0000 7ff95c649700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9541a4b80 con 0x7ff954100540 2023-11-27T15:24:57.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff95a3e5700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff954100540 0x7ff954072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:57.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff95abe6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9541055e0 0x7ff9540738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:57.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff95a3e5700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff954100540 0x7ff954072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:50540/0 (socket says 172.21.15.8:50540) 2023-11-27T15:24:57.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff95a3e5700 1 -- 172.21.15.8:0/90507063 learned_addr learned my addr 172.21.15.8:0/90507063 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:57.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff959be4700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff954100ee0 0x7ff954073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:57.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff95a3e5700 1 -- 172.21.15.8:0/90507063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff954100ee0 msgr2=0x7ff954073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:57.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff95a3e5700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff954100ee0 0x7ff954073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:57.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff95a3e5700 1 -- 172.21.15.8:0/90507063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9541055e0 msgr2=0x7ff9540738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:57.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff95a3e5700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9541055e0 0x7ff9540738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:57.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff95a3e5700 1 -- 172.21.15.8:0/90507063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500095f0 con 0x7ff954100540 2023-11-27T15:24:57.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.830+0000 7ff95a3e5700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff954100540 0x7ff954072df0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff9500098e0 tx=0x7ff95000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:57.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.831+0000 7ff94b7fe700 1 -- 172.21.15.8:0/90507063 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95001d070 con 0x7ff954100540 2023-11-27T15:24:57.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.831+0000 7ff95c649700 1 -- 172.21.15.8:0/90507063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff954110390 con 0x7ff954100540 2023-11-27T15:24:57.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.831+0000 7ff94b7fe700 1 -- 172.21.15.8:0/90507063 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff950022470 con 0x7ff954100540 2023-11-27T15:24:57.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.831+0000 7ff94b7fe700 1 -- 172.21.15.8:0/90507063 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950017650 con 0x7ff954100540 2023-11-27T15:24:57.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.831+0000 7ff95c649700 1 -- 172.21.15.8:0/90507063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff954110200 con 0x7ff954100540 2023-11-27T15:24:57.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.832+0000 7ff95c649700 1 -- 172.21.15.8:0/90507063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff954110520 con 0x7ff954100540 2023-11-27T15:24:57.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.834+0000 7ff94b7fe700 1 -- 172.21.15.8:0/90507063 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff950022850 con 0x7ff954100540 2023-11-27T15:24:57.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.835+0000 7ff94b7fe700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9400a11e0 0x7ff9400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:57.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.835+0000 7ff94b7fe700 1 -- 172.21.15.8:0/90507063 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff9500cd840 con 0x7ff954100540 2023-11-27T15:24:57.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.835+0000 7ff959be4700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9400a11e0 0x7ff9400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:57.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.836+0000 7ff959be4700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9400a11e0 0x7ff9400a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff9540745e0 tx=0x7ff944009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:57.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:57.839+0000 7ff94b7fe700 1 -- 172.21.15.8:0/90507063 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff950096c60 con 0x7ff954100540 2023-11-27T15:24:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:57 smithi008 ceph-mon[128788]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-11-27T15:24:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:57 smithi029 ceph-mon[131166]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-11-27T15:24:58.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.092+0000 7ff95c649700 1 -- 172.21.15.8:0/90507063 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff954074320 con 0x7ff954100540 2023-11-27T15:24:58.096 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.092+0000 7ff94b7fe700 1 -- 172.21.15.8:0/90507063 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff950085040 con 0x7ff954100540 2023-11-27T15:24:58.096 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:58.097 INFO:teuthology.orchestra.run.smithi008.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-27T15:24:58.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 -- 172.21.15.8:0/90507063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9400a11e0 msgr2=0x7ff9400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9400a11e0 0x7ff9400a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff9540745e0 tx=0x7ff944009260 comp rx=0 tx=0).stop 2023-11-27T15:24:58.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 -- 172.21.15.8:0/90507063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff954100540 msgr2=0x7ff954072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff954100540 0x7ff954072df0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff9500098e0 tx=0x7ff95000f690 comp rx=0 tx=0).stop 2023-11-27T15:24:58.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 -- 172.21.15.8:0/90507063 shutdown_connections 2023-11-27T15:24:58.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9400a11e0 0x7ff9400a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.099 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff954100540 0x7ff954072df0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9541055e0 0x7ff9540738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 --2- 172.21.15.8:0/90507063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff954100ee0 0x7ff954073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 -- 172.21.15.8:0/90507063 >> 172.21.15.8:0/90507063 conn(0x7ff9540fbdc0 msgr2=0x7ff9540fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:58.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 -- 172.21.15.8:0/90507063 shutdown_connections 2023-11-27T15:24:58.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.095+0000 7ff95c649700 1 -- 172.21.15.8:0/90507063 wait complete. 2023-11-27T15:24:58.507 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-11-27T15:24:58.508 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:24:58.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.714+0000 7fe0197ff700 1 -- 172.21.15.8:0/3657803684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe014101e50 msgr2=0x7fe014108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.714+0000 7fe0197ff700 1 --2- 172.21.15.8:0/3657803684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe014101e50 0x7fe014108710 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00b0a0 tx=0x7fe00c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.714+0000 7fe0197ff700 1 -- 172.21.15.8:0/3657803684 shutdown_connections 2023-11-27T15:24:58.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.714+0000 7fe0197ff700 1 --2- 172.21.15.8:0/3657803684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe014101e50 0x7fe014108710 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.714+0000 7fe0197ff700 1 --2- 172.21.15.8:0/3657803684 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0141002d0 0x7fe0141006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.714+0000 7fe0197ff700 1 --2- 172.21.15.8:0/3657803684 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0141014c0 0x7fe014101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.714+0000 7fe0197ff700 1 -- 172.21.15.8:0/3657803684 >> 172.21.15.8:0/3657803684 conn(0x7fe0140fb8b0 msgr2=0x7fe0140fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:58.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.714+0000 7fe0197ff700 1 -- 172.21.15.8:0/3657803684 shutdown_connections 2023-11-27T15:24:58.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.714+0000 7fe0197ff700 1 -- 172.21.15.8:0/3657803684 wait complete. 2023-11-27T15:24:58.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.715+0000 7fe0197ff700 1 Processor -- start 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.715+0000 7fe0197ff700 1 -- start start 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0197ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0141002d0 0x7fe0140748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0197ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0141014c0 0x7fe014072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0197ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe014101e50 0x7fe014073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0197ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe014073bc0 con 0x7fe014101e50 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0197ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe014073d30 con 0x7fe0141014c0 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0197ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0141a01b0 con 0x7fe0141002d0 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0137fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe014101e50 0x7fe014073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0137fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe014101e50 0x7fe014073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45302/0 (socket says 172.21.15.8:45302) 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0137fe700 1 -- 172.21.15.8:0/2378844180 learned_addr learned my addr 172.21.15.8:0/2378844180 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe012ffd700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0141002d0 0x7fe0140748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe00bfff700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0141014c0 0x7fe014072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0137fe700 1 -- 172.21.15.8:0/2378844180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0141002d0 msgr2=0x7fe0140748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0137fe700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0141002d0 0x7fe0140748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0137fe700 1 -- 172.21.15.8:0/2378844180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0141014c0 msgr2=0x7fe014072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0137fe700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0141014c0 0x7fe014072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.716+0000 7fe0137fe700 1 -- 172.21.15.8:0/2378844180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c009d00 con 0x7fe014101e50 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.717+0000 7fe0137fe700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe014101e50 0x7fe014073420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00d010 tx=0x7fe00c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.717+0000 7fe010ff9700 1 -- 172.21.15.8:0/2378844180 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00e040 con 0x7fe014101e50 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.717+0000 7fe010ff9700 1 -- 172.21.15.8:0/2378844180 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe00c003e10 con 0x7fe014101e50 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.717+0000 7fe0197ff700 1 -- 172.21.15.8:0/2378844180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01410bca0 con 0x7fe014101e50 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.717+0000 7fe010ff9700 1 -- 172.21.15.8:0/2378844180 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0052a0 con 0x7fe014101e50 2023-11-27T15:24:58.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.717+0000 7fe0197ff700 1 -- 172.21.15.8:0/2378844180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01410bb10 con 0x7fe014101e50 2023-11-27T15:24:58.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.718+0000 7fe0197ff700 1 -- 172.21.15.8:0/2378844180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0141077d0 con 0x7fe014101e50 2023-11-27T15:24:58.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.720+0000 7fe010ff9700 1 -- 172.21.15.8:0/2378844180 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe00c01d480 con 0x7fe014101e50 2023-11-27T15:24:58.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.721+0000 7fe010ff9700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe0000a1260 0x7fe0000a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.722+0000 7fe010ff9700 1 -- 172.21.15.8:0/2378844180 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe00c0ce0e0 con 0x7fe014101e50 2023-11-27T15:24:58.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.722+0000 7fe012ffd700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe0000a1260 0x7fe0000a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.722+0000 7fe012ffd700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe0000a1260 0x7fe0000a3710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe014101320 tx=0x7fe004009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:58.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.725+0000 7fe010ff9700 1 -- 172.21.15.8:0/2378844180 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe00c0973f0 con 0x7fe014101e50 2023-11-27T15:24:58.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:58 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/90507063' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-27T15:24:58.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:58 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/90507063' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-27T15:24:58.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:58 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/90507063' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-27T15:24:58.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.936+0000 7fe0197ff700 1 -- 172.21.15.8:0/2378844180 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe01402cfe0 con 0x7fe014101e50 2023-11-27T15:24:58.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.937+0000 7fe010ff9700 1 -- 172.21.15.8:0/2378844180 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24344 (secure 0 0 0) 0x7fe00c09aa10 con 0x7fe014101e50 2023-11-27T15:24:58.941 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:24:58.942 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":90,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:24:40.249579+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:24:58.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 -- 172.21.15.8:0/2378844180 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe0000a1260 msgr2=0x7fe0000a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe0000a1260 0x7fe0000a3710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe014101320 tx=0x7fe004009260 comp rx=0 tx=0).stop 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 -- 172.21.15.8:0/2378844180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe014101e50 msgr2=0x7fe014073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe014101e50 0x7fe014073420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00d010 tx=0x7fe00c00bc80 comp rx=0 tx=0).stop 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 -- 172.21.15.8:0/2378844180 shutdown_connections 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe0000a1260 0x7fe0000a3710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe0141002d0 0x7fe0140748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe014101e50 0x7fe014073420 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 --2- 172.21.15.8:0/2378844180 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe0141014c0 0x7fe014072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 -- 172.21.15.8:0/2378844180 >> 172.21.15.8:0/2378844180 conn(0x7fe0140fb8b0 msgr2=0x7fe014104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 -- 172.21.15.8:0/2378844180 shutdown_connections 2023-11-27T15:24:58.944 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:58.940+0000 7fe0197ff700 1 -- 172.21.15.8:0/2378844180 wait complete. 2023-11-27T15:24:58.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:23:25.166148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:23:26.584539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:23:30.130965+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:24:58.963 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:24:59.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.168+0000 7f3d333ff700 1 -- 172.21.15.8:0/3965511783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d2c074b30 msgr2=0x7f3d2c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:59.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.168+0000 7f3d333ff700 1 --2- 172.21.15.8:0/3965511783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d2c074b30 0x7f3d2c072f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009990 tx=0x7f3d28009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.168+0000 7f3d333ff700 1 -- 172.21.15.8:0/3965511783 shutdown_connections 2023-11-27T15:24:59.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.168+0000 7f3d333ff700 1 --2- 172.21.15.8:0/3965511783 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d2c074b30 0x7f3d2c072f80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.168+0000 7f3d333ff700 1 --2- 172.21.15.8:0/3965511783 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d2c108150 0x7f3d2c10a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.168+0000 7f3d333ff700 1 --2- 172.21.15.8:0/3965511783 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d2c073550 0x7f3d2c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.168+0000 7f3d333ff700 1 -- 172.21.15.8:0/3965511783 >> 172.21.15.8:0/3965511783 conn(0x7f3d2c0fb870 msgr2=0x7f3d2c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:59.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.168+0000 7f3d333ff700 1 -- 172.21.15.8:0/3965511783 shutdown_connections 2023-11-27T15:24:59.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.168+0000 7f3d333ff700 1 -- 172.21.15.8:0/3965511783 wait complete. 2023-11-27T15:24:59.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.169+0000 7f3d333ff700 1 Processor -- start 2023-11-27T15:24:59.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.169+0000 7f3d333ff700 1 -- start start 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d333ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d2c073550 0x7f3d2c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d333ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d2c074b30 0x7f3d2c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d333ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d2c108150 0x7f3d2c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d333ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c19fdd0 con 0x7f3d2c108150 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d333ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c19ff10 con 0x7f3d2c074b30 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d333ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c1a00a0 con 0x7f3d2c073550 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d2c073550 0x7f3d2c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3199c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d2c108150 0x7f3d2c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3119b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d2c073550 0x7f3d2c198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:50570/0 (socket says 172.21.15.8:50570) 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3119b700 1 -- 172.21.15.8:0/370824110 learned_addr learned my addr 172.21.15.8:0/370824110 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:59.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3099a700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d2c074b30 0x7f3d2c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3119b700 1 -- 172.21.15.8:0/370824110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d2c074b30 msgr2=0x7f3d2c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3119b700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d2c074b30 0x7f3d2c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3119b700 1 -- 172.21.15.8:0/370824110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d2c108150 msgr2=0x7f3d2c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3119b700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d2c108150 0x7f3d2c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3119b700 1 -- 172.21.15.8:0/370824110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d280095f0 con 0x7f3d2c073550 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3199c700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d2c108150 0x7f3d2c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.170+0000 7f3d3099a700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d2c074b30 0x7f3d2c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.171+0000 7f3d3119b700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d2c073550 0x7f3d2c198250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d28005720 tx=0x7f3d28009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.171+0000 7f3d227fc700 1 -- 172.21.15.8:0/370824110 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2801d070 con 0x7f3d2c073550 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.171+0000 7f3d227fc700 1 -- 172.21.15.8:0/370824110 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d28021410 con 0x7f3d2c073550 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.171+0000 7f3d333ff700 1 -- 172.21.15.8:0/370824110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2c10de50 con 0x7f3d2c073550 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.171+0000 7f3d227fc700 1 -- 172.21.15.8:0/370824110 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28017670 con 0x7f3d2c073550 2023-11-27T15:24:59.176 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.171+0000 7f3d333ff700 1 -- 172.21.15.8:0/370824110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c10dcc0 con 0x7f3d2c073550 2023-11-27T15:24:59.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.172+0000 7f3d333ff700 1 -- 172.21.15.8:0/370824110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c106800 con 0x7f3d2c073550 2023-11-27T15:24:59.177 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.174+0000 7f3d227fc700 1 -- 172.21.15.8:0/370824110 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f3d28021580 con 0x7f3d2c073550 2023-11-27T15:24:59.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.175+0000 7f3d227fc700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3d180a11e0 0x7f3d180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:59.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.176+0000 7f3d227fc700 1 -- 172.21.15.8:0/370824110 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3d280ce1e0 con 0x7f3d2c073550 2023-11-27T15:24:59.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.176+0000 7f3d3099a700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3d180a11e0 0x7f3d180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:59.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.176+0000 7f3d3099a700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3d180a11e0 0x7f3d180a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c074850 tx=0x7f3d1c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:59.181 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.178+0000 7f3d227fc700 1 -- 172.21.15.8:0/370824110 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3d280974f0 con 0x7f3d2c073550 2023-11-27T15:24:59.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.390+0000 7f3d333ff700 1 -- 172.21.15.8:0/370824110 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d2c067a50 con 0x7f3d2c073550 2023-11-27T15:24:59.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.390+0000 7f3d227fc700 1 -- 172.21.15.8:0/370824110 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d280300a0 con 0x7f3d2c073550 2023-11-27T15:24:59.394 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:24:59.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 -- 172.21.15.8:0/370824110 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3d180a11e0 msgr2=0x7f3d180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:59.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3d180a11e0 0x7f3d180a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c074850 tx=0x7f3d1c009260 comp rx=0 tx=0).stop 2023-11-27T15:24:59.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 -- 172.21.15.8:0/370824110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d2c073550 msgr2=0x7f3d2c198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:59.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d2c073550 0x7f3d2c198250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d28005720 tx=0x7f3d28009580 comp rx=0 tx=0).stop 2023-11-27T15:24:59.397 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 -- 172.21.15.8:0/370824110 shutdown_connections 2023-11-27T15:24:59.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3d180a11e0 0x7f3d180a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3d2c073550 0x7f3d2c198250 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d2c108150 0x7f3d2c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 --2- 172.21.15.8:0/370824110 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3d2c074b30 0x7f3d2c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 -- 172.21.15.8:0/370824110 >> 172.21.15.8:0/370824110 conn(0x7f3d2c0fb870 msgr2=0x7f3d2c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:59.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 -- 172.21.15.8:0/370824110 shutdown_connections 2023-11-27T15:24:59.398 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.393+0000 7f3d333ff700 1 -- 172.21.15.8:0/370824110 wait complete. 2023-11-27T15:24:59.416 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:24:59.625 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.622+0000 7f9e6ddff700 1 -- 172.21.15.8:0/608060579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e68101e10 msgr2=0x7f9e681086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.622+0000 7f9e6ddff700 1 --2- 172.21.15.8:0/608060579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e68101e10 0x7f9e681086d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000b0a0 tx=0x7f9e6000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.622+0000 7f9e6ddff700 1 -- 172.21.15.8:0/608060579 shutdown_connections 2023-11-27T15:24:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.622+0000 7f9e6ddff700 1 --2- 172.21.15.8:0/608060579 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68100290 0x7f9e681006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.622+0000 7f9e6ddff700 1 --2- 172.21.15.8:0/608060579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e68101e10 0x7f9e681086d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.622+0000 7f9e6ddff700 1 --2- 172.21.15.8:0/608060579 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68101480 0x7f9e681018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.622+0000 7f9e6ddff700 1 -- 172.21.15.8:0/608060579 >> 172.21.15.8:0/608060579 conn(0x7f9e680fb870 msgr2=0x7f9e680fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.622+0000 7f9e6ddff700 1 -- 172.21.15.8:0/608060579 shutdown_connections 2023-11-27T15:24:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.622+0000 7f9e6ddff700 1 -- 172.21.15.8:0/608060579 wait complete. 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.623+0000 7f9e6ddff700 1 Processor -- start 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.623+0000 7f9e6ddff700 1 -- start start 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.623+0000 7f9e6ddff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68100290 0x7f9e680748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.623+0000 7f9e6ddff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68101480 0x7f9e68072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.623+0000 7f9e6ddff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e68101e10 0x7f9e68073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.623+0000 7f9e6ddff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68073bc0 con 0x7f9e68101e10 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.623+0000 7f9e6ddff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68073d30 con 0x7f9e68100290 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.623+0000 7f9e6ddff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e681a0160 con 0x7f9e68101480 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e67fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e68101e10 0x7f9e68073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e67fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e68101e10 0x7f9e68073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45348/0 (socket says 172.21.15.8:45348) 2023-11-27T15:24:59.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e67fff700 1 -- 172.21.15.8:0/2553868934 learned_addr learned my addr 172.21.15.8:0/2553868934 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e677fe700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68100290 0x7f9e680748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e5ffff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68101480 0x7f9e68072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e67fff700 1 -- 172.21.15.8:0/2553868934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68101480 msgr2=0x7f9e68072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e67fff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68101480 0x7f9e68072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e67fff700 1 -- 172.21.15.8:0/2553868934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68100290 msgr2=0x7f9e680748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e67fff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68100290 0x7f9e680748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e67fff700 1 -- 172.21.15.8:0/2553868934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e60009d00 con 0x7f9e68101e10 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e5ffff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68101480 0x7f9e68072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.624+0000 7f9e67fff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e68101e10 0x7f9e68073420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000d010 tx=0x7f9e6000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.625+0000 7f9e657fa700 1 -- 172.21.15.8:0/2553868934 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6000e040 con 0x7f9e68101e10 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.625+0000 7f9e657fa700 1 -- 172.21.15.8:0/2553868934 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e60003e10 con 0x7f9e68101e10 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.625+0000 7f9e6ddff700 1 -- 172.21.15.8:0/2553868934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6810bc70 con 0x7f9e68101e10 2023-11-27T15:24:59.630 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.625+0000 7f9e657fa700 1 -- 172.21.15.8:0/2553868934 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e600052a0 con 0x7f9e68101e10 2023-11-27T15:24:59.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.625+0000 7f9e6ddff700 1 -- 172.21.15.8:0/2553868934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6810bae0 con 0x7f9e68101e10 2023-11-27T15:24:59.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.626+0000 7f9e6ddff700 1 -- 172.21.15.8:0/2553868934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e68107840 con 0x7f9e68101e10 2023-11-27T15:24:59.631 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.628+0000 7f9e657fa700 1 -- 172.21.15.8:0/2553868934 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9e60005400 con 0x7f9e68101e10 2023-11-27T15:24:59.632 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.629+0000 7f9e657fa700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e540a1260 0x7f9e540a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:59.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.629+0000 7f9e657fa700 1 -- 172.21.15.8:0/2553868934 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9e600cdf50 con 0x7f9e68101e10 2023-11-27T15:24:59.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.630+0000 7f9e677fe700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e540a1260 0x7f9e540a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:59.633 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.630+0000 7f9e677fe700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e540a1260 0x7f9e540a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9e681012e0 tx=0x7f9e58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:59.637 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.633+0000 7f9e657fa700 1 -- 172.21.15.8:0/2553868934 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9e60097260 con 0x7f9e68101e10 2023-11-27T15:24:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:59 smithi060 ceph-mon[131518]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:59 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2378844180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:24:59 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/370824110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:24:59.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.849+0000 7f9e6ddff700 1 -- 172.21.15.8:0/2553868934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e68067a50 con 0x7f9e68101e10 2023-11-27T15:24:59.853 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.850+0000 7f9e657fa700 1 -- 172.21.15.8:0/2553868934 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e6009a880 con 0x7f9e68101e10 2023-11-27T15:24:59.854 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:24:59.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 -- 172.21.15.8:0/2553868934 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e540a1260 msgr2=0x7f9e540a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:59.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e540a1260 0x7f9e540a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9e681012e0 tx=0x7f9e58009260 comp rx=0 tx=0).stop 2023-11-27T15:24:59.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 -- 172.21.15.8:0/2553868934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e68101e10 msgr2=0x7f9e68073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:59.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e68101e10 0x7f9e68073420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000d010 tx=0x7f9e6000bc80 comp rx=0 tx=0).stop 2023-11-27T15:24:59.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 -- 172.21.15.8:0/2553868934 shutdown_connections 2023-11-27T15:24:59.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e68101e10 0x7f9e68073420 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.856 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e68100290 0x7f9e680748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e540a1260 0x7f9e540a3710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 --2- 172.21.15.8:0/2553868934 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e68101480 0x7f9e68072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:59.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 -- 172.21.15.8:0/2553868934 >> 172.21.15.8:0/2553868934 conn(0x7f9e680fb870 msgr2=0x7f9e68104980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:59.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 -- 172.21.15.8:0/2553868934 shutdown_connections 2023-11-27T15:24:59.857 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:24:59.852+0000 7f9e6ddff700 1 -- 172.21.15.8:0/2553868934 wait complete. 2023-11-27T15:24:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:59 smithi008 ceph-mon[128788]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:59 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2378844180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:24:59 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/370824110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:24:59.875 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:24:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:59 smithi029 ceph-mon[131166]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:24:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:59 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2378844180' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:24:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:24:59 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/370824110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:25:00.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.078+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/945447764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2b4100290 msgr2=0x7fe2b41006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.082 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.078+0000 7fe2bc5ff700 1 --2- 172.21.15.8:0/945447764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2b4100290 0x7fe2b41006a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00098f0 tx=0x7fe2b0009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:00.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.078+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/945447764 shutdown_connections 2023-11-27T15:25:00.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.078+0000 7fe2bc5ff700 1 --2- 172.21.15.8:0/945447764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b4101e10 0x7fe2b41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.078+0000 7fe2bc5ff700 1 --2- 172.21.15.8:0/945447764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2b4100290 0x7fe2b41006a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.078+0000 7fe2bc5ff700 1 --2- 172.21.15.8:0/945447764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2b4101480 0x7fe2b41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.078+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/945447764 >> 172.21.15.8:0/945447764 conn(0x7fe2b40fb870 msgr2=0x7fe2b40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:00.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.078+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/945447764 shutdown_connections 2023-11-27T15:25:00.083 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.078+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/945447764 wait complete. 2023-11-27T15:25:00.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.079+0000 7fe2bc5ff700 1 Processor -- start 2023-11-27T15:25:00.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.079+0000 7fe2bc5ff700 1 -- start start 2023-11-27T15:25:00.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2bc5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b4100290 0x7fe2b40748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2bc5ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2b4101480 0x7fe2b4072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2bc5ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2b4101e10 0x7fe2b4073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2bc5ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b4073bc0 con 0x7fe2b4101e10 2023-11-27T15:25:00.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2bc5ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b4073d30 con 0x7fe2b4101480 2023-11-27T15:25:00.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2bc5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b41a0120 con 0x7fe2b4100290 2023-11-27T15:25:00.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2b9b9a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2b4101480 0x7fe2b4072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2bab9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2b4101e10 0x7fe2b4073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2b9b9a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2b4101480 0x7fe2b4072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:44906/0 (socket says 172.21.15.8:44906) 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2b9b9a700 1 -- 172.21.15.8:0/3680582 learned_addr learned my addr 172.21.15.8:0/3680582 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.080+0000 7fe2ba39b700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b4100290 0x7fe2b40748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2b9b9a700 1 -- 172.21.15.8:0/3680582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b4100290 msgr2=0x7fe2b40748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2b9b9a700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b4100290 0x7fe2b40748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2b9b9a700 1 -- 172.21.15.8:0/3680582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2b4101e10 msgr2=0x7fe2b4073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2b9b9a700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2b4101e10 0x7fe2b4073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2b9b9a700 1 -- 172.21.15.8:0/3680582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a40095f0 con 0x7fe2b4101480 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2ba39b700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b4100290 0x7fe2b40748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2b9b9a700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2b4101480 0x7fe2b4072ee0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400ec10 tx=0x7fe2a400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2ab7fe700 1 -- 172.21.15.8:0/3680582 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400cd40 con 0x7fe2b4101480 2023-11-27T15:25:00.086 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/3680582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b0009520 con 0x7fe2b4101480 2023-11-27T15:25:00.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2ab7fe700 1 -- 172.21.15.8:0/3680582 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2a40044e0 con 0x7fe2b4101480 2023-11-27T15:25:00.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2ab7fe700 1 -- 172.21.15.8:0/3680582 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4010430 con 0x7fe2b4101480 2023-11-27T15:25:00.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.081+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/3680582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b410bc40 con 0x7fe2b4101480 2023-11-27T15:25:00.087 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.082+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/3680582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b4107980 con 0x7fe2b4101480 2023-11-27T15:25:00.088 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.084+0000 7fe2ab7fe700 1 -- 172.21.15.8:0/3680582 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe2a4003680 con 0x7fe2b4101480 2023-11-27T15:25:00.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.086+0000 7fe2ab7fe700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2a00a1260 0x7fe2a00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.086+0000 7fe2ab7fe700 1 -- 172.21.15.8:0/3680582 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe2a4014070 con 0x7fe2b4101480 2023-11-27T15:25:00.089 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.086+0000 7fe2ba39b700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2a00a1260 0x7fe2a00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.090 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.087+0000 7fe2ba39b700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2a00a1260 0x7fe2a00a3710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00098c0 tx=0x7fe2b0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:00.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.088+0000 7fe2ab7fe700 1 -- 172.21.15.8:0/3680582 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe2a4096770 con 0x7fe2b4101480 2023-11-27T15:25:00.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.297+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/3680582 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2b4067a50 con 0x7fe2b4101480 2023-11-27T15:25:00.302 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.298+0000 7fe2ab7fe700 1 -- 172.21.15.8:0/3680582 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2a4099d90 con 0x7fe2b4101480 2023-11-27T15:25:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-11-27T15:25:00.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.300+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/3680582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2a00a1260 msgr2=0x7fe2a00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.300+0000 7fe2bc5ff700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2a00a1260 0x7fe2a00a3710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00098c0 tx=0x7fe2b0019040 comp rx=0 tx=0).stop 2023-11-27T15:25:00.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.300+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/3680582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2b4101480 msgr2=0x7fe2b4072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.304 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.300+0000 7fe2bc5ff700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2b4101480 0x7fe2b4072ee0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400ec10 tx=0x7fe2a400ef20 comp rx=0 tx=0).stop 2023-11-27T15:25:00.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.300+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/3680582 shutdown_connections 2023-11-27T15:25:00.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.300+0000 7fe2bc5ff700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe2a00a1260 0x7fe2a00a3710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.300+0000 7fe2bc5ff700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2b4100290 0x7fe2b40748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.300+0000 7fe2bc5ff700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe2b4101e10 0x7fe2b4073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.300+0000 7fe2bc5ff700 1 --2- 172.21.15.8:0/3680582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2b4101480 0x7fe2b4072ee0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.301+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/3680582 >> 172.21.15.8:0/3680582 conn(0x7fe2b40fb870 msgr2=0x7fe2b4104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:00.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.301+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/3680582 shutdown_connections 2023-11-27T15:25:00.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.301+0000 7fe2bc5ff700 1 -- 172.21.15.8:0/3680582 wait complete. 2023-11-27T15:25:00.323 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:25:00.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.529+0000 7fc219bff700 1 -- 172.21.15.8:0/4140565438 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc214101480 msgr2=0x7fc2141018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.529+0000 7fc219bff700 1 --2- 172.21.15.8:0/4140565438 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc214101480 0x7fc2141018d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009990 tx=0x7fc1fc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.529+0000 7fc219bff700 1 -- 172.21.15.8:0/4140565438 shutdown_connections 2023-11-27T15:25:00.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.529+0000 7fc219bff700 1 --2- 172.21.15.8:0/4140565438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc214101e10 0x7fc2141086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.529+0000 7fc219bff700 1 --2- 172.21.15.8:0/4140565438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc214100290 0x7fc2141006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.529+0000 7fc219bff700 1 --2- 172.21.15.8:0/4140565438 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc214101480 0x7fc2141018d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.529+0000 7fc219bff700 1 -- 172.21.15.8:0/4140565438 >> 172.21.15.8:0/4140565438 conn(0x7fc2140fb870 msgr2=0x7fc2140fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:00.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.529+0000 7fc219bff700 1 -- 172.21.15.8:0/4140565438 shutdown_connections 2023-11-27T15:25:00.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.529+0000 7fc219bff700 1 -- 172.21.15.8:0/4140565438 wait complete. 2023-11-27T15:25:00.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.530+0000 7fc219bff700 1 Processor -- start 2023-11-27T15:25:00.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.530+0000 7fc219bff700 1 -- start start 2023-11-27T15:25:00.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.530+0000 7fc219bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc214100290 0x7fc214198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.530+0000 7fc219bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc214101480 0x7fc2141987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.530+0000 7fc219bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc214101e10 0x7fc21419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.530+0000 7fc219bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21419fde0 con 0x7fc214101e10 2023-11-27T15:25:00.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.530+0000 7fc219bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21419ff20 con 0x7fc214100290 2023-11-27T15:25:00.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.530+0000 7fc219bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2141a00b0 con 0x7fc214101480 2023-11-27T15:25:00.535 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.531+0000 7fc213fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc214101e10 0x7fc21419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.531+0000 7fc2137fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc214100290 0x7fc214198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.531+0000 7fc2137fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc214100290 0x7fc214198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:44930/0 (socket says 172.21.15.8:44930) 2023-11-27T15:25:00.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.531+0000 7fc20bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc214101480 0x7fc2141987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.531+0000 7fc2137fe700 1 -- 172.21.15.8:0/3875706120 learned_addr learned my addr 172.21.15.8:0/3875706120 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:00.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.531+0000 7fc2137fe700 1 -- 172.21.15.8:0/3875706120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc214101480 msgr2=0x7fc2141987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.531+0000 7fc2137fe700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc214101480 0x7fc2141987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.531+0000 7fc2137fe700 1 -- 172.21.15.8:0/3875706120 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc214101e10 msgr2=0x7fc21419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.531+0000 7fc2137fe700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc214101e10 0x7fc21419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.531+0000 7fc2137fe700 1 -- 172.21.15.8:0/3875706120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc0095f0 con 0x7fc214100290 2023-11-27T15:25:00.536 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.532+0000 7fc2137fe700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc214100290 0x7fc214198260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc20400e970 tx=0x7fc20400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:00.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.532+0000 7fc2117fa700 1 -- 172.21.15.8:0/3875706120 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204009790 con 0x7fc214100290 2023-11-27T15:25:00.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.532+0000 7fc2117fa700 1 -- 172.21.15.8:0/3875706120 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc20400c7e0 con 0x7fc214100290 2023-11-27T15:25:00.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.532+0000 7fc219bff700 1 -- 172.21.15.8:0/3875706120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21410bc50 con 0x7fc214100290 2023-11-27T15:25:00.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.532+0000 7fc2117fa700 1 -- 172.21.15.8:0/3875706120 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204003c70 con 0x7fc214100290 2023-11-27T15:25:00.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.532+0000 7fc219bff700 1 -- 172.21.15.8:0/3875706120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21410bac0 con 0x7fc214100290 2023-11-27T15:25:00.537 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.533+0000 7fc219bff700 1 -- 172.21.15.8:0/3875706120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2141928e0 con 0x7fc214100290 2023-11-27T15:25:00.541 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.538+0000 7fc2117fa700 1 -- 172.21.15.8:0/3875706120 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc20400cc50 con 0x7fc214100290 2023-11-27T15:25:00.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.540+0000 7fc2117fa700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc2000a11e0 0x7fc2000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.540+0000 7fc2117fa700 1 -- 172.21.15.8:0/3875706120 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc204014070 con 0x7fc214100290 2023-11-27T15:25:00.543 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.540+0000 7fc20bfff700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc2000a11e0 0x7fc2000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.540+0000 7fc20bfff700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc2000a11e0 0x7fc2000a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00b2e0 tx=0x7fc1fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:00.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.541+0000 7fc2117fa700 1 -- 172.21.15.8:0/3875706120 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc204096380 con 0x7fc214100290 2023-11-27T15:25:00.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:00 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2553868934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:00.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:00 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3680582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:00.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.755+0000 7fc219bff700 1 -- 172.21.15.8:0/3875706120 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc214067a50 con 0x7fc214100290 2023-11-27T15:25:00.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.756+0000 7fc2117fa700 1 -- 172.21.15.8:0/3875706120 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc204003dd0 con 0x7fc214100290 2023-11-27T15:25:00.759 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:25:00.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 -- 172.21.15.8:0/3875706120 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc2000a11e0 msgr2=0x7fc2000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc2000a11e0 0x7fc2000a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00b2e0 tx=0x7fc1fc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 -- 172.21.15.8:0/3875706120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc214100290 msgr2=0x7fc214198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc214100290 0x7fc214198260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc20400e970 tx=0x7fc20400ec80 comp rx=0 tx=0).stop 2023-11-27T15:25:00.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 -- 172.21.15.8:0/3875706120 shutdown_connections 2023-11-27T15:25:00.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc214101e10 0x7fc21419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc214100290 0x7fc214198260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc2000a11e0 0x7fc2000a3690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 --2- 172.21.15.8:0/3875706120 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc214101480 0x7fc2141987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 -- 172.21.15.8:0/3875706120 >> 172.21.15.8:0/3875706120 conn(0x7fc2140fb870 msgr2=0x7fc214106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:00.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 -- 172.21.15.8:0/3875706120 shutdown_connections 2023-11-27T15:25:00.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.758+0000 7fc219bff700 1 -- 172.21.15.8:0/3875706120 wait complete. 2023-11-27T15:25:00.780 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:25:00.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:00 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2553868934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:00.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:00 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3680582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:00.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:00 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2553868934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:00.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:00 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3680582' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:00.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.980+0000 7ff5a23ff700 1 -- 172.21.15.8:0/1456638736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff59c1014c0 msgr2=0x7ff59c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.980+0000 7ff5a23ff700 1 --2- 172.21.15.8:0/1456638736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff59c1014c0 0x7ff59c101910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff590009990 tx=0x7ff590009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.980+0000 7ff5a23ff700 1 -- 172.21.15.8:0/1456638736 shutdown_connections 2023-11-27T15:25:00.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.980+0000 7ff5a23ff700 1 --2- 172.21.15.8:0/1456638736 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff59c1002d0 0x7ff59c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.980+0000 7ff5a23ff700 1 --2- 172.21.15.8:0/1456638736 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff59c101e50 0x7ff59c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.980+0000 7ff5a23ff700 1 --2- 172.21.15.8:0/1456638736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff59c1014c0 0x7ff59c101910 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.980+0000 7ff5a23ff700 1 -- 172.21.15.8:0/1456638736 >> 172.21.15.8:0/1456638736 conn(0x7ff59c0fb8b0 msgr2=0x7ff59c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:00.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.980+0000 7ff5a23ff700 1 -- 172.21.15.8:0/1456638736 shutdown_connections 2023-11-27T15:25:00.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.980+0000 7ff5a23ff700 1 -- 172.21.15.8:0/1456638736 wait complete. 2023-11-27T15:25:00.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.982+0000 7ff5a23ff700 1 Processor -- start 2023-11-27T15:25:00.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.982+0000 7ff5a23ff700 1 -- start start 2023-11-27T15:25:00.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.982+0000 7ff5a23ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff59c1002d0 0x7ff59c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff5a23ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff59c1014c0 0x7ff59c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff5a23ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff59c101e50 0x7ff59c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff5a23ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c19fdc0 con 0x7ff59c1014c0 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff5a23ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c19ff00 con 0x7ff59c101e50 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff5a23ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c1a0090 con 0x7ff59c1002d0 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff59b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff59c1014c0 0x7ff59c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff59bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff59c1002d0 0x7ff59c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff59b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff59c1014c0 0x7ff59c198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46960/0 (socket says 172.21.15.8:46960) 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff59b7fe700 1 -- 172.21.15.8:0/2416751051 learned_addr learned my addr 172.21.15.8:0/2416751051 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff5a099c700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff59c101e50 0x7ff59c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff59b7fe700 1 -- 172.21.15.8:0/2416751051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff59c1002d0 msgr2=0x7ff59c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff59b7fe700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff59c1002d0 0x7ff59c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff59b7fe700 1 -- 172.21.15.8:0/2416751051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff59c101e50 msgr2=0x7ff59c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:00.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff59b7fe700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff59c101e50 0x7ff59c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:00.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff59b7fe700 1 -- 172.21.15.8:0/2416751051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5900095f0 con 0x7ff59c1014c0 2023-11-27T15:25:00.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff5a099c700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff59c101e50 0x7ff59c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:00.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.983+0000 7ff59b7fe700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff59c1014c0 0x7ff59c198780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff59000c010 tx=0x7ff590004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:00.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.984+0000 7ff5997fa700 1 -- 172.21.15.8:0/2416751051 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59001d070 con 0x7ff59c1014c0 2023-11-27T15:25:00.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.984+0000 7ff5997fa700 1 -- 172.21.15.8:0/2416751051 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff590022470 con 0x7ff59c1014c0 2023-11-27T15:25:00.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.984+0000 7ff5a23ff700 1 -- 172.21.15.8:0/2416751051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c10bc80 con 0x7ff59c1014c0 2023-11-27T15:25:00.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.984+0000 7ff5997fa700 1 -- 172.21.15.8:0/2416751051 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590017650 con 0x7ff59c1014c0 2023-11-27T15:25:00.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.984+0000 7ff5a23ff700 1 -- 172.21.15.8:0/2416751051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c10baf0 con 0x7ff59c1014c0 2023-11-27T15:25:00.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.985+0000 7ff5a23ff700 1 -- 172.21.15.8:0/2416751051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c067af0 con 0x7ff59c1014c0 2023-11-27T15:25:00.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.987+0000 7ff5997fa700 1 -- 172.21.15.8:0/2416751051 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff5900225e0 con 0x7ff59c1014c0 2023-11-27T15:25:00.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.989+0000 7ff5997fa700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff5840a1260 0x7ff5840a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:00.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.989+0000 7ff5997fa700 1 -- 172.21.15.8:0/2416751051 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff5900ce8c0 con 0x7ff59c1014c0 2023-11-27T15:25:00.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.989+0000 7ff59bfff700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff5840a1260 0x7ff5840a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:00.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.990+0000 7ff59bfff700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff5840a1260 0x7ff5840a3710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff59c101320 tx=0x7ff58c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:00.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:00.991+0000 7ff5997fa700 1 -- 172.21.15.8:0/2416751051 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff590097c50 con 0x7ff59c1014c0 2023-11-27T15:25:01.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.248+0000 7ff5a23ff700 1 -- 172.21.15.8:0/2416751051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff59c1a0c60 con 0x7ff59c1014c0 2023-11-27T15:25:01.252 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.249+0000 7ff5997fa700 1 -- 172.21.15.8:0/2416751051 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10768 (secure 0 0 0) 0x7ff590027090 con 0x7ff59c1014c0 2023-11-27T15:25:01.252 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:01.253 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:23:27.167939+0000","modified":"2023-11-27T15:23:59.241472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24595,"mds_1":24593,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24593":{"gid":24593,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24595":{"gid":24595,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:25:01.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 -- 172.21.15.8:0/2416751051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff5840a1260 msgr2=0x7ff5840a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:01.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff5840a1260 0x7ff5840a3710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff59c101320 tx=0x7ff58c009260 comp rx=0 tx=0).stop 2023-11-27T15:25:01.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 -- 172.21.15.8:0/2416751051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff59c1014c0 msgr2=0x7ff59c198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:01.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff59c1014c0 0x7ff59c198780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff59000c010 tx=0x7ff590004730 comp rx=0 tx=0).stop 2023-11-27T15:25:01.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 -- 172.21.15.8:0/2416751051 shutdown_connections 2023-11-27T15:25:01.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff5840a1260 0x7ff5840a3710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.255 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff59c1002d0 0x7ff59c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff59c101e50 0x7ff59c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 --2- 172.21.15.8:0/2416751051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff59c1014c0 0x7ff59c198780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 -- 172.21.15.8:0/2416751051 >> 172.21.15.8:0/2416751051 conn(0x7ff59c0fb8b0 msgr2=0x7ff59c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:01.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 -- 172.21.15.8:0/2416751051 shutdown_connections 2023-11-27T15:25:01.256 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.251+0000 7ff5a23ff700 1 -- 172.21.15.8:0/2416751051 wait complete. 2023-11-27T15:25:01.256 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2023-11-27T15:25:01.274 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:25:01.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.475+0000 7f8c435ff700 1 -- 172.21.15.8:0/3021414161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c3c1014c0 msgr2=0x7f8c3c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:01.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.475+0000 7f8c435ff700 1 --2- 172.21.15.8:0/3021414161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c3c1014c0 0x7f8c3c101910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009990 tx=0x7f8c2c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.475+0000 7f8c435ff700 1 -- 172.21.15.8:0/3021414161 shutdown_connections 2023-11-27T15:25:01.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.475+0000 7f8c435ff700 1 --2- 172.21.15.8:0/3021414161 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c101e50 0x7f8c3c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.475+0000 7f8c435ff700 1 --2- 172.21.15.8:0/3021414161 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c3c1002d0 0x7f8c3c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.475+0000 7f8c435ff700 1 --2- 172.21.15.8:0/3021414161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c3c1014c0 0x7f8c3c101910 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.475+0000 7f8c435ff700 1 -- 172.21.15.8:0/3021414161 >> 172.21.15.8:0/3021414161 conn(0x7f8c3c0fb8b0 msgr2=0x7f8c3c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:01.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.475+0000 7f8c435ff700 1 -- 172.21.15.8:0/3021414161 shutdown_connections 2023-11-27T15:25:01.479 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.475+0000 7f8c435ff700 1 -- 172.21.15.8:0/3021414161 wait complete. 2023-11-27T15:25:01.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.476+0000 7f8c435ff700 1 Processor -- start 2023-11-27T15:25:01.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.476+0000 7f8c435ff700 1 -- start start 2023-11-27T15:25:01.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c435ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c3c1002d0 0x7f8c3c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:01.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c435ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c3c1014c0 0x7f8c3c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c435ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c101e50 0x7f8c3c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c435ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c19fe20 con 0x7f8c3c1002d0 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c435ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c19ff60 con 0x7f8c3c1014c0 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c435ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c1a00f0 con 0x7f8c3c101e50 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c4139b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c3c1002d0 0x7f8c3c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c4139b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c3c1002d0 0x7f8c3c1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46974/0 (socket says 172.21.15.8:46974) 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c4139b700 1 -- 172.21.15.8:0/2256979065 learned_addr learned my addr 172.21.15.8:0/2256979065 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c40b9a700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c3c1014c0 0x7f8c3c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c41b9c700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c101e50 0x7f8c3c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c4139b700 1 -- 172.21.15.8:0/2256979065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c101e50 msgr2=0x7f8c3c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:01.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c4139b700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c101e50 0x7f8c3c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c4139b700 1 -- 172.21.15.8:0/2256979065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c3c1014c0 msgr2=0x7f8c3c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:01.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c4139b700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c3c1014c0 0x7f8c3c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.477+0000 7f8c4139b700 1 -- 172.21.15.8:0/2256979065 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0095f0 con 0x7f8c3c1002d0 2023-11-27T15:25:01.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.478+0000 7f8c4139b700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c3c1002d0 0x7f8c3c1982a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800e970 tx=0x7f8c3800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:01.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.478+0000 7f8c327fc700 1 -- 172.21.15.8:0/2256979065 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38009790 con 0x7f8c3c1002d0 2023-11-27T15:25:01.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.478+0000 7f8c327fc700 1 -- 172.21.15.8:0/2256979065 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3800c7e0 con 0x7f8c3c1002d0 2023-11-27T15:25:01.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.478+0000 7f8c327fc700 1 -- 172.21.15.8:0/2256979065 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38003c70 con 0x7f8c3c1002d0 2023-11-27T15:25:01.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.478+0000 7f8c435ff700 1 -- 172.21.15.8:0/2256979065 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c10bc80 con 0x7f8c3c1002d0 2023-11-27T15:25:01.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.478+0000 7f8c435ff700 1 -- 172.21.15.8:0/2256979065 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c10baf0 con 0x7f8c3c1002d0 2023-11-27T15:25:01.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.479+0000 7f8c435ff700 1 -- 172.21.15.8:0/2256979065 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c1a07f0 con 0x7f8c3c1002d0 2023-11-27T15:25:01.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.481+0000 7f8c327fc700 1 -- 172.21.15.8:0/2256979065 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8c3800cc50 con 0x7f8c3c1002d0 2023-11-27T15:25:01.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.482+0000 7f8c327fc700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8c280a11e0 0x7f8c280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:01.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.482+0000 7f8c327fc700 1 -- 172.21.15.8:0/2256979065 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c38014070 con 0x7f8c3c1002d0 2023-11-27T15:25:01.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.483+0000 7f8c40b9a700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8c280a11e0 0x7f8c280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:01.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.483+0000 7f8c40b9a700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8c280a11e0 0x7f8c280a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00b2e0 tx=0x7f8c2c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:01.489 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.486+0000 7f8c327fc700 1 -- 172.21.15.8:0/2256979065 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8c38096220 con 0x7f8c3c1002d0 2023-11-27T15:25:01.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.698+0000 7f8c435ff700 1 -- 172.21.15.8:0/2256979065 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c3c1a0da0 con 0x7f8c3c1002d0 2023-11-27T15:25:01.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.699+0000 7f8c327fc700 1 -- 172.21.15.8:0/2256979065 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24344 (secure 0 0 0) 0x7f8c38003dd0 con 0x7f8c3c1002d0 2023-11-27T15:25:01.703 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:01.704 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":90,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:24:40.249579+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:25:01.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 -- 172.21.15.8:0/2256979065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8c280a11e0 msgr2=0x7f8c280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:01.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8c280a11e0 0x7f8c280a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00b2e0 tx=0x7f8c2c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 -- 172.21.15.8:0/2256979065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c3c1002d0 msgr2=0x7f8c3c1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:01.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c3c1002d0 0x7f8c3c1982a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800e970 tx=0x7f8c3800ec80 comp rx=0 tx=0).stop 2023-11-27T15:25:01.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 -- 172.21.15.8:0/2256979065 shutdown_connections 2023-11-27T15:25:01.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8c280a11e0 0x7f8c280a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c3c101e50 0x7f8c3c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c3c1002d0 0x7f8c3c1982a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.706 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 --2- 172.21.15.8:0/2256979065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c3c1014c0 0x7f8c3c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 -- 172.21.15.8:0/2256979065 >> 172.21.15.8:0/2256979065 conn(0x7f8c3c0fb8b0 msgr2=0x7f8c3c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:01.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 -- 172.21.15.8:0/2256979065 shutdown_connections 2023-11-27T15:25:01.707 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.702+0000 7f8c435ff700 1 -- 172.21.15.8:0/2256979065 wait complete. 2023-11-27T15:25:01.725 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:23:25.166148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:23:26.584539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:23:30.130965+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:25:01.726 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:25:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:01 smithi060 ceph-mon[131518]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:25:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:01 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3875706120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:01 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2416751051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:01.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:01 smithi008 ceph-mon[128788]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:25:01.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:01 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3875706120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:01.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:01 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2416751051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:01.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:01 smithi029 ceph-mon[131166]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:25:01.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:01 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3875706120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:01.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:01 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2416751051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:01.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.929+0000 7f75f45ff700 1 -- 172.21.15.8:0/72150573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75ec108150 msgr2=0x7f75ec10a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:01.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.929+0000 7f75f45ff700 1 --2- 172.21.15.8:0/72150573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75ec108150 0x7f75ec10a5d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f75e400b0a0 tx=0x7f75e400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.929+0000 7f75f45ff700 1 -- 172.21.15.8:0/72150573 shutdown_connections 2023-11-27T15:25:01.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.929+0000 7f75f45ff700 1 --2- 172.21.15.8:0/72150573 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec074b30 0x7f75ec072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.929+0000 7f75f45ff700 1 --2- 172.21.15.8:0/72150573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75ec108150 0x7f75ec10a5d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.929+0000 7f75f45ff700 1 --2- 172.21.15.8:0/72150573 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75ec073550 0x7f75ec0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.929+0000 7f75f45ff700 1 -- 172.21.15.8:0/72150573 >> 172.21.15.8:0/72150573 conn(0x7f75ec0fb870 msgr2=0x7f75ec0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:01.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.929+0000 7f75f45ff700 1 -- 172.21.15.8:0/72150573 shutdown_connections 2023-11-27T15:25:01.933 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.929+0000 7f75f45ff700 1 -- 172.21.15.8:0/72150573 wait complete. 2023-11-27T15:25:01.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.930+0000 7f75f45ff700 1 Processor -- start 2023-11-27T15:25:01.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.930+0000 7f75f45ff700 1 -- start start 2023-11-27T15:25:01.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.930+0000 7f75f45ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec073550 0x7f75ec1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:01.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f45ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75ec074b30 0x7f75ec198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:01.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f45ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75ec108150 0x7f75ec19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:01.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f45ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec19fe60 con 0x7f75ec108150 2023-11-27T15:25:01.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f45ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec19ffa0 con 0x7f75ec074b30 2023-11-27T15:25:01.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f45ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1a0130 con 0x7f75ec073550 2023-11-27T15:25:01.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f2b9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75ec108150 0x7f75ec19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f2b9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75ec108150 0x7f75ec19f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46992/0 (socket says 172.21.15.8:46992) 2023-11-27T15:25:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f2b9c700 1 -- 172.21.15.8:0/386965079 learned_addr learned my addr 172.21.15.8:0/386965079 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f239b700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec073550 0x7f75ec1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f1b9a700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75ec074b30 0x7f75ec198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f2b9c700 1 -- 172.21.15.8:0/386965079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec073550 msgr2=0x7f75ec1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f2b9c700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec073550 0x7f75ec1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f2b9c700 1 -- 172.21.15.8:0/386965079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75ec074b30 msgr2=0x7f75ec198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f2b9c700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75ec074b30 0x7f75ec198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.931+0000 7f75f2b9c700 1 -- 172.21.15.8:0/386965079 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e4009d00 con 0x7f75ec108150 2023-11-27T15:25:01.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.932+0000 7f75f2b9c700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75ec108150 0x7f75ec19f890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009fd0 tx=0x7f75e400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.932+0000 7f75e37fe700 1 -- 172.21.15.8:0/386965079 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e400e040 con 0x7f75ec108150 2023-11-27T15:25:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.932+0000 7f75f45ff700 1 -- 172.21.15.8:0/386965079 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec10de70 con 0x7f75ec108150 2023-11-27T15:25:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.932+0000 7f75e37fe700 1 -- 172.21.15.8:0/386965079 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e40094a0 con 0x7f75ec108150 2023-11-27T15:25:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.932+0000 7f75e37fe700 1 -- 172.21.15.8:0/386965079 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e401b6c0 con 0x7f75ec108150 2023-11-27T15:25:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.932+0000 7f75f45ff700 1 -- 172.21.15.8:0/386965079 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec10dce0 con 0x7f75ec108150 2023-11-27T15:25:01.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.933+0000 7f75f45ff700 1 -- 172.21.15.8:0/386965079 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec106800 con 0x7f75ec108150 2023-11-27T15:25:01.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.935+0000 7f75e37fe700 1 -- 172.21.15.8:0/386965079 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f75e401d470 con 0x7f75ec108150 2023-11-27T15:25:01.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.936+0000 7f75e37fe700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f75d80a11e0 0x7f75d80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:01.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.936+0000 7f75e37fe700 1 -- 172.21.15.8:0/386965079 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f75e40cdfe0 con 0x7f75ec108150 2023-11-27T15:25:01.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.937+0000 7f75f239b700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f75d80a11e0 0x7f75d80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:01.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.937+0000 7f75f239b700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f75d80a11e0 0x7f75d80a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f75ec074f00 tx=0x7f75e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:01.943 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:01.940+0000 7f75e37fe700 1 -- 172.21.15.8:0/386965079 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f75e40972f0 con 0x7f75ec108150 2023-11-27T15:25:02.153 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.150+0000 7f75f45ff700 1 -- 172.21.15.8:0/386965079 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75ec061700 con 0x7f75ec108150 2023-11-27T15:25:02.154 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.150+0000 7f75e37fe700 1 -- 172.21.15.8:0/386965079 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75e409a910 con 0x7f75ec108150 2023-11-27T15:25:02.154 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:25:02.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 -- 172.21.15.8:0/386965079 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f75d80a11e0 msgr2=0x7f75d80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f75d80a11e0 0x7f75d80a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f75ec074f00 tx=0x7f75e8009260 comp rx=0 tx=0).stop 2023-11-27T15:25:02.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 -- 172.21.15.8:0/386965079 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75ec108150 msgr2=0x7f75ec19f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75ec108150 0x7f75ec19f890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009fd0 tx=0x7f75e400bce0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 -- 172.21.15.8:0/386965079 shutdown_connections 2023-11-27T15:25:02.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f75d80a11e0 0x7f75d80a3690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f75ec073550 0x7f75ec1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.157 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f75ec108150 0x7f75ec19f890 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 --2- 172.21.15.8:0/386965079 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f75ec074b30 0x7f75ec198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 -- 172.21.15.8:0/386965079 >> 172.21.15.8:0/386965079 conn(0x7f75ec0fb870 msgr2=0x7f75ec0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:02.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 -- 172.21.15.8:0/386965079 shutdown_connections 2023-11-27T15:25:02.158 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.153+0000 7f75f45ff700 1 -- 172.21.15.8:0/386965079 wait complete. 2023-11-27T15:25:02.176 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:25:02.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.381+0000 7f7ce65ff700 1 -- 172.21.15.8:0/1898737270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce0100290 msgr2=0x7f7ce01006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.381+0000 7f7ce65ff700 1 --2- 172.21.15.8:0/1898737270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce0100290 0x7f7ce01006a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009990 tx=0x7f7cd0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.381+0000 7f7ce65ff700 1 -- 172.21.15.8:0/1898737270 shutdown_connections 2023-11-27T15:25:02.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.381+0000 7f7ce65ff700 1 --2- 172.21.15.8:0/1898737270 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce0101e10 0x7f7ce01086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.381+0000 7f7ce65ff700 1 --2- 172.21.15.8:0/1898737270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce0100290 0x7f7ce01006a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.381+0000 7f7ce65ff700 1 --2- 172.21.15.8:0/1898737270 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce0101480 0x7f7ce01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.381+0000 7f7ce65ff700 1 -- 172.21.15.8:0/1898737270 >> 172.21.15.8:0/1898737270 conn(0x7f7ce00fb870 msgr2=0x7f7ce00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:02.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.381+0000 7f7ce65ff700 1 -- 172.21.15.8:0/1898737270 shutdown_connections 2023-11-27T15:25:02.385 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.381+0000 7f7ce65ff700 1 -- 172.21.15.8:0/1898737270 wait complete. 2023-11-27T15:25:02.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.382+0000 7f7ce65ff700 1 Processor -- start 2023-11-27T15:25:02.386 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7ce65ff700 1 -- start start 2023-11-27T15:25:02.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7ce65ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce0100290 0x7f7ce0198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7ce65ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce0101480 0x7f7ce0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7ce65ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce0101e10 0x7f7ce019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7ce65ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce019fda0 con 0x7f7ce0101480 2023-11-27T15:25:02.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7ce65ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce019fee0 con 0x7f7ce0100290 2023-11-27T15:25:02.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7ce65ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce01a0070 con 0x7f7ce0101e10 2023-11-27T15:25:02.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce0100290 0x7f7ce0198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.388 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce0100290 0x7f7ce0198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45016/0 (socket says 172.21.15.8:45016) 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7cdffff700 1 -- 172.21.15.8:0/2423297567 learned_addr learned my addr 172.21.15.8:0/2423297567 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.383+0000 7f7cd7fff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce0101480 0x7f7ce0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7ce4b9c700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce0101e10 0x7f7ce019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7cdffff700 1 -- 172.21.15.8:0/2423297567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce0101e10 msgr2=0x7f7ce019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7cdffff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce0101e10 0x7f7ce019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7cdffff700 1 -- 172.21.15.8:0/2423297567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce0101480 msgr2=0x7f7ce0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7cdffff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce0101480 0x7f7ce0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7cdffff700 1 -- 172.21.15.8:0/2423297567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd00095f0 con 0x7f7ce0100290 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7cdffff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce0100290 0x7f7ce0198220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00098e0 tx=0x7f7cd0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7cddffb700 1 -- 172.21.15.8:0/2423297567 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd001d070 con 0x7f7ce0100290 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7cddffb700 1 -- 172.21.15.8:0/2423297567 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cd00054f0 con 0x7f7ce0100290 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7ce65ff700 1 -- 172.21.15.8:0/2423297567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce010bc40 con 0x7f7ce0100290 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.384+0000 7f7cddffb700 1 -- 172.21.15.8:0/2423297567 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0004ca0 con 0x7f7ce0100290 2023-11-27T15:25:02.389 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.385+0000 7f7ce65ff700 1 -- 172.21.15.8:0/2423297567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce010bab0 con 0x7f7ce0100290 2023-11-27T15:25:02.390 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.385+0000 7f7ce65ff700 1 -- 172.21.15.8:0/2423297567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce0067a50 con 0x7f7ce0100290 2023-11-27T15:25:02.391 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.388+0000 7f7cddffb700 1 -- 172.21.15.8:0/2423297567 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f7cd000fb90 con 0x7f7ce0100290 2023-11-27T15:25:02.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.389+0000 7f7cddffb700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ccc0a11e0 0x7f7ccc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.392 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.389+0000 7f7cddffb700 1 -- 172.21.15.8:0/2423297567 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7cd00ce510 con 0x7f7ce0100290 2023-11-27T15:25:02.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.389+0000 7f7cd7fff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ccc0a11e0 0x7f7ccc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.393 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.390+0000 7f7cd7fff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ccc0a11e0 0x7f7ccc0a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8007c60 tx=0x7f7cc80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:02.394 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.391+0000 7f7cddffb700 1 -- 172.21.15.8:0/2423297567 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7cd0097820 con 0x7f7ce0100290 2023-11-27T15:25:02.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.602+0000 7f7ce65ff700 1 -- 172.21.15.8:0/2423297567 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7ce01a0a30 con 0x7f7ce0100290 2023-11-27T15:25:02.606 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.603+0000 7f7cddffb700 1 -- 172.21.15.8:0/2423297567 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7cd009ae40 con 0x7f7ce0100290 2023-11-27T15:25:02.606 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:25:02.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.605+0000 7f7ce65ff700 1 -- 172.21.15.8:0/2423297567 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ccc0a11e0 msgr2=0x7f7ccc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.605+0000 7f7ce65ff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ccc0a11e0 0x7f7ccc0a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8007c60 tx=0x7f7cc80073d0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.609 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.605+0000 7f7ce65ff700 1 -- 172.21.15.8:0/2423297567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce0100290 msgr2=0x7f7ce0198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.605+0000 7f7ce65ff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce0100290 0x7f7ce0198220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00098e0 tx=0x7f7cd0004730 comp rx=0 tx=0).stop 2023-11-27T15:25:02.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.605+0000 7f7ce65ff700 1 -- 172.21.15.8:0/2423297567 shutdown_connections 2023-11-27T15:25:02.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.605+0000 7f7ce65ff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7ccc0a11e0 0x7f7ccc0a3690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.605+0000 7f7ce65ff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7ce0101e10 0x7f7ce019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.605+0000 7f7ce65ff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7ce0100290 0x7f7ce0198220 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.605+0000 7f7ce65ff700 1 --2- 172.21.15.8:0/2423297567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ce0101480 0x7f7ce0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.605+0000 7f7ce65ff700 1 -- 172.21.15.8:0/2423297567 >> 172.21.15.8:0/2423297567 conn(0x7f7ce00fb870 msgr2=0x7f7ce00fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:02.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.606+0000 7f7ce65ff700 1 -- 172.21.15.8:0/2423297567 shutdown_connections 2023-11-27T15:25:02.610 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.606+0000 7f7ce65ff700 1 -- 172.21.15.8:0/2423297567 wait complete. 2023-11-27T15:25:02.629 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:25:02.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:02 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2256979065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:02.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:02 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/386965079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:25:02.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.833+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1157614374 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4cbc108150 msgr2=0x7f4cbc10a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.833+0000 7f4cc09ff700 1 --2- 172.21.15.8:0/1157614374 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4cbc108150 0x7f4cbc10a5d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400b0a0 tx=0x7f4cb400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.833+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1157614374 shutdown_connections 2023-11-27T15:25:02.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.833+0000 7f4cc09ff700 1 --2- 172.21.15.8:0/1157614374 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4cbc108150 0x7f4cbc10a5d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.833+0000 7f4cc09ff700 1 --2- 172.21.15.8:0/1157614374 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc074b30 0x7f4cbc072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.833+0000 7f4cc09ff700 1 --2- 172.21.15.8:0/1157614374 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cbc073550 0x7f4cbc0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.833+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1157614374 >> 172.21.15.8:0/1157614374 conn(0x7f4cbc0fb870 msgr2=0x7f4cbc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:02.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.833+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1157614374 shutdown_connections 2023-11-27T15:25:02.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.833+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1157614374 wait complete. 2023-11-27T15:25:02.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.834+0000 7f4cc09ff700 1 Processor -- start 2023-11-27T15:25:02.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.834+0000 7f4cc09ff700 1 -- start start 2023-11-27T15:25:02.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.834+0000 7f4cc09ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cbc073550 0x7f4cbc198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.834+0000 7f4cc09ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4cbc074b30 0x7f4cbc198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.834+0000 7f4cc09ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc108150 0x7f4cbc19f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.834+0000 7f4cc09ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc19fea0 con 0x7f4cbc074b30 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.834+0000 7f4cc09ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc19ffe0 con 0x7f4cbc108150 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.834+0000 7f4cc09ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cbc1a0170 con 0x7f4cbc073550 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cb3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4cbc074b30 0x7f4cbc198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cba59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cbc073550 0x7f4cbc198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cb3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4cbc074b30 0x7f4cbc198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47020/0 (socket says 172.21.15.8:47020) 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cb3fff700 1 -- 172.21.15.8:0/1215167226 learned_addr learned my addr 172.21.15.8:0/1215167226 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cbad9d700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc108150 0x7f4cbc19f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cb3fff700 1 -- 172.21.15.8:0/1215167226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cbc073550 msgr2=0x7f4cbc198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cb3fff700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cbc073550 0x7f4cbc198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cb3fff700 1 -- 172.21.15.8:0/1215167226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc108150 msgr2=0x7f4cbc19f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cb3fff700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc108150 0x7f4cbc19f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cb3fff700 1 -- 172.21.15.8:0/1215167226 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb4009d00 con 0x7f4cbc074b30 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.835+0000 7f4cb3fff700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4cbc074b30 0x7f4cbc198860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400e950 tx=0x7f4ca400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.836+0000 7f4cb37fe700 1 -- 172.21.15.8:0/1215167226 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4009770 con 0x7f4cbc074b30 2023-11-27T15:25:02.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.836+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1215167226 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cbc10de90 con 0x7f4cbc074b30 2023-11-27T15:25:02.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.836+0000 7f4cb37fe700 1 -- 172.21.15.8:0/1215167226 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ca400c7e0 con 0x7f4cbc074b30 2023-11-27T15:25:02.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.836+0000 7f4cb37fe700 1 -- 172.21.15.8:0/1215167226 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4003cf0 con 0x7f4cbc074b30 2023-11-27T15:25:02.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.836+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1215167226 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cbc10dd00 con 0x7f4cbc074b30 2023-11-27T15:25:02.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.837+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1215167226 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cbc106800 con 0x7f4cbc074b30 2023-11-27T15:25:02.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.839+0000 7f4cb37fe700 1 -- 172.21.15.8:0/1215167226 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f4ca400c950 con 0x7f4cbc074b30 2023-11-27T15:25:02.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.841+0000 7f4cb37fe700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4ca80a1260 0x7f4ca80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.841+0000 7f4cb37fe700 1 -- 172.21.15.8:0/1215167226 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4ca4014070 con 0x7f4cbc074b30 2023-11-27T15:25:02.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.841+0000 7f4cba59c700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4ca80a1260 0x7f4ca80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.842+0000 7f4cba59c700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4ca80a1260 0x7f4ca80a3710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc074f00 tx=0x7f4cac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:02.848 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:02.845+0000 7f4cb37fe700 1 -- 172.21.15.8:0/1215167226 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4ca4096240 con 0x7f4cbc074b30 2023-11-27T15:25:02.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:02 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2256979065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:02.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:02 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/386965079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:02 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2256979065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:02.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:02 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/386965079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:25:03.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.055+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1215167226 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4cbc061700 con 0x7f4cbc074b30 2023-11-27T15:25:03.059 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.056+0000 7f4cb37fe700 1 -- 172.21.15.8:0/1215167226 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ca4099860 con 0x7f4cbc074b30 2023-11-27T15:25:03.059 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-11-27T15:25:03.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.058+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1215167226 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4ca80a1260 msgr2=0x7f4ca80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:03.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4ca80a1260 0x7f4ca80a3710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc074f00 tx=0x7f4cac009260 comp rx=0 tx=0).stop 2023-11-27T15:25:03.062 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1215167226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4cbc074b30 msgr2=0x7f4cbc198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:03.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4cbc074b30 0x7f4cbc198860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400e950 tx=0x7f4ca400ec60 comp rx=0 tx=0).stop 2023-11-27T15:25:03.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1215167226 shutdown_connections 2023-11-27T15:25:03.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4ca80a1260 0x7f4ca80a3710 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4cbc073550 0x7f4cbc198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4cbc108150 0x7f4cbc19f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 --2- 172.21.15.8:0/1215167226 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4cbc074b30 0x7f4cbc198860 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1215167226 >> 172.21.15.8:0/1215167226 conn(0x7f4cbc0fb870 msgr2=0x7f4cbc0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:03.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1215167226 shutdown_connections 2023-11-27T15:25:03.063 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.059+0000 7f4cc09ff700 1 -- 172.21.15.8:0/1215167226 wait complete. 2023-11-27T15:25:03.081 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:25:03.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.290+0000 7fbac43ff700 1 -- 172.21.15.8:0/2282106221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc100290 msgr2=0x7fbabc1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:03.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.290+0000 7fbac43ff700 1 --2- 172.21.15.8:0/2282106221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc100290 0x7fbabc1006a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009990 tx=0x7fbab8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.290+0000 7fbac43ff700 1 -- 172.21.15.8:0/2282106221 shutdown_connections 2023-11-27T15:25:03.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.290+0000 7fbac43ff700 1 --2- 172.21.15.8:0/2282106221 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbabc101e10 0x7fbabc1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.290+0000 7fbac43ff700 1 --2- 172.21.15.8:0/2282106221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc100290 0x7fbabc1006a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.290+0000 7fbac43ff700 1 --2- 172.21.15.8:0/2282106221 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbabc101480 0x7fbabc1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.290+0000 7fbac43ff700 1 -- 172.21.15.8:0/2282106221 >> 172.21.15.8:0/2282106221 conn(0x7fbabc0fb870 msgr2=0x7fbabc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:03.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.290+0000 7fbac43ff700 1 -- 172.21.15.8:0/2282106221 shutdown_connections 2023-11-27T15:25:03.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.290+0000 7fbac43ff700 1 -- 172.21.15.8:0/2282106221 wait complete. 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.291+0000 7fbac43ff700 1 Processor -- start 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.291+0000 7fbac43ff700 1 -- start start 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.291+0000 7fbac43ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbabc100290 0x7fbabc0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.291+0000 7fbac43ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbabc101480 0x7fbabc072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac43ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc101e10 0x7fbabc073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac43ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc073bc0 con 0x7fbabc101e10 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac43ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc073d30 con 0x7fbabc100290 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac43ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc1a0060 con 0x7fbabc101480 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac299c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc101e10 0x7fbabc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac199a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbabc101480 0x7fbabc072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac219b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbabc100290 0x7fbabc0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:03.296 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac299c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc101e10 0x7fbabc073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47044/0 (socket says 172.21.15.8:47044) 2023-11-27T15:25:03.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac299c700 1 -- 172.21.15.8:0/242144788 learned_addr learned my addr 172.21.15.8:0/242144788 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:03.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac199a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbabc101480 0x7fbabc072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:56704/0 (socket says 172.21.15.8:56704) 2023-11-27T15:25:03.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac299c700 1 -- 172.21.15.8:0/242144788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbabc101480 msgr2=0x7fbabc072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:03.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac299c700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbabc101480 0x7fbabc072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac299c700 1 -- 172.21.15.8:0/242144788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbabc100290 msgr2=0x7fbabc0748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:03.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac299c700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbabc100290 0x7fbabc0748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.297 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.292+0000 7fbac299c700 1 -- 172.21.15.8:0/242144788 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab80095f0 con 0x7fbabc101e10 2023-11-27T15:25:03.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.293+0000 7fbac299c700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc101e10 0x7fbabc073420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbab400ef30 tx=0x7fbab400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:03.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.293+0000 7fbab37fe700 1 -- 172.21.15.8:0/242144788 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4018070 con 0x7fbabc101e10 2023-11-27T15:25:03.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.293+0000 7fbab37fe700 1 -- 172.21.15.8:0/242144788 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbab400f040 con 0x7fbabc101e10 2023-11-27T15:25:03.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.293+0000 7fbac43ff700 1 -- 172.21.15.8:0/242144788 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc10bc40 con 0x7fbabc101e10 2023-11-27T15:25:03.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.293+0000 7fbab37fe700 1 -- 172.21.15.8:0/242144788 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab400cd00 con 0x7fbabc101e10 2023-11-27T15:25:03.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.293+0000 7fbac43ff700 1 -- 172.21.15.8:0/242144788 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc10bab0 con 0x7fbabc101e10 2023-11-27T15:25:03.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.294+0000 7fbac43ff700 1 -- 172.21.15.8:0/242144788 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc1a0740 con 0x7fbabc101e10 2023-11-27T15:25:03.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.296+0000 7fbab37fe700 1 -- 172.21.15.8:0/242144788 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fbab4003a00 con 0x7fbabc101e10 2023-11-27T15:25:03.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.297+0000 7fbab37fe700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbaa80a1260 0x7fbaa80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:03.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.297+0000 7fbab37fe700 1 -- 172.21.15.8:0/242144788 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbab40cc8b0 con 0x7fbabc101e10 2023-11-27T15:25:03.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.298+0000 7fbac219b700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbaa80a1260 0x7fbaa80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:03.301 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.298+0000 7fbac219b700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbaa80a1260 0x7fbaa80a3710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbab800b2e0 tx=0x7fbab8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:03.305 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.301+0000 7fbab37fe700 1 -- 172.21.15.8:0/242144788 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbab4095bc0 con 0x7fbabc101e10 2023-11-27T15:25:03.517 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.514+0000 7fbac43ff700 1 -- 172.21.15.8:0/242144788 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbabc107200 con 0x7fbabc101e10 2023-11-27T15:25:03.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.514+0000 7fbab37fe700 1 -- 172.21.15.8:0/242144788 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbab40991e0 con 0x7fbabc101e10 2023-11-27T15:25:03.518 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:25:03.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 -- 172.21.15.8:0/242144788 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbaa80a1260 msgr2=0x7fbaa80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:03.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbaa80a1260 0x7fbaa80a3710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbab800b2e0 tx=0x7fbab8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 -- 172.21.15.8:0/242144788 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc101e10 msgr2=0x7fbabc073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:03.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc101e10 0x7fbabc073420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbab400ef30 tx=0x7fbab400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 -- 172.21.15.8:0/242144788 shutdown_connections 2023-11-27T15:25:03.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc101e10 0x7fbabc073420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fbabc100290 0x7fbabc0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fbaa80a1260 0x7fbaa80a3710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 --2- 172.21.15.8:0/242144788 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fbabc101480 0x7fbabc072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 -- 172.21.15.8:0/242144788 >> 172.21.15.8:0/242144788 conn(0x7fbabc0fb870 msgr2=0x7fbabc0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:03.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 -- 172.21.15.8:0/242144788 shutdown_connections 2023-11-27T15:25:03.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.517+0000 7fbac43ff700 1 -- 172.21.15.8:0/242144788 wait complete. 2023-11-27T15:25:03.540 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:25:03.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.741+0000 7ff931bff700 1 -- 172.21.15.8:0/622898904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff92c1002d0 msgr2=0x7ff92c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:03.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.741+0000 7ff931bff700 1 --2- 172.21.15.8:0/622898904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff92c1002d0 0x7ff92c1006e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009990 tx=0x7ff91c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.741+0000 7ff931bff700 1 -- 172.21.15.8:0/622898904 shutdown_connections 2023-11-27T15:25:03.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.741+0000 7ff931bff700 1 --2- 172.21.15.8:0/622898904 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c101e50 0x7ff92c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.741+0000 7ff931bff700 1 --2- 172.21.15.8:0/622898904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff92c1002d0 0x7ff92c1006e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.741+0000 7ff931bff700 1 --2- 172.21.15.8:0/622898904 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff92c1014c0 0x7ff92c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.741+0000 7ff931bff700 1 -- 172.21.15.8:0/622898904 >> 172.21.15.8:0/622898904 conn(0x7ff92c0fb8b0 msgr2=0x7ff92c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:03.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.741+0000 7ff931bff700 1 -- 172.21.15.8:0/622898904 shutdown_connections 2023-11-27T15:25:03.745 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.741+0000 7ff931bff700 1 -- 172.21.15.8:0/622898904 wait complete. 2023-11-27T15:25:03.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.742+0000 7ff931bff700 1 Processor -- start 2023-11-27T15:25:03.747 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.742+0000 7ff931bff700 1 -- start start 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.742+0000 7ff931bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff92c1002d0 0x7ff92c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.742+0000 7ff931bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff92c1014c0 0x7ff92c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.742+0000 7ff931bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c101e50 0x7ff92c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.742+0000 7ff931bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c073bc0 con 0x7ff92c1002d0 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.742+0000 7ff931bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c073d30 con 0x7ff92c101e50 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.742+0000 7ff931bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c1a0120 con 0x7ff92c1014c0 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff92c1002d0 0x7ff92c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff92c1002d0 0x7ff92c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47056/0 (socket says 172.21.15.8:47056) 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92b7fe700 1 -- 172.21.15.8:0/2735521504 learned_addr learned my addr 172.21.15.8:0/2735521504 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92affd700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff92c1014c0 0x7ff92c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92bfff700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c101e50 0x7ff92c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92b7fe700 1 -- 172.21.15.8:0/2735521504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff92c1014c0 msgr2=0x7ff92c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92b7fe700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff92c1014c0 0x7ff92c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.748 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92b7fe700 1 -- 172.21.15.8:0/2735521504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c101e50 msgr2=0x7ff92c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:03.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92b7fe700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c101e50 0x7ff92c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:03.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92b7fe700 1 -- 172.21.15.8:0/2735521504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c0095f0 con 0x7ff92c1002d0 2023-11-27T15:25:03.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92bfff700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c101e50 0x7ff92c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:03.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.743+0000 7ff92b7fe700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff92c1002d0 0x7ff92c0748a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff91c004730 tx=0x7ff91c004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:03.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.744+0000 7ff928ff9700 1 -- 172.21.15.8:0/2735521504 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c01d070 con 0x7ff92c1002d0 2023-11-27T15:25:03.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.744+0000 7ff928ff9700 1 -- 172.21.15.8:0/2735521504 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff91c021410 con 0x7ff92c1002d0 2023-11-27T15:25:03.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.744+0000 7ff931bff700 1 -- 172.21.15.8:0/2735521504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c10bcc0 con 0x7ff92c1002d0 2023-11-27T15:25:03.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.744+0000 7ff928ff9700 1 -- 172.21.15.8:0/2735521504 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c017650 con 0x7ff92c1002d0 2023-11-27T15:25:03.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.744+0000 7ff931bff700 1 -- 172.21.15.8:0/2735521504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c10bb30 con 0x7ff92c1002d0 2023-11-27T15:25:03.749 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.745+0000 7ff931bff700 1 -- 172.21.15.8:0/2735521504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c107810 con 0x7ff92c1002d0 2023-11-27T15:25:03.750 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.747+0000 7ff928ff9700 1 -- 172.21.15.8:0/2735521504 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff91c021810 con 0x7ff92c1002d0 2023-11-27T15:25:03.751 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.748+0000 7ff928ff9700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9140a11e0 0x7ff9140a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:03.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.748+0000 7ff928ff9700 1 -- 172.21.15.8:0/2735521504 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff91c0ce2b0 con 0x7ff92c1002d0 2023-11-27T15:25:03.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.748+0000 7ff92affd700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9140a11e0 0x7ff9140a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:03.752 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.749+0000 7ff92affd700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9140a11e0 0x7ff9140a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff920009c60 tx=0x7ff920009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:03.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:03.752+0000 7ff928ff9700 1 -- 172.21.15.8:0/2735521504 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff91c0975c0 con 0x7ff92c1002d0 2023-11-27T15:25:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:03 smithi060 ceph-mon[131518]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:25:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:03 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2423297567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:03 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1215167226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:03 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/242144788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:03.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:03 smithi008 ceph-mon[128788]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:25:03.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:03 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2423297567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:03.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:03 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1215167226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:03.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:03 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/242144788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:03.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:03 smithi029 ceph-mon[131166]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:25:03.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:03 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2423297567' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:03.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:03 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1215167226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:03.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:03 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/242144788' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:04.008 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.005+0000 7ff931bff700 1 -- 172.21.15.8:0/2735521504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff92c067af0 con 0x7ff92c1002d0 2023-11-27T15:25:04.009 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.005+0000 7ff928ff9700 1 -- 172.21.15.8:0/2735521504 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10768 (secure 0 0 0) 0x7ff91c09abe0 con 0x7ff92c1002d0 2023-11-27T15:25:04.009 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:04.009 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:23:27.167939+0000","modified":"2023-11-27T15:23:59.241472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24595,"mds_1":24593,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24593":{"gid":24593,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24595":{"gid":24595,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:25:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 -- 172.21.15.8:0/2735521504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9140a11e0 msgr2=0x7ff9140a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9140a11e0 0x7ff9140a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff920009c60 tx=0x7ff920009260 comp rx=0 tx=0).stop 2023-11-27T15:25:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 -- 172.21.15.8:0/2735521504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff92c1002d0 msgr2=0x7ff92c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff92c1002d0 0x7ff92c0748a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff91c004730 tx=0x7ff91c004760 comp rx=0 tx=0).stop 2023-11-27T15:25:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 -- 172.21.15.8:0/2735521504 shutdown_connections 2023-11-27T15:25:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff92c101e50 0x7ff92c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff92c1002d0 0x7ff92c0748a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff9140a11e0 0x7ff9140a3690 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 --2- 172.21.15.8:0/2735521504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff92c1014c0 0x7ff92c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 -- 172.21.15.8:0/2735521504 >> 172.21.15.8:0/2735521504 conn(0x7ff92c0fb8b0 msgr2=0x7ff92c104950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:04.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 -- 172.21.15.8:0/2735521504 shutdown_connections 2023-11-27T15:25:04.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.008+0000 7ff931bff700 1 -- 172.21.15.8:0/2735521504 wait complete. 2023-11-27T15:25:04.013 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2023-11-27T15:25:04.031 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:25:04.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.230+0000 7f26e2bff700 1 -- 172.21.15.8:0/1650326933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26dc101e50 msgr2=0x7f26dc108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.230+0000 7f26e2bff700 1 --2- 172.21.15.8:0/1650326933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26dc101e50 0x7f26dc108710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f26d400b0a0 tx=0x7f26d400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.230+0000 7f26e2bff700 1 -- 172.21.15.8:0/1650326933 shutdown_connections 2023-11-27T15:25:04.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.230+0000 7f26e2bff700 1 --2- 172.21.15.8:0/1650326933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26dc101e50 0x7f26dc108710 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.230+0000 7f26e2bff700 1 --2- 172.21.15.8:0/1650326933 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26dc1002d0 0x7f26dc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.230+0000 7f26e2bff700 1 --2- 172.21.15.8:0/1650326933 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc1014c0 0x7f26dc101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.230+0000 7f26e2bff700 1 -- 172.21.15.8:0/1650326933 >> 172.21.15.8:0/1650326933 conn(0x7f26dc0fb8b0 msgr2=0x7f26dc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:04.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.230+0000 7f26e2bff700 1 -- 172.21.15.8:0/1650326933 shutdown_connections 2023-11-27T15:25:04.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.230+0000 7f26e2bff700 1 -- 172.21.15.8:0/1650326933 wait complete. 2023-11-27T15:25:04.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.232+0000 7f26e2bff700 1 Processor -- start 2023-11-27T15:25:04.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.232+0000 7f26e2bff700 1 -- start start 2023-11-27T15:25:04.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26e2bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26dc1002d0 0x7f26dc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:04.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26e2bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26dc1014c0 0x7f26dc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:04.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26e2bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc101e50 0x7f26dc19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:04.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26e2bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc19fef0 con 0x7f26dc1002d0 2023-11-27T15:25:04.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26e2bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc1a0030 con 0x7f26dc1014c0 2023-11-27T15:25:04.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26e2bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc1a01c0 con 0x7f26dc101e50 2023-11-27T15:25:04.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26e099b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26dc1002d0 0x7f26dc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:04.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26dbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26dc1014c0 0x7f26dc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:04.238 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26dbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26dc1014c0 0x7f26dc1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45102/0 (socket says 172.21.15.8:45102) 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26dbfff700 1 -- 172.21.15.8:0/3703775065 learned_addr learned my addr 172.21.15.8:0/3703775065 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26e119c700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc101e50 0x7f26dc19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26dbfff700 1 -- 172.21.15.8:0/3703775065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc101e50 msgr2=0x7f26dc19f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26dbfff700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc101e50 0x7f26dc19f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26dbfff700 1 -- 172.21.15.8:0/3703775065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26dc1002d0 msgr2=0x7f26dc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26dbfff700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26dc1002d0 0x7f26dc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26dbfff700 1 -- 172.21.15.8:0/3703775065 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d4009d00 con 0x7f26dc1014c0 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.233+0000 7f26e099b700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26dc1002d0 0x7f26dc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.234+0000 7f26dbfff700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26dc1014c0 0x7f26dc1988b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f26c800e9a0 tx=0x7f26c800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.234+0000 7f26d9ffb700 1 -- 172.21.15.8:0/3703775065 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8009790 con 0x7f26dc1014c0 2023-11-27T15:25:04.239 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.234+0000 7f26e2bff700 1 -- 172.21.15.8:0/3703775065 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc10bca0 con 0x7f26dc1014c0 2023-11-27T15:25:04.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.234+0000 7f26d9ffb700 1 -- 172.21.15.8:0/3703775065 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26c8010430 con 0x7f26dc1014c0 2023-11-27T15:25:04.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.234+0000 7f26d9ffb700 1 -- 172.21.15.8:0/3703775065 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8018680 con 0x7f26dc1014c0 2023-11-27T15:25:04.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.234+0000 7f26e2bff700 1 -- 172.21.15.8:0/3703775065 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc10bb10 con 0x7f26dc1014c0 2023-11-27T15:25:04.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.235+0000 7f26e2bff700 1 -- 172.21.15.8:0/3703775065 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26dc1a0900 con 0x7f26dc1014c0 2023-11-27T15:25:04.240 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.237+0000 7f26d9ffb700 1 -- 172.21.15.8:0/3703775065 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f26c80105a0 con 0x7f26dc1014c0 2023-11-27T15:25:04.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.238+0000 7f26d9ffb700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26cc0a11e0 0x7f26cc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:04.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.239+0000 7f26d9ffb700 1 -- 172.21.15.8:0/3703775065 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f26c8014070 con 0x7f26dc1014c0 2023-11-27T15:25:04.242 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.239+0000 7f26e099b700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26cc0a11e0 0x7f26cc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:04.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.239+0000 7f26e099b700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26cc0a11e0 0x7f26cc0a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f26dc101320 tx=0x7f26d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:04.244 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.241+0000 7f26d9ffb700 1 -- 172.21.15.8:0/3703775065 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f26c8096220 con 0x7f26dc1014c0 2023-11-27T15:25:04.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.456+0000 7f26e2bff700 1 -- 172.21.15.8:0/3703775065 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26dc1a0eb0 con 0x7f26dc1014c0 2023-11-27T15:25:04.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.459+0000 7f26d9ffb700 1 -- 172.21.15.8:0/3703775065 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24344 (secure 0 0 0) 0x7f26c8099840 con 0x7f26dc1014c0 2023-11-27T15:25:04.463 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:04.463 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":90,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:24:40.249579+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:25:04.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 -- 172.21.15.8:0/3703775065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26cc0a11e0 msgr2=0x7f26cc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26cc0a11e0 0x7f26cc0a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f26dc101320 tx=0x7f26d0009260 comp rx=0 tx=0).stop 2023-11-27T15:25:04.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 -- 172.21.15.8:0/3703775065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26dc1014c0 msgr2=0x7f26dc1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26dc1014c0 0x7f26dc1988b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f26c800e9a0 tx=0x7f26c800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 -- 172.21.15.8:0/3703775065 shutdown_connections 2023-11-27T15:25:04.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f26cc0a11e0 0x7f26cc0a3690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f26dc101e50 0x7f26dc19f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26dc1002d0 0x7f26dc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 --2- 172.21.15.8:0/3703775065 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f26dc1014c0 0x7f26dc1988b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 -- 172.21.15.8:0/3703775065 >> 172.21.15.8:0/3703775065 conn(0x7f26dc0fb8b0 msgr2=0x7f26dc106d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:04.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 -- 172.21.15.8:0/3703775065 shutdown_connections 2023-11-27T15:25:04.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.461+0000 7f26e2bff700 1 -- 172.21.15.8:0/3703775065 wait complete. 2023-11-27T15:25:04.484 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:25:04.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.687+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1936553722 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3941014c0 msgr2=0x7fe394101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.687+0000 7fe39a9ff700 1 --2- 172.21.15.8:0/1936553722 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3941014c0 0x7fe394101910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009990 tx=0x7fe37c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.687+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1936553722 shutdown_connections 2023-11-27T15:25:04.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.687+0000 7fe39a9ff700 1 --2- 172.21.15.8:0/1936553722 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3941002d0 0x7fe3941006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.687+0000 7fe39a9ff700 1 --2- 172.21.15.8:0/1936553722 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe394101e50 0x7fe394108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.687+0000 7fe39a9ff700 1 --2- 172.21.15.8:0/1936553722 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe3941014c0 0x7fe394101910 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.687+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1936553722 >> 172.21.15.8:0/1936553722 conn(0x7fe3940fb8b0 msgr2=0x7fe3940fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:04.691 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.687+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1936553722 shutdown_connections 2023-11-27T15:25:04.692 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.687+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1936553722 wait complete. 2023-11-27T15:25:04.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.688+0000 7fe39a9ff700 1 Processor -- start 2023-11-27T15:25:04.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.688+0000 7fe39a9ff700 1 -- start start 2023-11-27T15:25:04.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.688+0000 7fe39a9ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3941002d0 0x7fe394196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:04.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.688+0000 7fe39a9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3941014c0 0x7fe394196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:04.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.688+0000 7fe39a9ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe394101e50 0x7fe39419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:04.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.688+0000 7fe39a9ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39419dbb0 con 0x7fe394101e50 2023-11-27T15:25:04.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.688+0000 7fe39a9ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39419dcf0 con 0x7fe3941002d0 2023-11-27T15:25:04.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.688+0000 7fe39a9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39419de80 con 0x7fe3941014c0 2023-11-27T15:25:04.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe398f9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe394101e50 0x7fe39419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:04.694 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe398f9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe394101e50 0x7fe39419d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47076/0 (socket says 172.21.15.8:47076) 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe398f9c700 1 -- 172.21.15.8:0/1606623284 learned_addr learned my addr 172.21.15.8:0/1606623284 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe393fff700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3941002d0 0x7fe394196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe38bfff700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3941014c0 0x7fe394196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe398f9c700 1 -- 172.21.15.8:0/1606623284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3941014c0 msgr2=0x7fe394196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe398f9c700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3941014c0 0x7fe394196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe398f9c700 1 -- 172.21.15.8:0/1606623284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3941002d0 msgr2=0x7fe394196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe398f9c700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3941002d0 0x7fe394196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe398f9c700 1 -- 172.21.15.8:0/1606623284 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0095f0 con 0x7fe394101e50 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.689+0000 7fe398f9c700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe394101e50 0x7fe39419d5e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00ef30 tx=0x7fe38c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.690+0000 7fe391ffb700 1 -- 172.21.15.8:0/1606623284 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c018070 con 0x7fe394101e50 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.690+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1606623284 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39410bc80 con 0x7fe394101e50 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.690+0000 7fe391ffb700 1 -- 172.21.15.8:0/1606623284 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe38c00f040 con 0x7fe394101e50 2023-11-27T15:25:04.695 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.690+0000 7fe391ffb700 1 -- 172.21.15.8:0/1606623284 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38c00cd00 con 0x7fe394101e50 2023-11-27T15:25:04.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.690+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1606623284 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39410baf0 con 0x7fe394101e50 2023-11-27T15:25:04.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.691+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1606623284 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3941071b0 con 0x7fe394101e50 2023-11-27T15:25:04.696 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.693+0000 7fe391ffb700 1 -- 172.21.15.8:0/1606623284 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe38c003a00 con 0x7fe394101e50 2023-11-27T15:25:04.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.694+0000 7fe391ffb700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3800a11e0 0x7fe3800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:04.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.694+0000 7fe391ffb700 1 -- 172.21.15.8:0/1606623284 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe38c0cc8d0 con 0x7fe394101e50 2023-11-27T15:25:04.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.694+0000 7fe393fff700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3800a11e0 0x7fe3800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:04.698 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.695+0000 7fe393fff700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3800a11e0 0x7fe3800a3690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe394101320 tx=0x7fe384009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:04.701 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.698+0000 7fe391ffb700 1 -- 172.21.15.8:0/1606623284 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe38c095c40 con 0x7fe394101e50 2023-11-27T15:25:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:04 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2735521504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:04 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3703775065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:04.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:04 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2735521504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:04.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:04 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3703775065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:04 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2735521504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:04 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3703775065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:04.959 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.955+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1606623284 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe39402cfe0 con 0x7fe394101e50 2023-11-27T15:25:04.960 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.956+0000 7fe391ffb700 1 -- 172.21.15.8:0/1606623284 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10768 (secure 0 0 0) 0x7fe38c099260 con 0x7fe394101e50 2023-11-27T15:25:04.960 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:04.960 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:23:27.167939+0000","modified":"2023-11-27T15:23:59.241472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24595,"mds_1":24593,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24593":{"gid":24593,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24595":{"gid":24595,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:25:04.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1606623284 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3800a11e0 msgr2=0x7fe3800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3800a11e0 0x7fe3800a3690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe394101320 tx=0x7fe384009260 comp rx=0 tx=0).stop 2023-11-27T15:25:04.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1606623284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe394101e50 msgr2=0x7fe39419d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:04.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe394101e50 0x7fe39419d5e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00ef30 tx=0x7fe38c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1606623284 shutdown_connections 2023-11-27T15:25:04.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe394101e50 0x7fe39419d5e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe3941002d0 0x7fe394196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe3800a11e0 0x7fe3800a3690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 --2- 172.21.15.8:0/1606623284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe3941014c0 0x7fe394196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:04.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1606623284 >> 172.21.15.8:0/1606623284 conn(0x7fe3940fb8b0 msgr2=0x7fe3940fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:04.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1606623284 shutdown_connections 2023-11-27T15:25:04.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:04.959+0000 7fe39a9ff700 1 -- 172.21.15.8:0/1606623284 wait complete. 2023-11-27T15:25:04.964 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2023-11-27T15:25:04.982 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-27T15:23:27.167939+0000', 'modified': '2023-11-27T15:23:59.241472+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24595, 'mds_1': 24593, 'mds_2': 24539}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24535': {'gid': 24535, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/612130958', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 612130958}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 612130958}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.29:6837/1651468308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6836', 'nonce': 1651468308}, {'type': 'v1', 'addr': '172.21.15.29:6837', 'nonce': 1651468308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24547': {'gid': 24547, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6837/287097286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6836', 'nonce': 287097286}, {'type': 'v1', 'addr': '172.21.15.60:6837', 'nonce': 287097286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6839/3185908756', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6838', 'nonce': 3185908756}, {'type': 'v1', 'addr': '172.21.15.60:6839', 'nonce': 3185908756}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24593': {'gid': 24593, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.29:6839/2619830143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6838', 'nonce': 2619830143}, {'type': 'v1', 'addr': '172.21.15.29:6839', 'nonce': 2619830143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24595': {'gid': 24595, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.8:6841/110161953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.8:6840', 'nonce': 110161953}, {'type': 'v1', 'addr': '172.21.15.8:6841', 'nonce': 110161953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-27T15:25:04.983 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-27T15:25:04.983 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-11-27T15:25:05.187 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.182+0000 7f6b055ff700 1 -- 172.21.15.8:0/1327204012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b00100230 msgr2=0x7f6b00100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.182+0000 7f6b055ff700 1 --2- 172.21.15.8:0/1327204012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b00100230 0x7f6b00100640 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009990 tx=0x7f6af0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.182+0000 7f6b055ff700 1 -- 172.21.15.8:0/1327204012 shutdown_connections 2023-11-27T15:25:05.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.182+0000 7f6b055ff700 1 --2- 172.21.15.8:0/1327204012 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b00101db0 0x7f6b00108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.182+0000 7f6b055ff700 1 --2- 172.21.15.8:0/1327204012 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b00100230 0x7f6b00100640 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.182+0000 7f6b055ff700 1 --2- 172.21.15.8:0/1327204012 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b00101420 0x7f6b00101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.182+0000 7f6b055ff700 1 -- 172.21.15.8:0/1327204012 >> 172.21.15.8:0/1327204012 conn(0x7f6b000fb810 msgr2=0x7f6b000fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.182+0000 7f6b055ff700 1 -- 172.21.15.8:0/1327204012 shutdown_connections 2023-11-27T15:25:05.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.183+0000 7f6b055ff700 1 -- 172.21.15.8:0/1327204012 wait complete. 2023-11-27T15:25:05.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.183+0000 7f6b055ff700 1 Processor -- start 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6b055ff700 1 -- start start 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6b055ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b00100230 0x7f6b00198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6b055ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b00101420 0x7f6b00198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6b055ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b00101db0 0x7f6b0019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6b055ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0019fdd0 con 0x7f6b00101db0 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6b055ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0019ff10 con 0x7f6b00100230 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6b055ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b001a00a0 con 0x7f6b00101420 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6aff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b00101db0 0x7f6b0019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6aff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b00101db0 0x7f6b0019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47096/0 (socket says 172.21.15.8:47096) 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6afeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b00100230 0x7f6b00198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6aff7fe700 1 -- 172.21.15.8:0/2633280582 learned_addr learned my addr 172.21.15.8:0/2633280582 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.184+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b00101420 0x7f6b00198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6aff7fe700 1 -- 172.21.15.8:0/2633280582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b00101420 msgr2=0x7f6b00198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6aff7fe700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b00101420 0x7f6b00198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6aff7fe700 1 -- 172.21.15.8:0/2633280582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b00100230 msgr2=0x7f6b00198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6aff7fe700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b00100230 0x7f6b00198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6aff7fe700 1 -- 172.21.15.8:0/2633280582 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00095f0 con 0x7f6b00101db0 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6afeffd700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b00100230 0x7f6b00198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6aff7fe700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b00101db0 0x7f6b0019f800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6af800ef30 tx=0x7f6af800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6ae7fff700 1 -- 172.21.15.8:0/2633280582 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8018070 con 0x7f6b00101db0 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6b055ff700 1 -- 172.21.15.8:0/2633280582 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0010bbe0 con 0x7f6b00101db0 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6ae7fff700 1 -- 172.21.15.8:0/2633280582 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af800f040 con 0x7f6b00101db0 2023-11-27T15:25:05.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6ae7fff700 1 -- 172.21.15.8:0/2633280582 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af800cdb0 con 0x7f6b00101db0 2023-11-27T15:25:05.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.185+0000 7f6b055ff700 1 -- 172.21.15.8:0/2633280582 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0010ba50 con 0x7f6b00101db0 2023-11-27T15:25:05.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.189+0000 7f6ae7fff700 1 -- 172.21.15.8:0/2633280582 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f6af8003a00 con 0x7f6b00101db0 2023-11-27T15:25:05.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.190+0000 7f6ae7fff700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ae80a1260 0x7f6ae80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.190+0000 7f6b18e78b80 1 Processor -- start 2023-11-27T15:25:05.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.190+0000 7f6ae7fff700 1 -- 172.21.15.8:0/2633280582 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6af80cc8b0 con 0x7f6b00101db0 2023-11-27T15:25:05.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.190+0000 7f6afeffd700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ae80a1260 0x7f6ae80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.195 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.190+0000 7f6b18e78b80 1 -- start start 2023-11-27T15:25:05.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fc99fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fc9aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fc9b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fb9f010 con 0x55996fc9b0c0 2023-11-27T15:25:05.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fbf1850 con 0x55996fc99bb0 2023-11-27T15:25:05.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fbeace0 con 0x55996fc9a500 2023-11-27T15:25:05.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6aff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fc9b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6afe7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fc9aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6afeffd700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ae80a1260 0x7f6ae80a3710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6af000b2e0 tx=0x7f6af0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6afeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fc99fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6afe7fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fc9aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:56776/0 (socket says 172.21.15.8:56776) 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6aff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fc9b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47112/0 (socket says 172.21.15.8:47112) 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.191+0000 7f6afe7fc700 1 -- 172.21.15.8:0/1238550066 learned_addr learned my addr 172.21.15.8:0/1238550066 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.192+0000 7f6afe7fc700 1 -- 172.21.15.8:0/1238550066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 msgr2=0x55996fc99fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.192+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/1238550066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fc99fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.192+0000 7f6afe7fc700 1 -- 172.21.15.8:0/1238550066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 msgr2=0x55996fc9b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.192+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/1238550066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fc9b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.192+0000 7f6afe7fc700 1 -- 172.21.15.8:0/1238550066 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996fa6c5e0 con 0x55996fc9a500 2023-11-27T15:25:05.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.192+0000 7f6aff7fe700 1 --2- 172.21.15.8:0/1238550066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fc9b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:05.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.192+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/1238550066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fc9aae0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6af400bd70 tx=0x7f6af400f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c9114911f8117e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.192+0000 7f6ae57fa700 1 -- 172.21.15.8:0/1238550066 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af400fda0 con 0x55996fc9a500 2023-11-27T15:25:05.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.192+0000 7f6ae57fa700 1 -- 172.21.15.8:0/1238550066 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af401a920 con 0x55996fc9a500 2023-11-27T15:25:05.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.192+0000 7f6ae57fa700 1 -- 172.21.15.8:0/1238550066 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af40117c0 con 0x55996fc9a500 2023-11-27T15:25:05.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.193+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1238550066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 msgr2=0x55996fc9aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.193+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/1238550066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fc9aae0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6af400bd70 tx=0x7f6af400f640 comp rx=0 tx=0).stop 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.193+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1238550066 shutdown_connections 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.193+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/1238550066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fc9b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.193+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/1238550066 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fc99fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.193+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/1238550066 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fc9aae0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.193+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1238550066 >> 172.21.15.8:0/1238550066 conn(0x55996fc8f310 msgr2=0x55996fc94620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.193+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1238550066 shutdown_connections 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.193+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1238550066 wait complete. 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.194+0000 7f6b18e78b80 1 -- start start 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.194+0000 7f6b18e78b80 1 Processor -- start 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.194+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc99bb0 0x55996fca35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.194+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fca3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.194+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc9b0c0 0x55996fcaab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.194+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fb8c890 con 0x55996fc99bb0 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.194+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fb8c530 con 0x55996fc9b0c0 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.194+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fa6e8d0 con 0x55996fc9a500 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.194+0000 7f6b18e78b80 10 client.0 _ll_register_callbacks cb 0x55996fbaa730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc99bb0 0x55996fca35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6aff7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc9b0c0 0x55996fcaab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc99bb0 0x55996fca35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47124/0 (socket says 172.21.15.8:47124) 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6aff7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc9b0c0 0x55996fcaab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45148/0 (socket says 172.21.15.8:45148) 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afeffd700 1 -- 172.21.15.8:0/827580970 learned_addr learned my addr 172.21.15.8:0/827580970 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fca3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afeffd700 1 -- 172.21.15.8:0/827580970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 msgr2=0x55996fca3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afeffd700 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fca3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afeffd700 1 -- 172.21.15.8:0/827580970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc9b0c0 msgr2=0x55996fcaab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afeffd700 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc9b0c0 0x55996fcaab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afeffd700 1 -- 172.21.15.8:0/827580970 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af400b9a0 con 0x55996fc99bb0 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fca3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6afeffd700 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc99bb0 0x55996fca35c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6b00101280 tx=0x7f6af001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f03e32ad46109ac server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.195+0000 7f6ae2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.196+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/827580970 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af000f930 con 0x55996fc99bb0 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.196+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/827580970 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6af000fa90 con 0x55996fc99bb0 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.196+0000 7f6b18e78b80 1 -- 172.21.15.8:0/827580970 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55996fc8cad0 con 0x55996fc99bb0 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.196+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/827580970 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af000fc90 con 0x55996fc99bb0 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.196+0000 7f6b18e78b80 10 client.15126 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.196+0000 7f6b18e78b80 1 -- 172.21.15.8:0/827580970 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55996fbb1140 con 0x55996fc99bb0 2023-11-27T15:25:05.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.196+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/827580970 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55996fc8cad0 con 0x55996fc99bb0 2023-11-27T15:25:05.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.196+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/827580970 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6af0024410 con 0x55996fc99bb0 2023-11-27T15:25:05.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.196+0000 7f6b18e78b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:05.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.197+0000 7f6b18e78b80 10 client.15126 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:25:05.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.197+0000 7f6b18e78b80 20 client.15126 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:05.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.197+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55996fcae7c0 0x55996fcadbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.197+0000 7f6b18e78b80 4 client.15126 mds_command: new command op to 24539 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:05.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.197+0000 7f6b18e78b80 1 -- 172.21.15.8:0/827580970 --> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55996fbad050 con 0x55996fcae7c0 2023-11-27T15:25:05.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.197+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55996fcae7c0 0x55996fcadbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.198+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55996fcae7c0 0x55996fcadbf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57af38888295917f server_cookie=4e7d6aefddfb107c in_seq=0 out_seq=0 2023-11-27T15:25:05.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.198+0000 7f6ae2ff5700 10 client.15126 ms_handle_connect on v2:172.21.15.29:6836/1651468308 2023-11-27T15:25:05.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.198+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/827580970 <== mds.2 v2:172.21.15.29:6836/1651468308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55996fbad050 con 0x55996fcae7c0 2023-11-27T15:25:05.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.198+0000 7f6ae2ff5700 10 client.15126 handle_command_reply: tid=0 2023-11-27T15:25:05.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 client.15126 shutdown 2023-11-27T15:25:05.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 -- 172.21.15.8:0/827580970 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc99bb0 msgr2=0x55996fca35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc99bb0 0x55996fca35c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6b00101280 tx=0x7f6af001a050 comp rx=0 tx=0).stop 2023-11-27T15:25:05.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 -- 172.21.15.8:0/827580970 shutdown_connections 2023-11-27T15:25:05.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55996fcae7c0 0x55996fcadbf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc9b0c0 0x55996fcaab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc99bb0 0x55996fca35c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6ae2ff5700 0 client.15126 ms_handle_reset on v2:172.21.15.29:6836/1651468308 2023-11-27T15:25:05.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/827580970 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fca3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 -- 172.21.15.8:0/827580970 >> 172.21.15.8:0/827580970 conn(0x55996fc8f310 msgr2=0x55996fc93d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 -- 172.21.15.8:0/827580970 shutdown_connections 2023-11-27T15:25:05.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 1 -- 172.21.15.8:0/827580970 wait complete. 2023-11-27T15:25:05.209 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.199+0000 7f6b18e78b80 20 client.15126 trim_cache size 0 max 16384 2023-11-27T15:25:05.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.218+0000 7f6b18e78b80 1 Processor -- start 2023-11-27T15:25:05.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.218+0000 7f6b18e78b80 1 -- start start 2023-11-27T15:25:05.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.218+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fca22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.218+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fcaab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.218+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.218+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fba8fa0 con 0x55996fc9b0c0 2023-11-27T15:25:05.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.218+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fa70a30 con 0x55996fc99bb0 2023-11-27T15:25:05.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.218+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fc9af40 con 0x55996fc9a500 2023-11-27T15:25:05.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6aff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6aff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47140/0 (socket says 172.21.15.8:47140) 2023-11-27T15:25:05.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6aff7fe700 1 -- 172.21.15.8:0/2459401987 learned_addr learned my addr 172.21.15.8:0/2459401987 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/2459401987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fcaab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6afeffd700 1 --2- 172.21.15.8:0/2459401987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fca22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6aff7fe700 1 -- 172.21.15.8:0/2459401987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 msgr2=0x55996fcaab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6aff7fe700 1 --2- 172.21.15.8:0/2459401987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fcaab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6aff7fe700 1 -- 172.21.15.8:0/2459401987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 msgr2=0x55996fca22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6aff7fe700 1 --2- 172.21.15.8:0/2459401987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fca22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6aff7fe700 1 -- 172.21.15.8:0/2459401987 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996fbb0e90 con 0x55996fc9b0c0 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6aff7fe700 1 --2- 172.21.15.8:0/2459401987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca1460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6af8004d70 tx=0x7f6af8083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8c4157e3fd5aa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6ae5ffb700 1 -- 172.21.15.8:0/2459401987 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8084040 con 0x55996fc9b0c0 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.219+0000 7f6ae5ffb700 1 -- 172.21.15.8:0/2459401987 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af8004ad0 con 0x55996fc9b0c0 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.220+0000 7f6ae5ffb700 1 -- 172.21.15.8:0/2459401987 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8099b40 con 0x55996fc9b0c0 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.220+0000 7f6b18e78b80 1 -- 172.21.15.8:0/2459401987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 msgr2=0x55996fca1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.220+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/2459401987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca1460 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6af8004d70 tx=0x7f6af8083040 comp rx=0 tx=0).stop 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.220+0000 7f6b18e78b80 1 -- 172.21.15.8:0/2459401987 shutdown_connections 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.220+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/2459401987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca1460 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.220+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/2459401987 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fca22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.220+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/2459401987 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fcaab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.220+0000 7f6b18e78b80 1 -- 172.21.15.8:0/2459401987 >> 172.21.15.8:0/2459401987 conn(0x55996fc8f310 msgr2=0x55996fcb7550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.220+0000 7f6b18e78b80 1 -- 172.21.15.8:0/2459401987 shutdown_connections 2023-11-27T15:25:05.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.220+0000 7f6b18e78b80 1 -- 172.21.15.8:0/2459401987 wait complete. 2023-11-27T15:25:05.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.221+0000 7f6b18e78b80 1 -- start start 2023-11-27T15:25:05.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.221+0000 7f6b18e78b80 1 Processor -- start 2023-11-27T15:25:05.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.221+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fca0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.221+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fca0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.221+0000 7f6b18e78b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.221+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fca9630 con 0x55996fc9b0c0 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.221+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fca97a0 con 0x55996fc99bb0 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.221+0000 7f6b18e78b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996fca9960 con 0x55996fc9a500 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.221+0000 7f6b18e78b80 10 client.0 _ll_register_callbacks cb 0x55996fbaa730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6aff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6afeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fca0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6afeffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fca0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45172/0 (socket says 172.21.15.8:45172) 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6afeffd700 1 -- 172.21.15.8:0/1506863535 learned_addr learned my addr 172.21.15.8:0/1506863535 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fca0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6aff7fe700 1 -- 172.21.15.8:0/1506863535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 msgr2=0x55996fca0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6aff7fe700 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fca0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6aff7fe700 1 -- 172.21.15.8:0/1506863535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 msgr2=0x55996fca0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6aff7fe700 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fca0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6aff7fe700 1 -- 172.21.15.8:0/1506863535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0010bbe0 con 0x55996fc9b0c0 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6afe7fc700 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fca0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6afeffd700 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fca0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6aff7fe700 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca90f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009c10 tx=0x7f6af80997e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24f34093288fd67 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6ae2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/1506863535 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af80b92b0 con 0x55996fc9b0c0 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/1506863535 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6af809d910 con 0x55996fc9b0c0 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.222+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1506863535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6af401a920 con 0x55996fc9b0c0 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/1506863535 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6af8099b40 con 0x55996fc9b0c0 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6b18e78b80 10 client.15132 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:05.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1506863535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55996fa01c10 con 0x55996fc9b0c0 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/1506863535 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6af401a920 con 0x55996fc9b0c0 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/1506863535 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6af801b9d0 con 0x55996fc9b0c0 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6b18e78b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6b18e78b80 10 client.15132 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6b18e78b80 20 client.15132 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55996fd07740 0x55996fd04690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6b18e78b80 4 client.15132 mds_command: new command op to 24539 tid=0[{"prefix": "status"}] 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.223+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1506863535 --> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55996fbb9610 con 0x55996fd07740 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.224+0000 7f6afeffd700 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55996fd07740 0x55996fd04690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.224+0000 7f6afeffd700 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55996fd07740 0x55996fd04690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12e697a676425eab server_cookie=ba76f09ec385b76b in_seq=0 out_seq=0 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.224+0000 7f6ae2ff5700 10 client.15132 ms_handle_connect on v2:172.21.15.29:6836/1651468308 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.224+0000 7f6ae2ff5700 1 -- 172.21.15.8:0/1506863535 <== mds.2 v2:172.21.15.29:6836/1651468308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55996fbb9610 con 0x55996fd07740 2023-11-27T15:25:05.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.224+0000 7f6ae2ff5700 10 client.15132 handle_command_reply: tid=0 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.224+0000 7f6b18e78b80 1 client.15132 shutdown 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.224+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1506863535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 msgr2=0x55996fca90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.224+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca90f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009c10 tx=0x7f6af80997e0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.224+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1506863535 shutdown_connections 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55996fd07740 0x55996fd04690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55996fc9b0c0 0x55996fca90f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55996fc99bb0 0x55996fca0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b18e78b80 1 --2- 172.21.15.8:0/1506863535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55996fc9a500 0x55996fca0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1506863535 >> 172.21.15.8:0/1506863535 conn(0x55996fc8f310 msgr2=0x55996fca1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6ae2ff5700 0 client.15132 ms_handle_reset on v2:172.21.15.29:6836/1651468308 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1506863535 shutdown_connections 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b18e78b80 1 -- 172.21.15.8:0/1506863535 wait complete. 2023-11-27T15:25:05.234 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b18e78b80 20 client.15132 trim_cache size 0 max 16384 2023-11-27T15:25:05.235 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:25:05.235 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:25:05.235 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 2, 2023-11-27T15:25:05.235 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24539, 2023-11-27T15:25:05.235 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2023-11-27T15:25:05.235 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2023-11-27T15:25:05.235 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2023-11-27T15:25:05.235 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 66.988436492000005, 2023-11-27T15:25:05.235 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 24, 2023-11-27T15:25:05.236 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 90, 2023-11-27T15:25:05.236 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 88, 2023-11-27T15:25:05.236 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 116.415756827 2023-11-27T15:25:05.236 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:25:05.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b055ff700 1 -- 172.21.15.8:0/2633280582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ae80a1260 msgr2=0x7f6ae80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b055ff700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ae80a1260 0x7f6ae80a3710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6af000b2e0 tx=0x7f6af0005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b055ff700 1 -- 172.21.15.8:0/2633280582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b00101db0 msgr2=0x7f6b0019f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b055ff700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b00101db0 0x7f6b0019f800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6af800ef30 tx=0x7f6af800c3a0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.225+0000 7f6b055ff700 1 -- 172.21.15.8:0/2633280582 shutdown_connections 2023-11-27T15:25:05.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.226+0000 7f6b055ff700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b00101db0 0x7f6b0019f800 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.226+0000 7f6b055ff700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6b00100230 0x7f6b00198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.226+0000 7f6b055ff700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6ae80a1260 0x7f6ae80a3710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.226+0000 7f6b055ff700 1 --2- 172.21.15.8:0/2633280582 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6b00101420 0x7f6b00198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.226+0000 7f6b055ff700 1 -- 172.21.15.8:0/2633280582 >> 172.21.15.8:0/2633280582 conn(0x7f6b000fb810 msgr2=0x7f6b00106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.226+0000 7f6b055ff700 1 -- 172.21.15.8:0/2633280582 shutdown_connections 2023-11-27T15:25:05.237 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.226+0000 7f6b055ff700 1 -- 172.21.15.8:0/2633280582 wait complete. 2023-11-27T15:25:05.249 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-27T15:25:05.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.449+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2684308136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa528101420 msgr2=0x7fa528101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.449+0000 7fa52e7ff700 1 --2- 172.21.15.8:0/2684308136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa528101420 0x7fa528101870 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa510009990 tx=0x7fa510009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.449+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2684308136 shutdown_connections 2023-11-27T15:25:05.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.449+0000 7fa52e7ff700 1 --2- 172.21.15.8:0/2684308136 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa528101db0 0x7fa528108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.449+0000 7fa52e7ff700 1 --2- 172.21.15.8:0/2684308136 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa528100230 0x7fa528100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.449+0000 7fa52e7ff700 1 --2- 172.21.15.8:0/2684308136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa528101420 0x7fa528101870 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.449+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2684308136 >> 172.21.15.8:0/2684308136 conn(0x7fa5280fb810 msgr2=0x7fa5280fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.449+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2684308136 shutdown_connections 2023-11-27T15:25:05.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.449+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2684308136 wait complete. 2023-11-27T15:25:05.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.450+0000 7fa52e7ff700 1 Processor -- start 2023-11-27T15:25:05.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.450+0000 7fa52e7ff700 1 -- start start 2023-11-27T15:25:05.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa52e7ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa528100230 0x7fa528196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa52e7ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa528101420 0x7fa528196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa52e7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa528101db0 0x7fa52819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa52e7ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52819dba0 con 0x7fa528101420 2023-11-27T15:25:05.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa52e7ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52819dce0 con 0x7fa528100230 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa52e7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52819de70 con 0x7fa528101db0 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa528101420 0x7fa528196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa528101420 0x7fa528196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47176/0 (socket says 172.21.15.8:47176) 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa51ffff700 1 -- 172.21.15.8:0/2994783122 learned_addr learned my addr 172.21.15.8:0/2994783122 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa527fff700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa528100230 0x7fa528196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.451+0000 7fa52cd9c700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa528101db0 0x7fa52819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa51ffff700 1 -- 172.21.15.8:0/2994783122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa528101db0 msgr2=0x7fa52819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa51ffff700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa528101db0 0x7fa52819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa51ffff700 1 -- 172.21.15.8:0/2994783122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa528100230 msgr2=0x7fa528196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa51ffff700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa528100230 0x7fa528196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa51ffff700 1 -- 172.21.15.8:0/2994783122 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100095f0 con 0x7fa528101420 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa51ffff700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa528101420 0x7fa528196560 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa51000c010 tx=0x7fa510004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa525ffb700 1 -- 172.21.15.8:0/2994783122 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51001d070 con 0x7fa528101420 2023-11-27T15:25:05.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa525ffb700 1 -- 172.21.15.8:0/2994783122 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5100054f0 con 0x7fa528101420 2023-11-27T15:25:05.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2994783122 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52810bbf0 con 0x7fa528101420 2023-11-27T15:25:05.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa525ffb700 1 -- 172.21.15.8:0/2994783122 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510004ca0 con 0x7fa528101420 2023-11-27T15:25:05.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.452+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2994783122 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52810ba60 con 0x7fa528101420 2023-11-27T15:25:05.460 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.455+0000 7fa525ffb700 1 -- 172.21.15.8:0/2994783122 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fa51000fbc0 con 0x7fa528101420 2023-11-27T15:25:05.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa54206bb80 1 Processor -- start 2023-11-27T15:25:05.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa54206bb80 1 -- start start 2023-11-27T15:25:05.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa525ffb700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5140a11e0 0x7fa5140a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa525ffb700 1 -- 172.21.15.8:0/2994783122 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa5100ce130 con 0x7fa528101420 2023-11-27T15:25:05.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc3536fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3537ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc35380c0 0x558fc3538530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa52cd9c700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5140a11e0 0x7fa5140a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa54206bb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc343c010 con 0x558fc3537500 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa54206bb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc348e850 con 0x558fc3536bb0 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa54206bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc3487ce0 con 0x558fc35380c0 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3537ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.457+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3537ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47182/0 (socket says 172.21.15.8:47182) 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa51ffff700 1 -- 172.21.15.8:0/1686496805 learned_addr learned my addr 172.21.15.8:0/1686496805 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa52cd9c700 1 --2- 172.21.15.8:0/1686496805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc35380c0 0x558fc3538530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa527fff700 1 --2- 172.21.15.8:0/1686496805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc3536fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa51ffff700 1 -- 172.21.15.8:0/1686496805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc35380c0 msgr2=0x558fc3538530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa51ffff700 1 --2- 172.21.15.8:0/1686496805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc35380c0 0x558fc3538530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa51ffff700 1 -- 172.21.15.8:0/1686496805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 msgr2=0x558fc3536fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa51ffff700 1 --2- 172.21.15.8:0/1686496805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc3536fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa51ffff700 1 -- 172.21.15.8:0/1686496805 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fc33095e0 con 0x558fc3537500 2023-11-27T15:25:05.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa527fff700 1 --2- 172.21.15.8:0/1686496805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc3536fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:05.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa52cd9c700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5140a11e0 0x7fa5140a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa5200076f0 tx=0x7fa520007ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa51ffff700 1 --2- 172.21.15.8:0/1686496805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3537ae0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa510097b70 tx=0x7fa51009ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1981bb40c3d9d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa51e7fc700 1 -- 172.21.15.8:0/1686496805 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51009b100 con 0x558fc3537500 2023-11-27T15:25:05.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa51e7fc700 1 -- 172.21.15.8:0/1686496805 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa51009f130 con 0x558fc3537500 2023-11-27T15:25:05.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.458+0000 7fa51e7fc700 1 -- 172.21.15.8:0/1686496805 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100a3030 con 0x558fc3537500 2023-11-27T15:25:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.459+0000 7fa54206bb80 1 -- 172.21.15.8:0/1686496805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 msgr2=0x558fc3537ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.459+0000 7fa54206bb80 1 --2- 172.21.15.8:0/1686496805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3537ae0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa510097b70 tx=0x7fa51009ad10 comp rx=0 tx=0).stop 2023-11-27T15:25:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.459+0000 7fa54206bb80 1 -- 172.21.15.8:0/1686496805 shutdown_connections 2023-11-27T15:25:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.459+0000 7fa54206bb80 1 --2- 172.21.15.8:0/1686496805 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc35380c0 0x558fc3538530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.459+0000 7fa54206bb80 1 --2- 172.21.15.8:0/1686496805 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc3536fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.459+0000 7fa54206bb80 1 --2- 172.21.15.8:0/1686496805 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3537ae0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.459+0000 7fa54206bb80 1 -- 172.21.15.8:0/1686496805 >> 172.21.15.8:0/1686496805 conn(0x558fc352c310 msgr2=0x558fc3531620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.459+0000 7fa54206bb80 1 -- 172.21.15.8:0/1686496805 shutdown_connections 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.459+0000 7fa54206bb80 1 -- 172.21.15.8:0/1686496805 wait complete. 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.460+0000 7fa54206bb80 1 -- start start 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.460+0000 7fa54206bb80 1 Processor -- start 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc35401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3540720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc35380c0 0x558fc3547760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa54206bb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc3429890 con 0x558fc3537500 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa54206bb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc3429530 con 0x558fc3536bb0 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa54206bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc330b8d0 con 0x558fc35380c0 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa54206bb80 10 client.0 _ll_register_callbacks cb 0x558fc3447730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa527fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc35401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3540720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa527fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc35401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45212/0 (socket says 172.21.15.8:45212) 2023-11-27T15:25:05.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa527fff700 1 -- 172.21.15.8:0/2553974798 learned_addr learned my addr 172.21.15.8:0/2553974798 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.461+0000 7fa52cd9c700 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc35380c0 0x558fc3547760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa527fff700 1 -- 172.21.15.8:0/2553974798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc35380c0 msgr2=0x558fc3547760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa527fff700 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc35380c0 0x558fc3547760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa527fff700 1 -- 172.21.15.8:0/2553974798 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 msgr2=0x558fc3540720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa527fff700 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3540720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa527fff700 1 -- 172.21.15.8:0/2553974798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52810ba60 con 0x558fc3536bb0 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa51ffff700 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3540720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa527fff700 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc35401e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa51800bfa0 tx=0x7fa518012730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a66ed034d439ed server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa5077fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa5077fe700 1 -- 172.21.15.8:0/2553974798 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518004230 con 0x558fc3536bb0 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa54206bb80 1 -- 172.21.15.8:0/2553974798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fc3529ad0 con 0x558fc3536bb0 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.462+0000 7fa5077fe700 1 -- 172.21.15.8:0/2553974798 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa518004390 con 0x558fc3536bb0 2023-11-27T15:25:05.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa5077fe700 1 -- 172.21.15.8:0/2553974798 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa518004550 con 0x558fc3536bb0 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa54206bb80 10 client.25166 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa54206bb80 1 -- 172.21.15.8:0/2553974798 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558fc344e140 con 0x558fc3536bb0 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa5077fe700 1 -- 172.21.15.8:0/2553974798 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fc3529ad0 con 0x558fc3536bb0 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa5077fe700 1 -- 172.21.15.8:0/2553974798 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa518003aa0 con 0x558fc3536bb0 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa54206bb80 10 client.25166 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa54206bb80 10 client.25166 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa54206bb80 20 client.25166 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa54206bb80 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x558fc354b4f0 0x558fc354a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa54206bb80 4 client.25166 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.463+0000 7fa54206bb80 1 -- 172.21.15.8:0/2553974798 --> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fc344a050 con 0x558fc354b4f0 2023-11-27T15:25:05.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.464+0000 7fa51ffff700 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x558fc354b4f0 0x558fc354a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.464+0000 7fa51ffff700 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x558fc354b4f0 0x558fc354a8c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf8d467029e4bfb2 server_cookie=ee676d5c56dfb8eb in_seq=0 out_seq=0 2023-11-27T15:25:05.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.464+0000 7fa5077fe700 10 client.25166 ms_handle_connect on v2:172.21.15.29:6838/2619830143 2023-11-27T15:25:05.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa5077fe700 1 -- 172.21.15.8:0/2553974798 <== mds.1 v2:172.21.15.29:6838/2619830143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558fc344a050 con 0x558fc354b4f0 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa5077fe700 10 client.25166 handle_command_reply: tid=0 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 client.25166 shutdown 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 -- 172.21.15.8:0/2553974798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 msgr2=0x558fc35401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc35401e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa51800bfa0 tx=0x7fa518012730 comp rx=0 tx=0).stop 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 -- 172.21.15.8:0/2553974798 shutdown_connections 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x558fc354b4f0 0x558fc354a8c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc35380c0 0x558fc3547760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa5077fe700 0 client.25166 ms_handle_reset on v2:172.21.15.29:6838/2619830143 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc3536bb0 0x558fc35401e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 --2- 172.21.15.8:0/2553974798 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3540720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 -- 172.21.15.8:0/2553974798 >> 172.21.15.8:0/2553974798 conn(0x558fc352c310 msgr2=0x558fc3530d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 -- 172.21.15.8:0/2553974798 shutdown_connections 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.465+0000 7fa54206bb80 1 -- 172.21.15.8:0/2553974798 wait complete. 2023-11-27T15:25:05.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.466+0000 7fa54206bb80 20 client.25166 trim_cache size 0 max 16384 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa54206bb80 1 Processor -- start 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa54206bb80 1 -- start start 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3536bb0 0x558fc353aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3537500 0x558fc3547720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc35380c0 0x558fc353b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa54206bb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc348e850 con 0x558fc3536bb0 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa54206bb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc344a050 con 0x558fc35380c0 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa54206bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc3393ed0 con 0x558fc3537500 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa527fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3536bb0 0x558fc353aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa527fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3536bb0 0x558fc353aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47196/0 (socket says 172.21.15.8:47196) 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa527fff700 1 -- 172.21.15.8:0/3981367977 learned_addr learned my addr 172.21.15.8:0/3981367977 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.490 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.484+0000 7fa52cd9c700 1 --2- 172.21.15.8:0/3981367977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc35380c0 0x558fc353b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa51ffff700 1 --2- 172.21.15.8:0/3981367977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3537500 0x558fc3547720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa527fff700 1 -- 172.21.15.8:0/3981367977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3537500 msgr2=0x558fc3547720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa527fff700 1 --2- 172.21.15.8:0/3981367977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3537500 0x558fc3547720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa527fff700 1 -- 172.21.15.8:0/3981367977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc35380c0 msgr2=0x558fc353b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa527fff700 1 --2- 172.21.15.8:0/3981367977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc35380c0 0x558fc353b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa527fff700 1 -- 172.21.15.8:0/3981367977 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fc344de90 con 0x558fc3536bb0 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa51ffff700 1 --2- 172.21.15.8:0/3981367977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3537500 0x558fc3547720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa527fff700 1 --2- 172.21.15.8:0/3981367977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3536bb0 0x558fc353aef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa51800b930 tx=0x7fa518005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a925133add9825a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa51effd700 1 -- 172.21.15.8:0/3981367977 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51801b5a0 con 0x558fc3536bb0 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa51effd700 1 -- 172.21.15.8:0/3981367977 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa51801a3e0 con 0x558fc3536bb0 2023-11-27T15:25:05.491 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.485+0000 7fa51effd700 1 -- 172.21.15.8:0/3981367977 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51801be80 con 0x558fc3536bb0 2023-11-27T15:25:05.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.486+0000 7fa54206bb80 1 -- 172.21.15.8:0/3981367977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3536bb0 msgr2=0x558fc353aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.486+0000 7fa54206bb80 1 --2- 172.21.15.8:0/3981367977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3536bb0 0x558fc353aef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa51800b930 tx=0x7fa518005d40 comp rx=0 tx=0).stop 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.486+0000 7fa54206bb80 1 -- 172.21.15.8:0/3981367977 shutdown_connections 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.486+0000 7fa54206bb80 1 --2- 172.21.15.8:0/3981367977 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc35380c0 0x558fc353b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.486+0000 7fa54206bb80 1 --2- 172.21.15.8:0/3981367977 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3536bb0 0x558fc353aef0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.486+0000 7fa54206bb80 1 --2- 172.21.15.8:0/3981367977 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3537500 0x558fc3547720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.486+0000 7fa54206bb80 1 -- 172.21.15.8:0/3981367977 >> 172.21.15.8:0/3981367977 conn(0x558fc352c310 msgr2=0x558fc353e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.486+0000 7fa54206bb80 1 -- 172.21.15.8:0/3981367977 shutdown_connections 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.486+0000 7fa54206bb80 1 -- 172.21.15.8:0/3981367977 wait complete. 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.486+0000 7fa54206bb80 1 -- start start 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.487+0000 7fa54206bb80 1 Processor -- start 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.487+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3536bb0 0x558fc3542570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.487+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3542ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.487+0000 7fa54206bb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc35380c0 0x558fc359f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.487+0000 7fa54206bb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc3543150 con 0x558fc3537500 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.487+0000 7fa54206bb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc359f8d0 con 0x558fc35380c0 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.487+0000 7fa54206bb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc359fa40 con 0x558fc3536bb0 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.487+0000 7fa54206bb80 10 client.0 _ll_register_callbacks cb 0x7fa51800b9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.487+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3542ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa51ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3542ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47206/0 (socket says 172.21.15.8:47206) 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa51ffff700 1 -- 172.21.15.8:0/207227744 learned_addr learned my addr 172.21.15.8:0/207227744 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa527fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3536bb0 0x558fc3542570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa52cd9c700 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc35380c0 0x558fc359f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa51ffff700 1 -- 172.21.15.8:0/207227744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3536bb0 msgr2=0x558fc3542570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa51ffff700 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3536bb0 0x558fc3542570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa51ffff700 1 -- 172.21.15.8:0/207227744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc35380c0 msgr2=0x558fc359f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa51ffff700 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc35380c0 0x558fc359f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa51ffff700 1 -- 172.21.15.8:0/207227744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa518020040 con 0x558fc3537500 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa527fff700 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3536bb0 0x558fc3542570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa51ffff700 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3542ab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa510005f50 tx=0x7fa510005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430a6972b4f95df6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa5077fe700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa5077fe700 1 -- 172.21.15.8:0/207227744 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5100a3540 con 0x558fc3537500 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.488+0000 7fa5077fe700 1 -- 172.21.15.8:0/207227744 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa51009b100 con 0x558fc3537500 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa54206bb80 1 -- 172.21.15.8:0/207227744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa51009f130 con 0x558fc3537500 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa5077fe700 1 -- 172.21.15.8:0/207227744 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5100cb9c0 con 0x558fc3537500 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa54206bb80 10 client.15150 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa54206bb80 1 -- 172.21.15.8:0/207227744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558fc329ec10 con 0x558fc3537500 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa5077fe700 1 -- 172.21.15.8:0/207227744 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5100cb020 con 0x558fc3537500 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa5077fe700 1 -- 172.21.15.8:0/207227744 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa5100b9240 con 0x558fc3537500 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa54206bb80 10 client.15150 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:05.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa54206bb80 10 client.15150 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa54206bb80 20 client.15150 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa54206bb80 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x558fc35a4360 0x558fc35a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa54206bb80 4 client.15150 mds_command: new command op to 24593 tid=0[{"prefix": "status"}] 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.489+0000 7fa54206bb80 1 -- 172.21.15.8:0/207227744 --> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558fc3456610 con 0x558fc35a4360 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.490+0000 7fa527fff700 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x558fc35a4360 0x558fc35a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.490+0000 7fa527fff700 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x558fc35a4360 0x558fc35a0cb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a036f66a6fff30d server_cookie=785b3be6fa084979 in_seq=0 out_seq=0 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.490+0000 7fa5077fe700 10 client.15150 ms_handle_connect on v2:172.21.15.29:6838/2619830143 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.490+0000 7fa5077fe700 1 -- 172.21.15.8:0/207227744 <== mds.1 v2:172.21.15.29:6838/2619830143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x558fc3456610 con 0x558fc35a4360 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.490+0000 7fa5077fe700 10 client.15150 handle_command_reply: tid=0 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.490+0000 7fa54206bb80 1 client.15150 shutdown 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 1 -- 172.21.15.8:0/207227744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 msgr2=0x558fc3542ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3542ab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa510005f50 tx=0x7fa510005ab0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 1 -- 172.21.15.8:0/207227744 shutdown_connections 2023-11-27T15:25:05.499 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x558fc35a4360 0x558fc35a0cb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x558fc3536bb0 0x558fc3542570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x558fc35380c0 0x558fc359f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa5077fe700 0 client.15150 ms_handle_reset on v2:172.21.15.29:6838/2619830143 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 1 --2- 172.21.15.8:0/207227744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x558fc3537500 0x558fc3542ab0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 1 -- 172.21.15.8:0/207227744 >> 172.21.15.8:0/207227744 conn(0x558fc352c310 msgr2=0x558fc353e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 1 -- 172.21.15.8:0/207227744 shutdown_connections 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 1 -- 172.21.15.8:0/207227744 wait complete. 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.491+0000 7fa54206bb80 20 client.15150 trim_cache size 0 max 16384 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:25:05.500 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 1, 2023-11-27T15:25:05.501 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24593, 2023-11-27T15:25:05.501 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2023-11-27T15:25:05.501 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2023-11-27T15:25:05.501 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2023-11-27T15:25:05.501 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 68.059480336999997, 2023-11-27T15:25:05.501 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 24, 2023-11-27T15:25:05.501 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 90, 2023-11-27T15:25:05.501 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 88, 2023-11-27T15:25:05.501 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 105.15022272500001 2023-11-27T15:25:05.501 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:25:05.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2994783122 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5140a11e0 msgr2=0x7fa5140a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5140a11e0 0x7fa5140a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa5200076f0 tx=0x7fa520007ab0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2994783122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa528101420 msgr2=0x7fa528196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa528101420 0x7fa528196560 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa51000c010 tx=0x7fa510004730 comp rx=0 tx=0).stop 2023-11-27T15:25:05.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2994783122 shutdown_connections 2023-11-27T15:25:05.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa5140a11e0 0x7fa5140a3690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa528101db0 0x7fa52819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa528100230 0x7fa528196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 --2- 172.21.15.8:0/2994783122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa528101420 0x7fa528196560 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2994783122 >> 172.21.15.8:0/2994783122 conn(0x7fa5280fb810 msgr2=0x7fa528104950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2994783122 shutdown_connections 2023-11-27T15:25:05.503 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.492+0000 7fa52e7ff700 1 -- 172.21.15.8:0/2994783122 wait complete. 2023-11-27T15:25:05.514 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-27T15:25:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.715+0000 7f8c0679d700 1 -- 172.21.15.8:0/527384853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c000734c0 msgr2=0x7f8c00073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.715+0000 7f8c0679d700 1 --2- 172.21.15.8:0/527384853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c000734c0 0x7f8c00073930 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009990 tx=0x7f8bf0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.715+0000 7f8c0679d700 1 -- 172.21.15.8:0/527384853 shutdown_connections 2023-11-27T15:25:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.715+0000 7f8c0679d700 1 --2- 172.21.15.8:0/527384853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c00107f30 0x7f8c0010a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.715+0000 7f8c0679d700 1 --2- 172.21.15.8:0/527384853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c00074a90 0x7f8c00072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.715+0000 7f8c0679d700 1 --2- 172.21.15.8:0/527384853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c000734c0 0x7f8c00073930 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.715+0000 7f8c0679d700 1 -- 172.21.15.8:0/527384853 >> 172.21.15.8:0/527384853 conn(0x7f8c000fb6b0 msgr2=0x7f8c000fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.715+0000 7f8c0679d700 1 -- 172.21.15.8:0/527384853 shutdown_connections 2023-11-27T15:25:05.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.715+0000 7f8c0679d700 1 -- 172.21.15.8:0/527384853 wait complete. 2023-11-27T15:25:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.716+0000 7f8c0679d700 1 Processor -- start 2023-11-27T15:25:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c0679d700 1 -- start start 2023-11-27T15:25:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c0679d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c000734c0 0x7f8c00197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c0679d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c00074a90 0x7f8c00198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.723 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c0679d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c00107f30 0x7f8c0019f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c0679d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0019fb50 con 0x7f8c000734c0 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c0679d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0019fc90 con 0x7f8c00074a90 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c0679d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0019fe20 con 0x7f8c00107f30 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c0579b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c000734c0 0x7f8c00197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c04f9a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c00074a90 0x7f8c00198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c05f9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c00107f30 0x7f8c0019f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c04f9a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c00074a90 0x7f8c00198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45246/0 (socket says 172.21.15.8:45246) 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c0579b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c000734c0 0x7f8c00197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47232/0 (socket says 172.21.15.8:47232) 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.717+0000 7f8c04f9a700 1 -- 172.21.15.8:0/2086059897 learned_addr learned my addr 172.21.15.8:0/2086059897 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8c05f9c700 1 -- 172.21.15.8:0/2086059897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c00074a90 msgr2=0x7f8c00198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8c05f9c700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c00074a90 0x7f8c00198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8c05f9c700 1 -- 172.21.15.8:0/2086059897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c000734c0 msgr2=0x7f8c00197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8c05f9c700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c000734c0 0x7f8c00197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8c05f9c700 1 -- 172.21.15.8:0/2086059897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf00095f0 con 0x7f8c00107f30 2023-11-27T15:25:05.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8c0579b700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c000734c0 0x7f8c00197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:05.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8c05f9c700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c00107f30 0x7f8c0019f580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800efc0 tx=0x7f8bf800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8bf67fc700 1 -- 172.21.15.8:0/2086059897 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8018070 con 0x7f8c00107f30 2023-11-27T15:25:05.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8c0679d700 1 -- 172.21.15.8:0/2086059897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0010dc90 con 0x7f8c00107f30 2023-11-27T15:25:05.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8bf67fc700 1 -- 172.21.15.8:0/2086059897 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf800f040 con 0x7f8c00107f30 2023-11-27T15:25:05.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8bf67fc700 1 -- 172.21.15.8:0/2086059897 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800cd40 con 0x7f8c00107f30 2023-11-27T15:25:05.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.718+0000 7f8c0679d700 1 -- 172.21.15.8:0/2086059897 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0010db00 con 0x7f8c00107f30 2023-11-27T15:25:05.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.723+0000 7f8bf67fc700 1 -- 172.21.15.8:0/2086059897 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8bf8003a00 con 0x7f8c00107f30 2023-11-27T15:25:05.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.724+0000 7f8c1b389b80 1 Processor -- start 2023-11-27T15:25:05.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.724+0000 7f8c1b389b80 1 -- start start 2023-11-27T15:25:05.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.724+0000 7f8bf67fc700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8bec0a1260 0x7f8bec0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b671250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b671cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 0x56158b672700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b576010 con 0x56158b671790 2023-11-27T15:25:05.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b5c8850 con 0x56158b672260 2023-11-27T15:25:05.731 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8bf67fc700 1 -- 172.21.15.8:0/2086059897 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8bf80cc8e0 con 0x7f8c00107f30 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b5c1ce0 con 0x56158b670e40 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8bec0a1260 0x7f8bec0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c05f9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 0x56158b672700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c05f9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 0x56158b672700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45260/0 (socket says 172.21.15.8:45260) 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c05f9c700 1 -- 172.21.15.8:0/3551260832 learned_addr learned my addr 172.21.15.8:0/3551260832 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c0579b700 1 --2- 172.21.15.8:0/3551260832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b671250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c05f9c700 1 -- 172.21.15.8:0/3551260832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 msgr2=0x56158b671250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.725+0000 7f8c05f9c700 1 --2- 172.21.15.8:0/3551260832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b671250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.726+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/3551260832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b671cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.726+0000 7f8c05f9c700 1 -- 172.21.15.8:0/3551260832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 msgr2=0x56158b671cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.726+0000 7f8c05f9c700 1 --2- 172.21.15.8:0/3551260832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b671cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.726+0000 7f8c05f9c700 1 -- 172.21.15.8:0/3551260832 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56158b4435e0 con 0x56158b672260 2023-11-27T15:25:05.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.726+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/3551260832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b671cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:05.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.726+0000 7f8c05f9c700 1 --2- 172.21.15.8:0/3551260832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 0x56158b672700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8096320 tx=0x7f8bf8099680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9f9c366d133a0da server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.726+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8bec0a1260 0x7f8bec0a3710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0005cd0 tx=0x7f8bf0005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.726+0000 7f8beb7fe700 1 -- 172.21.15.8:0/3551260832 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8004ad0 con 0x56158b672260 2023-11-27T15:25:05.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.726+0000 7f8beb7fe700 1 -- 172.21.15.8:0/3551260832 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf80998b0 con 0x56158b672260 2023-11-27T15:25:05.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.726+0000 7f8beb7fe700 1 -- 172.21.15.8:0/3551260832 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8099b10 con 0x56158b672260 2023-11-27T15:25:05.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.727+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3551260832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 msgr2=0x56158b672700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.727+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/3551260832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 0x56158b672700 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8096320 tx=0x7f8bf8099680 comp rx=0 tx=0).stop 2023-11-27T15:25:05.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.727+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3551260832 shutdown_connections 2023-11-27T15:25:05.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.727+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/3551260832 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b671250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.727+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/3551260832 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 0x56158b672700 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.727+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/3551260832 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b671cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.727+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3551260832 >> 172.21.15.8:0/3551260832 conn(0x56158b666560 msgr2=0x56158b66b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.727+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3551260832 shutdown_connections 2023-11-27T15:25:05.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.727+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3551260832 wait complete. 2023-11-27T15:25:05.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.728+0000 7f8c1b389b80 1 -- start start 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.728+0000 7f8c1b389b80 1 Processor -- start 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b670e40 0x56158b67a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b67ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b672260 0x56158b681cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b563890 con 0x56158b671790 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b563530 con 0x56158b670e40 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b4458d0 con 0x56158b672260 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c1b389b80 10 client.0 _ll_register_callbacks cb 0x56158b581730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c0579b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b670e40 0x56158b67a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c04f9a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b67ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c0579b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b670e40 0x56158b67a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45266/0 (socket says 172.21.15.8:45266) 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c0579b700 1 -- 172.21.15.8:0/3713315172 learned_addr learned my addr 172.21.15.8:0/3713315172 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c05f9c700 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b672260 0x56158b681cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c0579b700 1 -- 172.21.15.8:0/3713315172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b672260 msgr2=0x56158b681cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c0579b700 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b672260 0x56158b681cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c0579b700 1 -- 172.21.15.8:0/3713315172 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 msgr2=0x56158b67ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c0579b700 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b67ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.729+0000 7f8c0579b700 1 -- 172.21.15.8:0/3713315172 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0010db00 con 0x56158b670e40 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.730+0000 7f8c0579b700 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b670e40 0x56158b67a750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc012d20 tx=0x7f8bfc00f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02648e10260c18 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.730+0000 7f8be8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.730+0000 7f8be8ff9700 1 -- 172.21.15.8:0/3713315172 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc017070 con 0x56158b670e40 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.730+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3713315172 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56158b663d20 con 0x56158b670e40 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.730+0000 7f8be8ff9700 1 -- 172.21.15.8:0/3713315172 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8bfc00fb80 con 0x56158b670e40 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.730+0000 7f8be8ff9700 1 -- 172.21.15.8:0/3713315172 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bfc00fd40 con 0x56158b670e40 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.730+0000 7f8c1b389b80 10 client.25184 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.730+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3713315172 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56158b588140 con 0x56158b670e40 2023-11-27T15:25:05.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.731+0000 7f8be8ff9700 1 -- 172.21.15.8:0/3713315172 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56158b663d20 con 0x56158b670e40 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.731+0000 7f8be8ff9700 1 -- 172.21.15.8:0/3713315172 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8bfc003c10 con 0x56158b670e40 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.731+0000 7f8c1b389b80 10 client.25184 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.731+0000 7f8c1b389b80 10 client.25184 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.731+0000 7f8c1b389b80 20 client.25184 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.731+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56158b685a00 0x56158b684da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.731+0000 7f8c1b389b80 4 client.25184 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.731+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3713315172 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56158b584050 con 0x56158b685a00 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.731+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56158b685a00 0x56158b684da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.732+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56158b685a00 0x56158b684da0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff3caf5bd56589a server_cookie=2aa584fc86e65588 in_seq=0 out_seq=0 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.732+0000 7f8be8ff9700 10 client.25184 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.732+0000 7f8be8ff9700 1 -- 172.21.15.8:0/3713315172 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56158b584050 con 0x56158b685a00 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.732+0000 7f8be8ff9700 10 client.25184 handle_command_reply: tid=0 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.732+0000 7f8c1b389b80 1 client.25184 shutdown 2023-11-27T15:25:05.741 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3713315172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b670e40 msgr2=0x56158b67a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b670e40 0x56158b67a750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc012d20 tx=0x7f8bfc00f6a0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3713315172 shutdown_connections 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56158b685a00 0x56158b684da0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b672260 0x56158b681cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b670e40 0x56158b67a750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/3713315172 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b67ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8be8ff9700 0 client.25184 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3713315172 >> 172.21.15.8:0/3713315172 conn(0x56158b666560 msgr2=0x56158b66afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3713315172 shutdown_connections 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 1 -- 172.21.15.8:0/3713315172 wait complete. 2023-11-27T15:25:05.742 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.733+0000 7f8c1b389b80 20 client.25184 trim_cache size 0 max 16384 2023-11-27T15:25:05.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c1b389b80 1 Processor -- start 2023-11-27T15:25:05.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c1b389b80 1 -- start start 2023-11-27T15:25:05.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b684da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b6852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 0x56158b681bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b671b60 con 0x56158b671790 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b678ec0 con 0x56158b672260 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b4cded0 con 0x56158b670e40 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c04f9a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b6852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c04f9a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b6852e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47258/0 (socket says 172.21.15.8:47258) 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c04f9a700 1 -- 172.21.15.8:0/4044788375 learned_addr learned my addr 172.21.15.8:0/4044788375 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c05f9c700 1 --2- 172.21.15.8:0/4044788375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 0x56158b681bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c0579b700 1 --2- 172.21.15.8:0/4044788375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b684da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.752+0000 7f8c04f9a700 1 -- 172.21.15.8:0/4044788375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 msgr2=0x56158b684da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.753+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/4044788375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b684da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.753+0000 7f8c04f9a700 1 -- 172.21.15.8:0/4044788375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 msgr2=0x56158b681bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.753+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/4044788375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 0x56158b681bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.753+0000 7f8c04f9a700 1 -- 172.21.15.8:0/4044788375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56158b587e90 con 0x56158b671790 2023-11-27T15:25:05.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.753+0000 7f8c0579b700 1 --2- 172.21.15.8:0/4044788375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b684da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:05.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.753+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/4044788375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b6852e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x56158b56b880 tx=0x7f8bf001a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35887a0c501aec2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.753+0000 7f8bebfff700 1 -- 172.21.15.8:0/4044788375 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf001d070 con 0x56158b671790 2023-11-27T15:25:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.753+0000 7f8bebfff700 1 -- 172.21.15.8:0/4044788375 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf00056a0 con 0x56158b671790 2023-11-27T15:25:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.753+0000 7f8bebfff700 1 -- 172.21.15.8:0/4044788375 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00047a0 con 0x56158b671790 2023-11-27T15:25:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 -- 172.21.15.8:0/4044788375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 msgr2=0x56158b6852e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/4044788375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b6852e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x56158b56b880 tx=0x7f8bf001a070 comp rx=0 tx=0).stop 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 -- 172.21.15.8:0/4044788375 shutdown_connections 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/4044788375 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b684da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/4044788375 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b672260 0x56158b681bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/4044788375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b671790 0x56158b6852e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 -- 172.21.15.8:0/4044788375 >> 172.21.15.8:0/4044788375 conn(0x56158b666560 msgr2=0x56158b68e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 -- 172.21.15.8:0/4044788375 shutdown_connections 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 -- 172.21.15.8:0/4044788375 wait complete. 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 -- start start 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.754+0000 7f8c1b389b80 1 Processor -- start 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b67ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b671790 0x56158b67cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c1b389b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b672260 0x56158b6d98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b67d530 con 0x56158b672260 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b67d670 con 0x56158b671790 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c1b389b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56158b6d9e30 con 0x56158b670e40 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c1b389b80 10 client.0 _ll_register_callbacks cb 0x7f8bfc0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c04f9a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b671790 0x56158b67cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c05f9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b672260 0x56158b6d98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c04f9a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b671790 0x56158b67cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45280/0 (socket says 172.21.15.8:45280) 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c04f9a700 1 -- 172.21.15.8:0/28696031 learned_addr learned my addr 172.21.15.8:0/28696031 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c0579b700 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b67ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.755+0000 7f8c04f9a700 1 -- 172.21.15.8:0/28696031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 msgr2=0x56158b67ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.756+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b67ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.756+0000 7f8c04f9a700 1 -- 172.21.15.8:0/28696031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b672260 msgr2=0x56158b6d98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.756+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b672260 0x56158b6d98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.756+0000 7f8c04f9a700 1 -- 172.21.15.8:0/28696031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf002d040 con 0x56158b671790 2023-11-27T15:25:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.756+0000 7f8c05f9c700 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b672260 0x56158b6d98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.756+0000 7f8c04f9a700 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b671790 0x56158b67cf60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00095c0 tx=0x7f8bf0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7353bbffaf69d4e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.756+0000 7f8be8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.756+0000 7f8be8ff9700 1 -- 172.21.15.8:0/28696031 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00339f0 con 0x56158b671790 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.756+0000 7f8be8ff9700 1 -- 172.21.15.8:0/28696031 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8bf0033b50 con 0x56158b671790 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.756+0000 7f8c1b389b80 1 -- 172.21.15.8:0/28696031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bf80998b0 con 0x56158b671790 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8be8ff9700 1 -- 172.21.15.8:0/28696031 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf0033d10 con 0x56158b671790 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8c1b389b80 10 client.25190 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8c1b389b80 1 -- 172.21.15.8:0/28696031 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56158b3d8c10 con 0x56158b671790 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8be8ff9700 1 -- 172.21.15.8:0/28696031 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf80998b0 con 0x56158b671790 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8be8ff9700 1 -- 172.21.15.8:0/28696031 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8bf000f620 con 0x56158b671790 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8c1b389b80 10 client.25190 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8c1b389b80 10 client.25190 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8c1b389b80 20 client.25190 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56158b6de6d0 0x56158b6db080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:05.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8c1b389b80 4 client.25190 mds_command: new command op to 24595 tid=0[{"prefix": "status"}] 2023-11-27T15:25:05.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.757+0000 7f8c1b389b80 1 -- 172.21.15.8:0/28696031 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56158b590610 con 0x56158b6de6d0 2023-11-27T15:25:05.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8c0579b700 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56158b6de6d0 0x56158b6db080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:05.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8c0579b700 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56158b6de6d0 0x56158b6db080 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1691bee61ad13949 server_cookie=dd17e2d3a1fd808f in_seq=0 out_seq=0 2023-11-27T15:25:05.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8be8ff9700 10 client.25190 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:05.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8be8ff9700 1 -- 172.21.15.8:0/28696031 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56158b590610 con 0x56158b6de6d0 2023-11-27T15:25:05.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8be8ff9700 10 client.25190 handle_command_reply: tid=0 2023-11-27T15:25:05.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8c1b389b80 1 client.25190 shutdown 2023-11-27T15:25:05.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8c1b389b80 1 -- 172.21.15.8:0/28696031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b671790 msgr2=0x56158b67cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b671790 0x56158b67cf60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00095c0 tx=0x7f8bf0005e80 comp rx=0 tx=0).stop 2023-11-27T15:25:05.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8c1b389b80 1 -- 172.21.15.8:0/28696031 shutdown_connections 2023-11-27T15:25:05.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56158b6de6d0 0x56158b6db080 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56158b670e40 0x56158b67ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8be8ff9700 0 client.25190 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56158b672260 0x56158b6d98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.758+0000 7f8c1b389b80 1 --2- 172.21.15.8:0/28696031 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56158b671790 0x56158b67cf60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c1b389b80 1 -- 172.21.15.8:0/28696031 >> 172.21.15.8:0/28696031 conn(0x56158b666560 msgr2=0x56158b684da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c1b389b80 1 -- 172.21.15.8:0/28696031 shutdown_connections 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c1b389b80 1 -- 172.21.15.8:0/28696031 wait complete. 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c1b389b80 20 client.25190 trim_cache size 0 max 16384 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 0, 2023-11-27T15:25:05.768 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24595, 2023-11-27T15:25:05.769 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2023-11-27T15:25:05.769 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2023-11-27T15:25:05.769 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2023-11-27T15:25:05.769 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 98.588714053000004, 2023-11-27T15:25:05.769 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 24, 2023-11-27T15:25:05.769 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 90, 2023-11-27T15:25:05.769 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 88, 2023-11-27T15:25:05.769 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 107.75297864700001 2023-11-27T15:25:05.769 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c0679d700 1 -- 172.21.15.8:0/2086059897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8bec0a1260 msgr2=0x7f8bec0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c0679d700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8bec0a1260 0x7f8bec0a3710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0005cd0 tx=0x7f8bf0005bc0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c0679d700 1 -- 172.21.15.8:0/2086059897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c00107f30 msgr2=0x7f8c0019f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c0679d700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c00107f30 0x7f8c0019f580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800efc0 tx=0x7f8bf800c3a0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c0679d700 1 -- 172.21.15.8:0/2086059897 shutdown_connections 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c0679d700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8bec0a1260 0x7f8bec0a3710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c0679d700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8c00107f30 0x7f8c0019f580 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c0679d700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c000734c0 0x7f8c00197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c0679d700 1 --2- 172.21.15.8:0/2086059897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8c00074a90 0x7f8c00198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.759+0000 7f8c0679d700 1 -- 172.21.15.8:0/2086059897 >> 172.21.15.8:0/2086059897 conn(0x7f8c000fb6b0 msgr2=0x7f8c000fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.760+0000 7f8c0679d700 1 -- 172.21.15.8:0/2086059897 shutdown_connections 2023-11-27T15:25:05.770 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:05.760+0000 7f8c0679d700 1 -- 172.21.15.8:0/2086059897 wait complete. 2023-11-27T15:25:05.780 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:05 smithi008 ceph-mon[128788]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-27T15:25:05.780 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:05 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1606623284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:05.782 INFO:tasks.cephadm:Setup complete, yielding 2023-11-27T15:25:05.783 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-27T15:25:05.791 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi008.front.sepia.ceph.com 2023-11-27T15:25:05.792 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph orch status' 2023-11-27T15:25:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:05 smithi060 ceph-mon[131518]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-27T15:25:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:05 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1606623284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:05 smithi029 ceph-mon[131166]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-27T15:25:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:05 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1606623284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:06.653 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:25:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:07 smithi060 ceph-mon[131518]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-11-27T15:25:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:07 smithi008 ceph-mon[128788]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-11-27T15:25:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:07 smithi029 ceph-mon[131166]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-11-27T15:25:08.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.195+0000 7fb6d5af1700 1 -- 172.21.15.8:0/153398333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6d0074b20 msgr2=0x7fb6d0072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:08.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.195+0000 7fb6d5af1700 1 --2- 172.21.15.8:0/153398333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6d0074b20 0x7fb6d0072f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009990 tx=0x7fb6c0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.195+0000 7fb6d5af1700 1 -- 172.21.15.8:0/153398333 shutdown_connections 2023-11-27T15:25:08.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.195+0000 7fb6d5af1700 1 --2- 172.21.15.8:0/153398333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6d01081c0 0x7fb6d010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.195+0000 7fb6d5af1700 1 --2- 172.21.15.8:0/153398333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6d0074b20 0x7fb6d0072f80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.195+0000 7fb6d5af1700 1 --2- 172.21.15.8:0/153398333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6d0073550 0x7fb6d00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.195+0000 7fb6d5af1700 1 -- 172.21.15.8:0/153398333 >> 172.21.15.8:0/153398333 conn(0x7fb6d00fb900 msgr2=0x7fb6d00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.195+0000 7fb6d5af1700 1 -- 172.21.15.8:0/153398333 shutdown_connections 2023-11-27T15:25:08.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.195+0000 7fb6d5af1700 1 -- 172.21.15.8:0/153398333 wait complete. 2023-11-27T15:25:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.196+0000 7fb6d5af1700 1 Processor -- start 2023-11-27T15:25:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.196+0000 7fb6d5af1700 1 -- start start 2023-11-27T15:25:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6d5af1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6d0073550 0x7fb6d0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6d5af1700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6d0074b20 0x7fb6d0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6d5af1700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6d01081c0 0x7fb6d019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6d5af1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d019fd90 con 0x7fb6d0073550 2023-11-27T15:25:08.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6d5af1700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d019fed0 con 0x7fb6d01081c0 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6d5af1700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d01a0060 con 0x7fb6d0074b20 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6cf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6d0073550 0x7fb6d0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6cf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6d0073550 0x7fb6d0198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47294/0 (socket says 172.21.15.8:47294) 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6cf7fe700 1 -- 172.21.15.8:0/637832853 learned_addr learned my addr 172.21.15.8:0/637832853 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6ceffd700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6d0074b20 0x7fb6d0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.197+0000 7fb6cffff700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6d01081c0 0x7fb6d019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6cf7fe700 1 -- 172.21.15.8:0/637832853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6d0074b20 msgr2=0x7fb6d0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6cf7fe700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6d0074b20 0x7fb6d0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6cf7fe700 1 -- 172.21.15.8:0/637832853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6d01081c0 msgr2=0x7fb6d019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6cf7fe700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6d01081c0 0x7fb6d019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6cf7fe700 1 -- 172.21.15.8:0/637832853 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8009c40 con 0x7fb6d0073550 2023-11-27T15:25:08.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6cf7fe700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6d0073550 0x7fb6d0198200 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0005910 tx=0x7fb6c00057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:08.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6ccff9700 1 -- 172.21.15.8:0/637832853 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c001d070 con 0x7fb6d0073550 2023-11-27T15:25:08.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6ccff9700 1 -- 172.21.15.8:0/637832853 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6c000fbb0 con 0x7fb6d0073550 2023-11-27T15:25:08.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6d5af1700 1 -- 172.21.15.8:0/637832853 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c00095f0 con 0x7fb6d0073550 2023-11-27T15:25:08.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6ccff9700 1 -- 172.21.15.8:0/637832853 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00173f0 con 0x7fb6d0073550 2023-11-27T15:25:08.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.198+0000 7fb6d5af1700 1 -- 172.21.15.8:0/637832853 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d010de50 con 0x7fb6d0073550 2023-11-27T15:25:08.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.199+0000 7fb6d5af1700 1 -- 172.21.15.8:0/637832853 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d0106870 con 0x7fb6d0073550 2023-11-27T15:25:08.206 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.201+0000 7fb6ccff9700 1 -- 172.21.15.8:0/637832853 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb6c0021410 con 0x7fb6d0073550 2023-11-27T15:25:08.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.203+0000 7fb6ccff9700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb6b80a11e0 0x7fb6b80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:08.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.203+0000 7fb6ccff9700 1 -- 172.21.15.8:0/637832853 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb6c00ce140 con 0x7fb6d0073550 2023-11-27T15:25:08.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.203+0000 7fb6ceffd700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb6b80a11e0 0x7fb6b80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:08.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.204+0000 7fb6ceffd700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb6b80a11e0 0x7fb6b80a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009c60 tx=0x7fb6c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:08.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.206+0000 7fb6ccff9700 1 -- 172.21.15.8:0/637832853 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb6c0097450 con 0x7fb6d0073550 2023-11-27T15:25:08.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.466+0000 7fb6d5af1700 1 -- 172.21.15.8:0/637832853 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6d0060f30 con 0x7fb6b80a11e0 2023-11-27T15:25:08.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.467+0000 7fb6ccff9700 1 -- 172.21.15.8:0/637832853 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb6d0060f30 con 0x7fb6b80a11e0 2023-11-27T15:25:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:Backend: cephadm 2023-11-27T15:25:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:Available: Yes 2023-11-27T15:25:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:Paused: No 2023-11-27T15:25:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.469+0000 7fb6d5af1700 1 -- 172.21.15.8:0/637832853 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb6b80a11e0 msgr2=0x7fb6b80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.469+0000 7fb6d5af1700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb6b80a11e0 0x7fb6b80a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009c60 tx=0x7fb6c4009260 comp rx=0 tx=0).stop 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.469+0000 7fb6d5af1700 1 -- 172.21.15.8:0/637832853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6d0073550 msgr2=0x7fb6d0198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.469+0000 7fb6d5af1700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6d0073550 0x7fb6d0198200 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0005910 tx=0x7fb6c00057a0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.470+0000 7fb6d5af1700 1 -- 172.21.15.8:0/637832853 shutdown_connections 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.470+0000 7fb6d5af1700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb6d01081c0 0x7fb6d019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.470+0000 7fb6d5af1700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6d0073550 0x7fb6d0198200 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.470+0000 7fb6d5af1700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb6b80a11e0 0x7fb6b80a3690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.470+0000 7fb6d5af1700 1 --2- 172.21.15.8:0/637832853 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb6d0074b20 0x7fb6d0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.470+0000 7fb6d5af1700 1 -- 172.21.15.8:0/637832853 >> 172.21.15.8:0/637832853 conn(0x7fb6d00fb900 msgr2=0x7fb6d00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.470+0000 7fb6d5af1700 1 -- 172.21.15.8:0/637832853 shutdown_connections 2023-11-27T15:25:08.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:08.470+0000 7fb6d5af1700 1 -- 172.21.15.8:0/637832853 wait complete. 2023-11-27T15:25:08.958 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph orch ps' 2023-11-27T15:25:09.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:09 smithi060 ceph-mon[131518]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:09.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:09 smithi060 ceph-mon[131518]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:09.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:09 smithi008 ceph-mon[128788]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:09.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:09 smithi008 ceph-mon[128788]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:09.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:09 smithi029 ceph-mon[131166]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:09.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:09 smithi029 ceph-mon[131166]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:09.839 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:25:10.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:10 smithi008 ceph-mon[128788]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-11-27T15:25:10.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:10 smithi060 ceph-mon[131518]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-11-27T15:25:10.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:10 smithi029 ceph-mon[131166]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-11-27T15:25:11.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.399+0000 7ffb5266a700 1 -- 172.21.15.8:0/3962594273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4c1002f0 msgr2=0x7ffb4c100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:11.403 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.399+0000 7ffb5266a700 1 --2- 172.21.15.8:0/3962594273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c100700 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009990 tx=0x7ffb3c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:11.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.399+0000 7ffb5266a700 1 -- 172.21.15.8:0/3962594273 shutdown_connections 2023-11-27T15:25:11.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.399+0000 7ffb5266a700 1 --2- 172.21.15.8:0/3962594273 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffb4c101e80 0x7ffb4c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:11.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.399+0000 7ffb5266a700 1 --2- 172.21.15.8:0/3962594273 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c100700 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:11.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.399+0000 7ffb5266a700 1 --2- 172.21.15.8:0/3962594273 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:11.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.399+0000 7ffb5266a700 1 -- 172.21.15.8:0/3962594273 >> 172.21.15.8:0/3962594273 conn(0x7ffb4c0fb900 msgr2=0x7ffb4c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:11.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.399+0000 7ffb5266a700 1 -- 172.21.15.8:0/3962594273 shutdown_connections 2023-11-27T15:25:11.404 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.399+0000 7ffb5266a700 1 -- 172.21.15.8:0/3962594273 wait complete. 2023-11-27T15:25:11.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.400+0000 7ffb5266a700 1 Processor -- start 2023-11-27T15:25:11.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.400+0000 7ffb5266a700 1 -- start start 2023-11-27T15:25:11.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.400+0000 7ffb5266a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:11.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.400+0000 7ffb5266a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:11.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.400+0000 7ffb5266a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4c101e80 0x7ffb4c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:11.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.400+0000 7ffb5266a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c19dc20 con 0x7ffb4c1002f0 2023-11-27T15:25:11.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.400+0000 7ffb5266a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c19dd60 con 0x7ffb4c101e80 2023-11-27T15:25:11.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.400+0000 7ffb5266a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c19def0 con 0x7ffb4c1014f0 2023-11-27T15:25:11.406 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb4bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb4bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46784/0 (socket says 172.21.15.8:46784) 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb4bfff700 1 -- 172.21.15.8:0/2911852906 learned_addr learned my addr 172.21.15.8:0/2911852906 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb4b7fe700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb50c07700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4c101e80 0x7ffb4c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb4bfff700 1 -- 172.21.15.8:0/2911852906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb4c1014f0 msgr2=0x7ffb4c1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb4bfff700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb4bfff700 1 -- 172.21.15.8:0/2911852906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4c101e80 msgr2=0x7ffb4c19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb4bfff700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4c101e80 0x7ffb4c19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb4bfff700 1 -- 172.21.15.8:0/2911852906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c0095f0 con 0x7ffb4c1002f0 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb50c07700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4c101e80 0x7ffb4c19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:11.407 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.401+0000 7ffb4bfff700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c196090 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c004740 tx=0x7ffb3c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.402+0000 7ffb497fa700 1 -- 172.21.15.8:0/2911852906 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c01d070 con 0x7ffb4c1002f0 2023-11-27T15:25:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.402+0000 7ffb497fa700 1 -- 172.21.15.8:0/2911852906 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb3c021410 con 0x7ffb4c1002f0 2023-11-27T15:25:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.402+0000 7ffb5266a700 1 -- 172.21.15.8:0/2911852906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c10bcb0 con 0x7ffb4c1002f0 2023-11-27T15:25:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.402+0000 7ffb497fa700 1 -- 172.21.15.8:0/2911852906 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c017650 con 0x7ffb4c1002f0 2023-11-27T15:25:11.408 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.402+0000 7ffb5266a700 1 -- 172.21.15.8:0/2911852906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c10bb20 con 0x7ffb4c1002f0 2023-11-27T15:25:11.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.403+0000 7ffb5266a700 1 -- 172.21.15.8:0/2911852906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c19e4e0 con 0x7ffb4c1002f0 2023-11-27T15:25:11.409 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.405+0000 7ffb497fa700 1 -- 172.21.15.8:0/2911852906 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ffb3c021580 con 0x7ffb4c1002f0 2023-11-27T15:25:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.406+0000 7ffb497fa700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ffb340a11d0 0x7ffb340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.407+0000 7ffb497fa700 1 -- 172.21.15.8:0/2911852906 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffb3c0cdff0 con 0x7ffb4c1002f0 2023-11-27T15:25:11.411 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.407+0000 7ffb4b7fe700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ffb340a11d0 0x7ffb340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:11.412 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.407+0000 7ffb4b7fe700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ffb340a11d0 0x7ffb340a3680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009c60 tx=0x7ffb40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:11.415 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.411+0000 7ffb497fa700 1 -- 172.21.15.8:0/2911852906 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ffb3c0974a0 con 0x7ffb4c1002f0 2023-11-27T15:25:11.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:11 smithi060 ceph-mon[131518]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-27T15:25:11.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:11 smithi008 ceph-mon[128788]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-27T15:25:11.629 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.624+0000 7ffb5266a700 1 -- 172.21.15.8:0/2911852906 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb4c060f30 con 0x7ffb340a11d0 2023-11-27T15:25:11.642 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.638+0000 7ffb497fa700 1 -- 172.21.15.8:0/2911852906 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7ffb4c060f30 con 0x7ffb340a11d0 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:mds.a smithi008 running (114s) 27s ago 113s 29.9M - 18.0.0-3906-g1c956514 ace6904b066b d8a917533494 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:mds.b smithi029 running (111s) 39s ago 111s 18.9M - 18.0.0-3906-g1c956514 ace6904b066b dcc0411c8ed8 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:mds.c smithi060 running (2m) 39s ago 2m 19.7M - 18.0.0-3906-g1c956514 ace6904b066b 229b83c376dc 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:mds.d smithi008 running (2m) 27s ago 2m 13.8M - 18.0.0-3906-g1c956514 ace6904b066b db1de43e3615 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:mds.e smithi029 running (2m) 39s ago 2m 14.9M - 18.0.0-3906-g1c956514 ace6904b066b 519815cda3b4 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:mds.f smithi060 running (2m) 39s ago 2m 18.6M - 18.0.0-3906-g1c956514 ace6904b066b 5cc9e2fdeb89 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:mds.g smithi008 running (117s) 27s ago 116s 13.8M - 18.0.0-3906-g1c956514 ace6904b066b 24375935d184 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:mds.h smithi029 running (2m) 39s ago 2m 20.2M - 18.0.0-3906-g1c956514 ace6904b066b edcff02d9e51 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:mds.i smithi060 running (2m) 39s ago 2m 20.0M - 18.0.0-3906-g1c956514 ace6904b066b 7a380fe8bc5b 2023-11-27T15:25:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:mds.j smithi008 running (2m) 27s ago 2m 13.8M - 18.0.0-3906-g1c956514 ace6904b066b 6e2cca282403 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:mds.k smithi029 running (2m) 39s ago 2m 16.6M - 18.0.0-3906-g1c956514 ace6904b066b 5207d321fe1c 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:mds.l smithi060 running (2m) 39s ago 2m 16.6M - 18.0.0-3906-g1c956514 ace6904b066b b49cc785c539 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.x smithi008 *:9283,8765 running (9m) 27s ago 9m 521M - 18.0.0-3906-g1c956514 ace6904b066b 33e28ad2124e 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.y smithi029 *:8443,8765 running (7m) 39s ago 7m 446M - 18.0.0-3906-g1c956514 ace6904b066b ef6a443154b6 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.z smithi060 *:8443,8765 running (7m) 39s ago 7m 447M - 18.0.0-3906-g1c956514 ace6904b066b 62605840e05a 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:mon.a smithi008 running (9m) 27s ago 9m 95.8M 2048M 18.0.0-3906-g1c956514 ace6904b066b a152f348fbab 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:mon.b smithi029 running (7m) 39s ago 7m 68.4M 2048M 18.0.0-3906-g1c956514 ace6904b066b 7f0ac6f85952 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:mon.c smithi060 running (7m) 39s ago 7m 65.6M 2048M 18.0.0-3906-g1c956514 ace6904b066b 581813c48943 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (7m) 27s ago 7m 73.8M 4319M 18.0.0-3906-g1c956514 ace6904b066b b9b5a783942e 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi029 running (6m) 39s ago 6m 80.4M 4293M 18.0.0-3906-g1c956514 ace6904b066b 9fd855af58ae 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi060 running (6m) 39s ago 6m 75.1M 4293M 18.0.0-3906-g1c956514 ace6904b066b 49385c7135c1 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi008 running (5m) 27s ago 5m 65.9M 4319M 18.0.0-3906-g1c956514 ace6904b066b 33961074c174 2023-11-27T15:25:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi029 running (5m) 39s ago 5m 62.9M 4293M 18.0.0-3906-g1c956514 ace6904b066b 04f211f66738 2023-11-27T15:25:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi060 running (5m) 39s ago 5m 71.5M 4293M 18.0.0-3906-g1c956514 ace6904b066b 6bc65239178e 2023-11-27T15:25:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:osd.6 smithi008 running (4m) 27s ago 4m 61.3M 4319M 18.0.0-3906-g1c956514 ace6904b066b 439ed790e946 2023-11-27T15:25:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:osd.7 smithi029 running (4m) 39s ago 4m 65.8M 4293M 18.0.0-3906-g1c956514 ace6904b066b d6ea72b2ad35 2023-11-27T15:25:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:osd.8 smithi060 running (3m) 39s ago 3m 61.8M 4293M 18.0.0-3906-g1c956514 ace6904b066b 5982d6e6e427 2023-11-27T15:25:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:osd.9 smithi008 running (3m) 27s ago 3m 59.0M 4319M 18.0.0-3906-g1c956514 ace6904b066b 9ed6522ca3c0 2023-11-27T15:25:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:osd.10 smithi029 running (3m) 39s ago 3m 62.1M 4293M 18.0.0-3906-g1c956514 ace6904b066b fdb5a8ead0f1 2023-11-27T15:25:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:osd.11 smithi060 running (2m) 39s ago 2m 54.6M 4293M 18.0.0-3906-g1c956514 ace6904b066b ae2e21c8379d 2023-11-27T15:25:11.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 -- 172.21.15.8:0/2911852906 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ffb340a11d0 msgr2=0x7ffb340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:11.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ffb340a11d0 0x7ffb340a3680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009c60 tx=0x7ffb40009260 comp rx=0 tx=0).stop 2023-11-27T15:25:11.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 -- 172.21.15.8:0/2911852906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffb4c1002f0 msgr2=0x7ffb4c196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:11.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c196090 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c004740 tx=0x7ffb3c009580 comp rx=0 tx=0).stop 2023-11-27T15:25:11.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 -- 172.21.15.8:0/2911852906 shutdown_connections 2023-11-27T15:25:11.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ffb4c101e80 0x7ffb4c19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:11.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffb4c1002f0 0x7ffb4c196090 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:11.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ffb340a11d0 0x7ffb340a3680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:11.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 --2- 172.21.15.8:0/2911852906 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ffb4c1014f0 0x7ffb4c1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:11.646 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 -- 172.21.15.8:0/2911852906 >> 172.21.15.8:0/2911852906 conn(0x7ffb4c0fb900 msgr2=0x7ffb4c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:11.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 -- 172.21.15.8:0/2911852906 shutdown_connections 2023-11-27T15:25:11.647 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:11.640+0000 7ffb5266a700 1 -- 172.21.15.8:0/2911852906 wait complete. 2023-11-27T15:25:11.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:11 smithi029 ceph-mon[131166]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-27T15:25:12.140 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph orch ls' 2023-11-27T15:25:13.008 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:25:13.564 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:13 smithi008 ceph-mon[128788]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:13.565 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:13 smithi008 ceph-mon[128788]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:25:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:13 smithi060 ceph-mon[131518]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:13 smithi060 ceph-mon[131518]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:25:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:13 smithi029 ceph-mon[131166]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:13 smithi029 ceph-mon[131166]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:25:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.570+0000 7faca911a700 1 -- 172.21.15.8:0/1881414070 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faca41014f0 msgr2=0x7faca4101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.570+0000 7faca911a700 1 --2- 172.21.15.8:0/1881414070 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faca41014f0 0x7faca4101940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fac98009990 tx=0x7fac98009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.570+0000 7faca911a700 1 -- 172.21.15.8:0/1881414070 shutdown_connections 2023-11-27T15:25:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.570+0000 7faca911a700 1 --2- 172.21.15.8:0/1881414070 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca4101e80 0x7faca4108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.570+0000 7faca911a700 1 --2- 172.21.15.8:0/1881414070 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca41002f0 0x7faca4100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.570+0000 7faca911a700 1 --2- 172.21.15.8:0/1881414070 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faca41014f0 0x7faca4101940 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.570+0000 7faca911a700 1 -- 172.21.15.8:0/1881414070 >> 172.21.15.8:0/1881414070 conn(0x7faca40fb900 msgr2=0x7faca40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.570+0000 7faca911a700 1 -- 172.21.15.8:0/1881414070 shutdown_connections 2023-11-27T15:25:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.570+0000 7faca911a700 1 -- 172.21.15.8:0/1881414070 wait complete. 2023-11-27T15:25:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.571+0000 7faca911a700 1 Processor -- start 2023-11-27T15:25:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.571+0000 7faca911a700 1 -- start start 2023-11-27T15:25:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca911a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faca41002f0 0x7faca41960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca911a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca41014f0 0x7faca4196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca911a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca4101e80 0x7faca419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca911a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca419dc50 con 0x7faca41002f0 2023-11-27T15:25:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca911a700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca419dd90 con 0x7faca41014f0 2023-11-27T15:25:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca911a700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca419df20 con 0x7faca4101e80 2023-11-27T15:25:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faca41002f0 0x7faca41960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:14.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faca41002f0 0x7faca41960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46810/0 (socket says 172.21.15.8:46810) 2023-11-27T15:25:14.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca2d9d700 1 -- 172.21.15.8:0/500278340 learned_addr learned my addr 172.21.15.8:0/500278340 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:14.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca259c700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca41014f0 0x7faca4196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:14.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca359e700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca4101e80 0x7faca419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:14.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca2d9d700 1 -- 172.21.15.8:0/500278340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca4101e80 msgr2=0x7faca419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:14.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca2d9d700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca4101e80 0x7faca419d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca2d9d700 1 -- 172.21.15.8:0/500278340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca41014f0 msgr2=0x7faca4196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:14.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca2d9d700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca41014f0 0x7faca4196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.572+0000 7faca2d9d700 1 -- 172.21.15.8:0/500278340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980095f0 con 0x7faca41002f0 2023-11-27T15:25:14.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.573+0000 7faca2d9d700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faca41002f0 0x7faca41960c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fac9400e9a0 tx=0x7fac9400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:14.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.573+0000 7fac8bfff700 1 -- 172.21.15.8:0/500278340 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94009790 con 0x7faca41002f0 2023-11-27T15:25:14.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.573+0000 7fac8bfff700 1 -- 172.21.15.8:0/500278340 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac9400c810 con 0x7faca41002f0 2023-11-27T15:25:14.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.573+0000 7fac8bfff700 1 -- 172.21.15.8:0/500278340 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94003c70 con 0x7faca41002f0 2023-11-27T15:25:14.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.573+0000 7faca911a700 1 -- 172.21.15.8:0/500278340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca410bc90 con 0x7faca41002f0 2023-11-27T15:25:14.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.573+0000 7faca911a700 1 -- 172.21.15.8:0/500278340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca410bb00 con 0x7faca41002f0 2023-11-27T15:25:14.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.574+0000 7faca911a700 1 -- 172.21.15.8:0/500278340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca41078e0 con 0x7faca41002f0 2023-11-27T15:25:14.580 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.576+0000 7fac8bfff700 1 -- 172.21.15.8:0/500278340 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fac9400cc80 con 0x7faca41002f0 2023-11-27T15:25:14.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.578+0000 7fac8bfff700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac8c0a1150 0x7fac8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:14.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.578+0000 7fac8bfff700 1 -- 172.21.15.8:0/500278340 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fac94014070 con 0x7faca41002f0 2023-11-27T15:25:14.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.578+0000 7faca259c700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac8c0a1150 0x7fac8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:14.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.579+0000 7faca259c700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac8c0a1150 0x7fac8c0a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac9800b2e0 tx=0x7fac98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:14.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.582+0000 7fac8bfff700 1 -- 172.21.15.8:0/500278340 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fac940961e0 con 0x7faca41002f0 2023-11-27T15:25:14.800 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.796+0000 7faca911a700 1 -- 172.21.15.8:0/500278340 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7faca4061c90 con 0x7fac8c0a1150 2023-11-27T15:25:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.801+0000 7fac8bfff700 1 -- 172.21.15.8:0/500278340 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7faca4061c90 con 0x7fac8c0a1150 2023-11-27T15:25:14.806 INFO:teuthology.orchestra.run.smithi008.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-11-27T15:25:14.806 INFO:teuthology.orchestra.run.smithi008.stdout:mds.all 12/12 42s ago 2m smithi008=a;smithi008=d;smithi008=g;smithi008=j;smithi029=b;smithi029=e;smithi029=h;smithi029=k;smithi060=c;smithi060=f;smithi060=i;smithi060=l;count:12 2023-11-27T15:25:14.806 INFO:teuthology.orchestra.run.smithi008.stdout:mgr 3/3 42s ago 7m smithi008=x;smithi029=y;smithi060=z;count:3 2023-11-27T15:25:14.806 INFO:teuthology.orchestra.run.smithi008.stdout:mon 3/3 42s ago 7m smithi008:172.21.15.8=a;smithi029:172.21.15.29=b;smithi060:172.21.15.60=c;count:3 2023-11-27T15:25:14.807 INFO:teuthology.orchestra.run.smithi008.stdout:osd 12 42s ago - 2023-11-27T15:25:14.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 -- 172.21.15.8:0/500278340 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac8c0a1150 msgr2=0x7fac8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:14.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac8c0a1150 0x7fac8c0a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac9800b2e0 tx=0x7fac98005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 -- 172.21.15.8:0/500278340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faca41002f0 msgr2=0x7faca41960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:14.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faca41002f0 0x7faca41960c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fac9400e9a0 tx=0x7fac9400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 -- 172.21.15.8:0/500278340 shutdown_connections 2023-11-27T15:25:14.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fac8c0a1150 0x7fac8c0a3600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7faca4101e80 0x7faca419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faca41002f0 0x7faca41960c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 --2- 172.21.15.8:0/500278340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7faca41014f0 0x7faca4196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:14.809 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 -- 172.21.15.8:0/500278340 >> 172.21.15.8:0/500278340 conn(0x7faca40fb900 msgr2=0x7faca4104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:14.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 -- 172.21.15.8:0/500278340 shutdown_connections 2023-11-27T15:25:14.810 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:14.804+0000 7faca911a700 1 -- 172.21.15.8:0/500278340 wait complete. 2023-11-27T15:25:15.308 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph orch host ls' 2023-11-27T15:25:15.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:15 smithi008 ceph-mon[128788]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:25:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:15 smithi060 ceph-mon[131518]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:25:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:15 smithi029 ceph-mon[131166]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:25:16.174 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:25:16.755 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:16 smithi008 ceph-mon[128788]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:16.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:16 smithi060 ceph-mon[131518]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:16.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:16 smithi029 ceph-mon[131166]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:17.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.753+0000 7f80e08c8700 1 -- 172.21.15.8:0/1234459587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80dc1002f0 msgr2=0x7f80dc100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:17.758 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.753+0000 7f80e08c8700 1 --2- 172.21.15.8:0/1234459587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80dc1002f0 0x7f80dc100700 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009990 tx=0x7f80cc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:17.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.753+0000 7f80e08c8700 1 -- 172.21.15.8:0/1234459587 shutdown_connections 2023-11-27T15:25:17.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.753+0000 7f80e08c8700 1 --2- 172.21.15.8:0/1234459587 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80dc101e80 0x7f80dc108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:17.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.753+0000 7f80e08c8700 1 --2- 172.21.15.8:0/1234459587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80dc1002f0 0x7f80dc100700 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:17.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.753+0000 7f80e08c8700 1 --2- 172.21.15.8:0/1234459587 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80dc1014f0 0x7f80dc101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:17.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.753+0000 7f80e08c8700 1 -- 172.21.15.8:0/1234459587 >> 172.21.15.8:0/1234459587 conn(0x7f80dc0fb900 msgr2=0x7f80dc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:17.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.753+0000 7f80e08c8700 1 -- 172.21.15.8:0/1234459587 shutdown_connections 2023-11-27T15:25:17.759 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.753+0000 7f80e08c8700 1 -- 172.21.15.8:0/1234459587 wait complete. 2023-11-27T15:25:17.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.754+0000 7f80e08c8700 1 Processor -- start 2023-11-27T15:25:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80e08c8700 1 -- start start 2023-11-27T15:25:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80e08c8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80dc1002f0 0x7f80dc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80e08c8700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80dc1014f0 0x7f80dc1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80e08c8700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80dc101e80 0x7f80dc19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80e08c8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc19fe20 con 0x7f80dc1002f0 2023-11-27T15:25:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80e08c8700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc19ff60 con 0x7f80dc101e80 2023-11-27T15:25:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80e08c8700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc1a00f0 con 0x7f80dc1014f0 2023-11-27T15:25:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80da59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80dc1002f0 0x7f80dc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80da59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80dc1002f0 0x7f80dc198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46828/0 (socket says 172.21.15.8:46828) 2023-11-27T15:25:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80da59c700 1 -- 172.21.15.8:0/320003984 learned_addr learned my addr 172.21.15.8:0/320003984 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80dad9d700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80dc101e80 0x7f80dc19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.755+0000 7f80d9d9b700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80dc1014f0 0x7f80dc1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80da59c700 1 -- 172.21.15.8:0/320003984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80dc1014f0 msgr2=0x7f80dc1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80da59c700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80dc1014f0 0x7f80dc1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80da59c700 1 -- 172.21.15.8:0/320003984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80dc101e80 msgr2=0x7f80dc19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80da59c700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80dc101e80 0x7f80dc19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80da59c700 1 -- 172.21.15.8:0/320003984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc0095f0 con 0x7f80dc1002f0 2023-11-27T15:25:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80da59c700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80dc1002f0 0x7f80dc198290 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f80cc005750 tx=0x7f80cc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80cb7fe700 1 -- 172.21.15.8:0/320003984 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc01d070 con 0x7f80dc1002f0 2023-11-27T15:25:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80cb7fe700 1 -- 172.21.15.8:0/320003984 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80cc021410 con 0x7f80dc1002f0 2023-11-27T15:25:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80e08c8700 1 -- 172.21.15.8:0/320003984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80dc10bcb0 con 0x7f80dc1002f0 2023-11-27T15:25:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80cb7fe700 1 -- 172.21.15.8:0/320003984 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc017650 con 0x7f80dc1002f0 2023-11-27T15:25:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.756+0000 7f80e08c8700 1 -- 172.21.15.8:0/320003984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80dc10bb20 con 0x7f80dc1002f0 2023-11-27T15:25:17.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.757+0000 7f80e08c8700 1 -- 172.21.15.8:0/320003984 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80dc066ea0 con 0x7f80dc1002f0 2023-11-27T15:25:17.765 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.760+0000 7f80cb7fe700 1 -- 172.21.15.8:0/320003984 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f80cc021580 con 0x7f80dc1002f0 2023-11-27T15:25:17.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.763+0000 7f80cb7fe700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80c40a1150 0x7f80c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:17.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.763+0000 7f80cb7fe700 1 -- 172.21.15.8:0/320003984 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f80cc0ce6c0 con 0x7f80dc1002f0 2023-11-27T15:25:17.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.763+0000 7f80d9d9b700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80c40a1150 0x7f80c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:17.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.763+0000 7f80d9d9b700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80c40a1150 0x7f80c40a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009c60 tx=0x7f80d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.764+0000 7f80cb7fe700 1 -- 172.21.15.8:0/320003984 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f80cc097b30 con 0x7f80dc1002f0 2023-11-27T15:25:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:17 smithi060 ceph-mon[131518]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:25:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:17 smithi008 ceph-mon[128788]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:25:17.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:17 smithi029 ceph-mon[131166]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:25:17.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.978+0000 7f80e08c8700 1 -- 172.21.15.8:0/320003984 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f80dc1a0b60 con 0x7f80c40a1150 2023-11-27T15:25:17.984 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.979+0000 7f80cb7fe700 1 -- 172.21.15.8:0/320003984 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7f80dc1a0b60 con 0x7f80c40a1150 2023-11-27T15:25:17.984 INFO:teuthology.orchestra.run.smithi008.stdout:HOST ADDR LABELS STATUS 2023-11-27T15:25:17.984 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008 172.21.15.8 2023-11-27T15:25:17.984 INFO:teuthology.orchestra.run.smithi008.stdout:smithi029 172.21.15.29 2023-11-27T15:25:17.985 INFO:teuthology.orchestra.run.smithi008.stdout:smithi060 172.21.15.60 2023-11-27T15:25:17.985 INFO:teuthology.orchestra.run.smithi008.stdout:3 hosts in cluster 2023-11-27T15:25:17.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.981+0000 7f80e08c8700 1 -- 172.21.15.8:0/320003984 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80c40a1150 msgr2=0x7f80c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:17.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.981+0000 7f80e08c8700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80c40a1150 0x7f80c40a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009c60 tx=0x7f80d0009260 comp rx=0 tx=0).stop 2023-11-27T15:25:17.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.981+0000 7f80e08c8700 1 -- 172.21.15.8:0/320003984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80dc1002f0 msgr2=0x7f80dc198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:17.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.981+0000 7f80e08c8700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80dc1002f0 0x7f80dc198290 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f80cc005750 tx=0x7f80cc009580 comp rx=0 tx=0).stop 2023-11-27T15:25:17.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.981+0000 7f80e08c8700 1 -- 172.21.15.8:0/320003984 shutdown_connections 2023-11-27T15:25:17.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.981+0000 7f80e08c8700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f80dc101e80 0x7f80dc19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:17.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.981+0000 7f80e08c8700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f80dc1002f0 0x7f80dc198290 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:17.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.981+0000 7f80e08c8700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f80c40a1150 0x7f80c40a3600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:17.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.981+0000 7f80e08c8700 1 --2- 172.21.15.8:0/320003984 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f80dc1014f0 0x7f80dc1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:17.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.981+0000 7f80e08c8700 1 -- 172.21.15.8:0/320003984 >> 172.21.15.8:0/320003984 conn(0x7f80dc0fb900 msgr2=0x7f80dc0fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:17.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.982+0000 7f80e08c8700 1 -- 172.21.15.8:0/320003984 shutdown_connections 2023-11-27T15:25:17.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:17.982+0000 7f80e08c8700 1 -- 172.21.15.8:0/320003984 wait complete. 2023-11-27T15:25:18.573 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph orch device ls' 2023-11-27T15:25:19.434 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:25:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:19 smithi060 ceph-mon[131518]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:19 smithi060 ceph-mon[131518]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:25:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:19 smithi008 ceph-mon[128788]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:19 smithi008 ceph-mon[128788]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:25:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:19 smithi029 ceph-mon[131166]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:19 smithi029 ceph-mon[131166]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:25:20.981 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.977+0000 7f8e42143700 1 -- 172.21.15.8:0/2638415416 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e3c1014f0 msgr2=0x7f8e3c101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:20.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.977+0000 7f8e42143700 1 --2- 172.21.15.8:0/2638415416 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e3c1014f0 0x7f8e3c101940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009990 tx=0x7f8e30009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:20.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.977+0000 7f8e42143700 1 -- 172.21.15.8:0/2638415416 shutdown_connections 2023-11-27T15:25:20.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.977+0000 7f8e42143700 1 --2- 172.21.15.8:0/2638415416 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e3c1002f0 0x7f8e3c100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:20.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.977+0000 7f8e42143700 1 --2- 172.21.15.8:0/2638415416 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e3c101e80 0x7f8e3c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:20.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.977+0000 7f8e42143700 1 --2- 172.21.15.8:0/2638415416 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e3c1014f0 0x7f8e3c101940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:20.982 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.977+0000 7f8e42143700 1 -- 172.21.15.8:0/2638415416 >> 172.21.15.8:0/2638415416 conn(0x7f8e3c0fb900 msgr2=0x7f8e3c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:20.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.977+0000 7f8e42143700 1 -- 172.21.15.8:0/2638415416 shutdown_connections 2023-11-27T15:25:20.983 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.977+0000 7f8e42143700 1 -- 172.21.15.8:0/2638415416 wait complete. 2023-11-27T15:25:20.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.978+0000 7f8e42143700 1 Processor -- start 2023-11-27T15:25:20.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e42143700 1 -- start start 2023-11-27T15:25:20.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e42143700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e3c1002f0 0x7f8e3c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:20.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e42143700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e3c1014f0 0x7f8e3c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:20.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e42143700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e3c101e80 0x7f8e3c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:20.985 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e42143700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c19fdc0 con 0x7f8e3c1002f0 2023-11-27T15:25:20.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e42143700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c19ff00 con 0x7f8e3c101e80 2023-11-27T15:25:20.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e42143700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1a0090 con 0x7f8e3c1014f0 2023-11-27T15:25:20.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e3b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e3c1002f0 0x7f8e3c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:20.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e3bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e3c101e80 0x7f8e3c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:20.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e3affd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e3c1014f0 0x7f8e3c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:20.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e3b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e3c1002f0 0x7f8e3c198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43464/0 (socket says 172.21.15.8:43464) 2023-11-27T15:25:20.986 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.979+0000 7f8e3b7fe700 1 -- 172.21.15.8:0/337032081 learned_addr learned my addr 172.21.15.8:0/337032081 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:20.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e3b7fe700 1 -- 172.21.15.8:0/337032081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e3c1014f0 msgr2=0x7f8e3c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:20.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e3b7fe700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e3c1014f0 0x7f8e3c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:20.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e3b7fe700 1 -- 172.21.15.8:0/337032081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e3c101e80 msgr2=0x7f8e3c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:20.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e3b7fe700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e3c101e80 0x7f8e3c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:20.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e3b7fe700 1 -- 172.21.15.8:0/337032081 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e300095f0 con 0x7f8e3c1002f0 2023-11-27T15:25:20.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e3b7fe700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e3c1002f0 0x7f8e3c198230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d9b0 tx=0x7f8e2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:20.987 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e38ff9700 1 -- 172.21.15.8:0/337032081 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c004020 con 0x7f8e3c1002f0 2023-11-27T15:25:20.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e38ff9700 1 -- 172.21.15.8:0/337032081 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e2c009730 con 0x7f8e3c1002f0 2023-11-27T15:25:20.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e42143700 1 -- 172.21.15.8:0/337032081 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c10bc50 con 0x7f8e3c1002f0 2023-11-27T15:25:20.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e38ff9700 1 -- 172.21.15.8:0/337032081 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0039e0 con 0x7f8e3c1002f0 2023-11-27T15:25:20.988 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.980+0000 7f8e42143700 1 -- 172.21.15.8:0/337032081 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c10bac0 con 0x7f8e3c1002f0 2023-11-27T15:25:20.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.981+0000 7f8e42143700 1 -- 172.21.15.8:0/337032081 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c1929b0 con 0x7f8e3c1002f0 2023-11-27T15:25:20.989 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.983+0000 7f8e38ff9700 1 -- 172.21.15.8:0/337032081 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8e2c0098a0 con 0x7f8e3c1002f0 2023-11-27T15:25:20.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.986+0000 7f8e38ff9700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8e240a11d0 0x7f8e240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:20.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.986+0000 7f8e38ff9700 1 -- 172.21.15.8:0/337032081 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8e2c0cc850 con 0x7f8e3c1002f0 2023-11-27T15:25:20.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.986+0000 7f8e3affd700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8e240a11d0 0x7f8e240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:20.990 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.987+0000 7f8e3affd700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8e240a11d0 0x7f8e240a3680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000b5c0 tx=0x7f8e30005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:20.994 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:20.990+0000 7f8e38ff9700 1 -- 172.21.15.8:0/337032081 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8e2c095c30 con 0x7f8e3c1002f0 2023-11-27T15:25:21.211 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.207+0000 7f8e42143700 1 -- 172.21.15.8:0/337032081 --> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e3c061cb0 con 0x7f8e240a11d0 2023-11-27T15:25:21.212 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.209+0000 7f8e38ff9700 1 -- 172.21.15.8:0/337032081 <== mgr.14152 v2:172.21.15.8:6800/1756612424 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f8e3c061cb0 con 0x7f8e240a11d0 2023-11-27T15:25:21.213 INFO:teuthology.orchestra.run.smithi008.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-11-27T15:25:21.213 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476004A480DGN 447G No 3m ago LVM detected, locked 2023-11-27T15:25:21.213 INFO:teuthology.orchestra.run.smithi008.stdout:smithi029 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009B480DGN 447G No 3m ago LVM detected, locked 2023-11-27T15:25:21.213 INFO:teuthology.orchestra.run.smithi008.stdout:smithi060 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476008A480DGN 447G No 2m ago LVM detected, locked 2023-11-27T15:25:21.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 -- 172.21.15.8:0/337032081 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8e240a11d0 msgr2=0x7f8e240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:21.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8e240a11d0 0x7f8e240a3680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000b5c0 tx=0x7f8e30005960 comp rx=0 tx=0).stop 2023-11-27T15:25:21.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 -- 172.21.15.8:0/337032081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e3c1002f0 msgr2=0x7f8e3c198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:21.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e3c1002f0 0x7f8e3c198230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d9b0 tx=0x7f8e2c00dcc0 comp rx=0 tx=0).stop 2023-11-27T15:25:21.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 -- 172.21.15.8:0/337032081 shutdown_connections 2023-11-27T15:25:21.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8e3c101e80 0x7f8e3c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:21.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e3c1002f0 0x7f8e3c198230 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:21.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8e240a11d0 0x7f8e240a3680 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:21.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 --2- 172.21.15.8:0/337032081 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8e3c1014f0 0x7f8e3c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:21.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 -- 172.21.15.8:0/337032081 >> 172.21.15.8:0/337032081 conn(0x7f8e3c0fb900 msgr2=0x7f8e3c106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:21.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.211+0000 7f8e42143700 1 -- 172.21.15.8:0/337032081 shutdown_connections 2023-11-27T15:25:21.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:21.212+0000 7f8e42143700 1 -- 172.21.15.8:0/337032081 wait complete. 2023-11-27T15:25:21.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:21 smithi008 ceph-mon[128788]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:25:21.734 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-27T15:25:21.744 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi008.front.sepia.ceph.com 2023-11-27T15:25:21.745 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph fs dump' 2023-11-27T15:25:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:21 smithi060 ceph-mon[131518]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:25:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:21 smithi029 ceph-mon[131166]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:25:22.606 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:25:22.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:22 smithi060 ceph-mon[131518]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:22.859 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:22 smithi008 ceph-mon[128788]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:22.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:22 smithi029 ceph-mon[131166]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:25:23.773 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:23 smithi008 ceph-mon[128788]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:25:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:23 smithi060 ceph-mon[131518]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:25:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:23 smithi029 ceph-mon[131166]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:25:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.010+0000 7f0abf646700 1 -- 172.21.15.8:0/3843979850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ab8073550 msgr2=0x7f0ab80739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.010+0000 7f0abf646700 1 --2- 172.21.15.8:0/3843979850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ab8073550 0x7f0ab80739c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009990 tx=0x7f0aa8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.010+0000 7f0abf646700 1 -- 172.21.15.8:0/3843979850 shutdown_connections 2023-11-27T15:25:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.010+0000 7f0abf646700 1 --2- 172.21.15.8:0/3843979850 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab81081c0 0x7f0ab810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.010+0000 7f0abf646700 1 --2- 172.21.15.8:0/3843979850 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8074b20 0x7f0ab8072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.010+0000 7f0abf646700 1 --2- 172.21.15.8:0/3843979850 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ab8073550 0x7f0ab80739c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.010+0000 7f0abf646700 1 -- 172.21.15.8:0/3843979850 >> 172.21.15.8:0/3843979850 conn(0x7f0ab80fb900 msgr2=0x7f0ab80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.010+0000 7f0abf646700 1 -- 172.21.15.8:0/3843979850 shutdown_connections 2023-11-27T15:25:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.010+0000 7f0abf646700 1 -- 172.21.15.8:0/3843979850 wait complete. 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.011+0000 7f0abf646700 1 Processor -- start 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.011+0000 7f0abf646700 1 -- start start 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abf646700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ab8073550 0x7f0ab8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abf646700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8074b20 0x7f0ab8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abf646700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab81081c0 0x7f0ab819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abf646700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab819fd90 con 0x7f0ab8073550 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abf646700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab819fed0 con 0x7f0ab8074b20 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abf646700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81a0060 con 0x7f0ab81081c0 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abd3e2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ab8073550 0x7f0ab8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abd3e2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ab8073550 0x7f0ab8198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43478/0 (socket says 172.21.15.8:43478) 2023-11-27T15:25:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abd3e2700 1 -- 172.21.15.8:0/3728859823 learned_addr learned my addr 172.21.15.8:0/3728859823 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:24.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abcbe1700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8074b20 0x7f0ab8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:24.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abdbe3700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab81081c0 0x7f0ab819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:24.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abd3e2700 1 -- 172.21.15.8:0/3728859823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab81081c0 msgr2=0x7f0ab819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:24.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abd3e2700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab81081c0 0x7f0ab819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abd3e2700 1 -- 172.21.15.8:0/3728859823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8074b20 msgr2=0x7f0ab8198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:24.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abd3e2700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8074b20 0x7f0ab8198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.012+0000 7f0abd3e2700 1 -- 172.21.15.8:0/3728859823 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80095f0 con 0x7f0ab8073550 2023-11-27T15:25:24.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.013+0000 7f0abd3e2700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ab8073550 0x7f0ab8198200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400d980 tx=0x7f0ab400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:24.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.013+0000 7f0aae7fc700 1 -- 172.21.15.8:0/3728859823 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40098d0 con 0x7f0ab8073550 2023-11-27T15:25:24.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.013+0000 7f0abf646700 1 -- 172.21.15.8:0/3728859823 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab810de40 con 0x7f0ab8073550 2023-11-27T15:25:24.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.013+0000 7f0aae7fc700 1 -- 172.21.15.8:0/3728859823 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ab4010460 con 0x7f0ab8073550 2023-11-27T15:25:24.018 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.013+0000 7f0aae7fc700 1 -- 172.21.15.8:0/3728859823 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400f620 con 0x7f0ab8073550 2023-11-27T15:25:24.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.013+0000 7f0abf646700 1 -- 172.21.15.8:0/3728859823 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab810dcb0 con 0x7f0ab8073550 2023-11-27T15:25:24.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.014+0000 7f0abf646700 1 -- 172.21.15.8:0/3728859823 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab8106870 con 0x7f0ab8073550 2023-11-27T15:25:24.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.016+0000 7f0aae7fc700 1 -- 172.21.15.8:0/3728859823 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0ab4010860 con 0x7f0ab8073550 2023-11-27T15:25:24.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.017+0000 7f0aae7fc700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0aa40a1150 0x7f0aa40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:24.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.018+0000 7f0aae7fc700 1 -- 172.21.15.8:0/3728859823 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0ab40cca10 con 0x7f0ab8073550 2023-11-27T15:25:24.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.018+0000 7f0abcbe1700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0aa40a1150 0x7f0aa40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:24.022 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.018+0000 7f0abcbe1700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0aa40a1150 0x7f0aa40a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800b5c0 tx=0x7f0aa8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:24.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.022+0000 7f0aae7fc700 1 -- 172.21.15.8:0/3728859823 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0ab4095df0 con 0x7f0ab8073550 2023-11-27T15:25:24.279 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.276+0000 7f0abf646700 1 -- 172.21.15.8:0/3728859823 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0ab8061990 con 0x7f0ab8073550 2023-11-27T15:25:24.280 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.276+0000 7f0aae7fc700 1 -- 172.21.15.8:0/3728859823 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2875 (secure 0 0 0) 0x7f0ab4020b50 con 0x7f0ab8073550 2023-11-27T15:25:24.280 INFO:teuthology.orchestra.run.smithi008.stdout:e24 2023-11-27T15:25:24.280 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:25:24.280 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 24 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:created 2023-11-27T15:23:27.167939+0000 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2023-11-27T15:23:59.241472+0000 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 300 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2023-11-27T15:25:24.281 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 0 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 3 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1,2 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=24595,1=24593,2=24539} 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2023-11-27T15:25:24.282 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3,4] 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 0 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.a{0:24595} state up:active seq 4 addr [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.i{0:24565} state up:standby-replay seq 1 addr [v2:172.21.15.60:6838/3185908756,v1:172.21.15.60:6839/3185908756] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.b{1:24593} state up:active seq 11 addr [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.c{1:24547} state up:standby-replay seq 1 addr [v2:172.21.15.60:6836/287097286,v1:172.21.15.60:6837/287097286] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.h{2:24539} state up:active seq 14 addr [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.283 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.f{2:24535} state up:standby-replay seq 1 addr [v2:172.21.15.60:6834/612130958,v1:172.21.15.60:6835/612130958] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.284 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:24.284 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:24.284 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2023-11-27T15:25:24.284 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:24.284 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.d{-1:14481} state up:standby seq 1 addr [v2:172.21.15.8:6836/2810841935,v1:172.21.15.8:6837/2810841935] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.284 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.g{-1:14538} state up:standby seq 1 addr [v2:172.21.15.8:6838/2208262973,v1:172.21.15.8:6839/2208262973] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.284 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.k{-1:24470} state up:standby seq 1 addr [v2:172.21.15.29:6832/192235656,v1:172.21.15.29:6833/192235656] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.284 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.l{-1:24478} state up:standby seq 1 addr [v2:172.21.15.60:6832/3958511263,v1:172.21.15.60:6833/3958511263] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.284 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.j{-1:24499} state up:standby seq 1 addr [v2:172.21.15.8:6834/2969037056,v1:172.21.15.8:6835/2969037056] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.284 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.e{-1:24506} state up:standby seq 1 addr [v2:172.21.15.29:6834/789896869,v1:172.21.15.29:6835/789896869] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:25:24.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 -- 172.21.15.8:0/3728859823 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0aa40a1150 msgr2=0x7f0aa40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:24.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0aa40a1150 0x7f0aa40a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800b5c0 tx=0x7f0aa8005960 comp rx=0 tx=0).stop 2023-11-27T15:25:24.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 -- 172.21.15.8:0/3728859823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ab8073550 msgr2=0x7f0ab8198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:24.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ab8073550 0x7f0ab8198200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400d980 tx=0x7f0ab400dc90 comp rx=0 tx=0).stop 2023-11-27T15:25:24.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 -- 172.21.15.8:0/3728859823 shutdown_connections 2023-11-27T15:25:24.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0aa40a1150 0x7f0aa40a3600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0ab81081c0 0x7f0ab819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ab8073550 0x7f0ab8198200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 --2- 172.21.15.8:0/3728859823 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0ab8074b20 0x7f0ab8198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 -- 172.21.15.8:0/3728859823 >> 172.21.15.8:0/3728859823 conn(0x7f0ab80fb900 msgr2=0x7f0ab80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 -- 172.21.15.8:0/3728859823 shutdown_connections 2023-11-27T15:25:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:24.279+0000 7f0abf646700 1 -- 172.21.15.8:0/3728859823 wait complete. 2023-11-27T15:25:24.286 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2023-11-27T15:25:24.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:24 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3728859823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-27T15:25:24.807 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph osd dump' 2023-11-27T15:25:24.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:24 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3728859823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-27T15:25:24.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:24 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3728859823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-27T15:25:25.673 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:25:25.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:25 smithi060 ceph-mon[131518]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-27T15:25:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:25 smithi008 ceph-mon[128788]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-27T15:25:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:25 smithi029 ceph-mon[131166]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-27T15:25:27.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.306+0000 7f7c5d43c700 1 -- 172.21.15.8:0/3130784359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c581014f0 msgr2=0x7f7c58101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:27.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.306+0000 7f7c5d43c700 1 --2- 172.21.15.8:0/3130784359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c581014f0 0x7f7c58101940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0098f0 tx=0x7f7c4c009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:27.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.306+0000 7f7c5d43c700 1 -- 172.21.15.8:0/3130784359 shutdown_connections 2023-11-27T15:25:27.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.306+0000 7f7c5d43c700 1 --2- 172.21.15.8:0/3130784359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c581002f0 0x7f7c58100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:27.311 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.306+0000 7f7c5d43c700 1 --2- 172.21.15.8:0/3130784359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c58101e80 0x7f7c58108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:27.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.306+0000 7f7c5d43c700 1 --2- 172.21.15.8:0/3130784359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c581014f0 0x7f7c58101940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:27.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.306+0000 7f7c5d43c700 1 -- 172.21.15.8:0/3130784359 >> 172.21.15.8:0/3130784359 conn(0x7f7c580fb900 msgr2=0x7f7c580fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:27.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.306+0000 7f7c5d43c700 1 -- 172.21.15.8:0/3130784359 shutdown_connections 2023-11-27T15:25:27.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.307+0000 7f7c5d43c700 1 -- 172.21.15.8:0/3130784359 wait complete. 2023-11-27T15:25:27.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c5d43c700 1 Processor -- start 2023-11-27T15:25:27.312 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c5d43c700 1 -- start start 2023-11-27T15:25:27.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c5d43c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c581002f0 0x7f7c581982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:27.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c5d43c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c581014f0 0x7f7c581987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:27.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c5d43c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c58101e80 0x7f7c5819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:27.313 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c5d43c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5819fe30 con 0x7f7c581014f0 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c5d43c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5819ff70 con 0x7f7c581002f0 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c5d43c700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c581a0100 con 0x7f7c58101e80 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c567fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c581014f0 0x7f7c581987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c56ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c581002f0 0x7f7c581982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c567fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c581014f0 0x7f7c581987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43496/0 (socket says 172.21.15.8:43496) 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.308+0000 7f7c567fc700 1 -- 172.21.15.8:0/82784668 learned_addr learned my addr 172.21.15.8:0/82784668 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c577fe700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c58101e80 0x7f7c5819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c56ffd700 1 -- 172.21.15.8:0/82784668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c58101e80 msgr2=0x7f7c5819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c56ffd700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c58101e80 0x7f7c5819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c56ffd700 1 -- 172.21.15.8:0/82784668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c581014f0 msgr2=0x7f7c581987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c56ffd700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c581014f0 0x7f7c581987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c56ffd700 1 -- 172.21.15.8:0/82784668 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50009d00 con 0x7f7c581002f0 2023-11-27T15:25:27.314 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c567fc700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c581014f0 0x7f7c581987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:27.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c56ffd700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c581002f0 0x7f7c581982a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800e9c0 tx=0x7f7c4800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:27.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c3ffff700 1 -- 172.21.15.8:0/82784668 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48004220 con 0x7f7c581002f0 2023-11-27T15:25:27.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c5d43c700 1 -- 172.21.15.8:0/82784668 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4c009520 con 0x7f7c581002f0 2023-11-27T15:25:27.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c3ffff700 1 -- 172.21.15.8:0/82784668 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c48004380 con 0x7f7c581002f0 2023-11-27T15:25:27.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.309+0000 7f7c3ffff700 1 -- 172.21.15.8:0/82784668 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48003c50 con 0x7f7c581002f0 2023-11-27T15:25:27.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.310+0000 7f7c5d43c700 1 -- 172.21.15.8:0/82784668 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5810bc50 con 0x7f7c581002f0 2023-11-27T15:25:27.315 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.311+0000 7f7c5d43c700 1 -- 172.21.15.8:0/82784668 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5810bde0 con 0x7f7c581002f0 2023-11-27T15:25:27.316 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.313+0000 7f7c3ffff700 1 -- 172.21.15.8:0/82784668 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7c4800c7c0 con 0x7f7c581002f0 2023-11-27T15:25:27.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.314+0000 7f7c3ffff700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7c400a1150 0x7f7c400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:27.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.314+0000 7f7c3ffff700 1 -- 172.21.15.8:0/82784668 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7c48014070 con 0x7f7c581002f0 2023-11-27T15:25:27.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.314+0000 7f7c567fc700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7c400a1150 0x7f7c400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:27.318 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.315+0000 7f7c567fc700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7c400a1150 0x7f7c400a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00c010 tx=0x7f7c4c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:27.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.318+0000 7f7c3ffff700 1 -- 172.21.15.8:0/82784668 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7c48096220 con 0x7f7c581002f0 2023-11-27T15:25:27.533 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.529+0000 7f7c5d43c700 1 -- 172.21.15.8:0/82784668 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f7c58066ea0 con 0x7f7c581002f0 2023-11-27T15:25:27.534 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.531+0000 7f7c3ffff700 1 -- 172.21.15.8:0/82784668 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) v1 ==== 56+0+5546 (secure 0 0 0) 0x7f7c48099840 con 0x7f7c581002f0 2023-11-27T15:25:27.534 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 90 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:created 2023-11-27T15:15:42.569283+0000 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2023-11-27T15:24:40.249579+0000 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:crush_version 27 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:full_ratio 0.95 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:backfillfull_ratio 0.9 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:nearfull_ratio 0.85 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:require_min_compat_client luminous 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:min_compat_client luminous 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:require_osd_release reef 2023-11-27T15:25:27.535 INFO:teuthology.orchestra.run.smithi008.stdout:stretch_mode_enabled false 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:max_osd 12 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.8:6802/225604885,v1:172.21.15.8:6803/225604885] [v2:172.21.15.8:6804/225604885,v1:172.21.15.8:6805/225604885] exists,up f832011f-25d1-4f98-adf6-6c2225afa50c 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6800/3314882267,v1:172.21.15.29:6801/3314882267] [v2:172.21.15.29:6802/3314882267,v1:172.21.15.29:6803/3314882267] exists,up 6b3d6a3d-e153-4cc3-9551-81eb07f635c4 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6800/3602448664,v1:172.21.15.60:6801/3602448664] [v2:172.21.15.60:6802/3602448664,v1:172.21.15.60:6803/3602448664] exists,up c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.8:6810/709452183,v1:172.21.15.8:6811/709452183] [v2:172.21.15.8:6812/709452183,v1:172.21.15.8:6813/709452183] exists,up 7b26fe4c-ca46-4542-9906-25396da80c99 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 up in weight 1 up_from 34 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6808/3472444298,v1:172.21.15.29:6809/3472444298] [v2:172.21.15.29:6810/3472444298,v1:172.21.15.29:6811/3472444298] exists,up 77ced007-63e0-4416-a2b1-bebc48d33c03 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 up in weight 1 up_from 39 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6808/1250742602,v1:172.21.15.60:6809/1250742602] [v2:172.21.15.60:6810/1250742602,v1:172.21.15.60:6811/1250742602] exists,up e4c2fbe2-63ef-4d43-a04f-38bef2ac6839 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:osd.6 up in weight 1 up_from 47 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.8:6818/1285430534,v1:172.21.15.8:6819/1285430534] [v2:172.21.15.8:6820/1285430534,v1:172.21.15.8:6821/1285430534] exists,up 309035f0-69df-47d3-b854-c1b480c200c0 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:osd.7 up in weight 1 up_from 52 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6816/2172314430,v1:172.21.15.29:6817/2172314430] [v2:172.21.15.29:6818/2172314430,v1:172.21.15.29:6819/2172314430] exists,up 2fcef10a-546a-4928-9341-7d31b79942be 2023-11-27T15:25:27.536 INFO:teuthology.orchestra.run.smithi008.stdout:osd.8 up in weight 1 up_from 59 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6816/2261251565,v1:172.21.15.60:6817/2261251565] [v2:172.21.15.60:6818/2261251565,v1:172.21.15.60:6819/2261251565] exists,up addef24e-8aa7-4f3d-be94-9decd5f22f87 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:osd.9 up in weight 1 up_from 67 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.8:6826/2087690159,v1:172.21.15.8:6827/2087690159] [v2:172.21.15.8:6828/2087690159,v1:172.21.15.8:6829/2087690159] exists,up 6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:osd.10 up in weight 1 up_from 72 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.29:6824/3342205417,v1:172.21.15.29:6825/3342205417] [v2:172.21.15.29:6826/3342205417,v1:172.21.15.29:6827/3342205417] exists,up 5dc133e8-62fc-4f5d-bde0-8454b3292bca 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:osd.11 up in weight 1 up_from 79 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6824/1307710245,v1:172.21.15.60:6825/1307710245] [v2:172.21.15.60:6826/1307710245,v1:172.21.15.60:6827/1307710245] exists,up ac26363a-c419-4b6d-8dc7-5e337cc9b565 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 2.c [2,1] 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 2.e [2,4] 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 2.10 [8,11] 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 3.11 [5,9] 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 3.24 [4,9] 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 3.34 [4,9] 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 4.f [9,8] 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 4.14 [9,8] 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 4.1c [5,8] 2023-11-27T15:25:27.537 INFO:teuthology.orchestra.run.smithi008.stdout:pg_upmap_items 4.1f [9,10] 2023-11-27T15:25:27.538 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:6801/2324276308 expires 2023-11-28T15:16:38.242903+0000 2023-11-27T15:25:27.538 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/2225314492 expires 2023-11-28T15:16:38.242903+0000 2023-11-27T15:25:27.538 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/3792241060 expires 2023-11-28T15:16:38.242903+0000 2023-11-27T15:25:27.538 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/3188200818 expires 2023-11-28T15:16:11.844441+0000 2023-11-27T15:25:27.538 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/128817483 expires 2023-11-28T15:16:11.844441+0000 2023-11-27T15:25:27.538 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/2304079767 expires 2023-11-28T15:16:11.844441+0000 2023-11-27T15:25:27.538 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:6800/560372725 expires 2023-11-28T15:16:11.844441+0000 2023-11-27T15:25:27.538 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:6800/2324276308 expires 2023-11-28T15:16:38.242903+0000 2023-11-27T15:25:27.538 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:6801/560372725 expires 2023-11-28T15:16:11.844441+0000 2023-11-27T15:25:27.538 INFO:teuthology.orchestra.run.smithi008.stdout:blocklist 172.21.15.8:0/3849980484 expires 2023-11-28T15:16:38.242903+0000 2023-11-27T15:25:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 -- 172.21.15.8:0/82784668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7c400a1150 msgr2=0x7f7c400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7c400a1150 0x7f7c400a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00c010 tx=0x7f7c4c019040 comp rx=0 tx=0).stop 2023-11-27T15:25:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 -- 172.21.15.8:0/82784668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c581002f0 msgr2=0x7f7c581982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c581002f0 0x7f7c581982a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800e9c0 tx=0x7f7c4800ecd0 comp rx=0 tx=0).stop 2023-11-27T15:25:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 -- 172.21.15.8:0/82784668 shutdown_connections 2023-11-27T15:25:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7c400a1150 0x7f7c400a3600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7c58101e80 0x7f7c5819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7c581002f0 0x7f7c581982a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 --2- 172.21.15.8:0/82784668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c581014f0 0x7f7c581987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:27.539 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 -- 172.21.15.8:0/82784668 >> 172.21.15.8:0/82784668 conn(0x7f7c580fb900 msgr2=0x7f7c580fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 -- 172.21.15.8:0/82784668 shutdown_connections 2023-11-27T15:25:27.540 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:27.533+0000 7f7c5d43c700 1 -- 172.21.15.8:0/82784668 wait complete. 2023-11-27T15:25:27.679 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:27 smithi008 ceph-mon[128788]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2023-11-27T15:25:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:27 smithi060 ceph-mon[131518]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2023-11-27T15:25:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:27 smithi029 ceph-mon[131166]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 7 op/s 2023-11-27T15:25:27.974 INFO:teuthology.run_tasks:Running task fs.ready... 2023-11-27T15:25:27.986 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:25:28.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.192+0000 7f45527ff700 1 -- 172.21.15.8:0/1244182567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f454c1002d0 msgr2=0x7f454c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:28.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.192+0000 7f45527ff700 1 --2- 172.21.15.8:0/1244182567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f454c1002d0 0x7f454c1006e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f453c009990 tx=0x7f453c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.192+0000 7f45527ff700 1 -- 172.21.15.8:0/1244182567 shutdown_connections 2023-11-27T15:25:28.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.192+0000 7f45527ff700 1 --2- 172.21.15.8:0/1244182567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454c101e50 0x7f454c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.192+0000 7f45527ff700 1 --2- 172.21.15.8:0/1244182567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f454c1002d0 0x7f454c1006e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.196 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.192+0000 7f45527ff700 1 --2- 172.21.15.8:0/1244182567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f454c1014c0 0x7f454c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.192+0000 7f45527ff700 1 -- 172.21.15.8:0/1244182567 >> 172.21.15.8:0/1244182567 conn(0x7f454c0fb8b0 msgr2=0x7f454c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:28.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.192+0000 7f45527ff700 1 -- 172.21.15.8:0/1244182567 shutdown_connections 2023-11-27T15:25:28.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.192+0000 7f45527ff700 1 -- 172.21.15.8:0/1244182567 wait complete. 2023-11-27T15:25:28.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.193+0000 7f45527ff700 1 Processor -- start 2023-11-27T15:25:28.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.193+0000 7f45527ff700 1 -- start start 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f45527ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f454c1002d0 0x7f454c1960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f45527ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f454c1014c0 0x7f454c1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f45527ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454c101e50 0x7f454c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f45527ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c19dc20 con 0x7f454c1002d0 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f45527ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c19dd60 con 0x7f454c101e50 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f45527ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c19def0 con 0x7f454c1014c0 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f454bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f454c1002d0 0x7f454c1960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f454bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f454c1002d0 0x7f454c1960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43510/0 (socket says 172.21.15.8:43510) 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f454bfff700 1 -- 172.21.15.8:0/3975905548 learned_addr learned my addr 172.21.15.8:0/3975905548 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f4550d9c700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454c101e50 0x7f454c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:28.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f4543fff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f454c1014c0 0x7f454c1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:28.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f454bfff700 1 -- 172.21.15.8:0/3975905548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f454c1014c0 msgr2=0x7f454c1965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:28.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f454bfff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f454c1014c0 0x7f454c1965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f454bfff700 1 -- 172.21.15.8:0/3975905548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454c101e50 msgr2=0x7f454c19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:28.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f454bfff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454c101e50 0x7f454c19d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f454bfff700 1 -- 172.21.15.8:0/3975905548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c0095f0 con 0x7f454c1002d0 2023-11-27T15:25:28.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.194+0000 7f4550d9c700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454c101e50 0x7f454c19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:28.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.195+0000 7f454bfff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f454c1002d0 0x7f454c1960a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f453c004730 tx=0x7f453c004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:28.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.195+0000 7f4549ffb700 1 -- 172.21.15.8:0/3975905548 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c01d070 con 0x7f454c1002d0 2023-11-27T15:25:28.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.195+0000 7f4549ffb700 1 -- 172.21.15.8:0/3975905548 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f453c021410 con 0x7f454c1002d0 2023-11-27T15:25:28.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.195+0000 7f45527ff700 1 -- 172.21.15.8:0/3975905548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454c10bca0 con 0x7f454c1002d0 2023-11-27T15:25:28.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.195+0000 7f4549ffb700 1 -- 172.21.15.8:0/3975905548 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c017650 con 0x7f454c1002d0 2023-11-27T15:25:28.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.195+0000 7f45527ff700 1 -- 172.21.15.8:0/3975905548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c10bb10 con 0x7f454c1002d0 2023-11-27T15:25:28.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.196+0000 7f45527ff700 1 -- 172.21.15.8:0/3975905548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c107860 con 0x7f454c1002d0 2023-11-27T15:25:28.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.198+0000 7f4549ffb700 1 -- 172.21.15.8:0/3975905548 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f453c021580 con 0x7f454c1002d0 2023-11-27T15:25:28.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.200+0000 7f4549ffb700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f45380a11d0 0x7f45380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:28.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.200+0000 7f4549ffb700 1 -- 172.21.15.8:0/3975905548 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f453c0cdfe0 con 0x7f454c1002d0 2023-11-27T15:25:28.204 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.200+0000 7f4543fff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f45380a11d0 0x7f45380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:28.205 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.201+0000 7f4543fff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f45380a11d0 0x7f45380a3680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4534009c60 tx=0x7f4534009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:28.208 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.204+0000 7f4549ffb700 1 -- 172.21.15.8:0/3975905548 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f453c097450 con 0x7f454c1002d0 2023-11-27T15:25:28.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.419+0000 7f45527ff700 1 -- 172.21.15.8:0/3975905548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f454c02cfe0 con 0x7f454c1002d0 2023-11-27T15:25:28.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.420+0000 7f4549ffb700 1 -- 172.21.15.8:0/3975905548 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24344 (secure 0 0 0) 0x7f453c09aa70 con 0x7f454c1002d0 2023-11-27T15:25:28.425 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:28.425 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":90,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:24:40.249579+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:25:28.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 -- 172.21.15.8:0/3975905548 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f45380a11d0 msgr2=0x7f45380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:28.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f45380a11d0 0x7f45380a3680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4534009c60 tx=0x7f4534009260 comp rx=0 tx=0).stop 2023-11-27T15:25:28.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 -- 172.21.15.8:0/3975905548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f454c1002d0 msgr2=0x7f454c1960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:28.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f454c1002d0 0x7f454c1960a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f453c004730 tx=0x7f453c004760 comp rx=0 tx=0).stop 2023-11-27T15:25:28.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 -- 172.21.15.8:0/3975905548 shutdown_connections 2023-11-27T15:25:28.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f454c101e50 0x7f454c19d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f454c1002d0 0x7f454c1960a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f45380a11d0 0x7f45380a3680 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 --2- 172.21.15.8:0/3975905548 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f454c1014c0 0x7f454c1965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 -- 172.21.15.8:0/3975905548 >> 172.21.15.8:0/3975905548 conn(0x7f454c0fb8b0 msgr2=0x7f454c1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:28.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 -- 172.21.15.8:0/3975905548 shutdown_connections 2023-11-27T15:25:28.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.423+0000 7f45527ff700 1 -- 172.21.15.8:0/3975905548 wait complete. 2023-11-27T15:25:28.447 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:23:25.166148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:23:26.584539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:23:30.130965+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:25:28.448 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:25:28.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.651+0000 7f90037ff700 1 -- 172.21.15.8:0/1184506535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc101480 msgr2=0x7f8ffc1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:28.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.651+0000 7f90037ff700 1 --2- 172.21.15.8:0/1184506535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc101480 0x7f8ffc1018d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0098f0 tx=0x7f8fec009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:28.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.651+0000 7f90037ff700 1 -- 172.21.15.8:0/1184506535 shutdown_connections 2023-11-27T15:25:28.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.651+0000 7f90037ff700 1 --2- 172.21.15.8:0/1184506535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ffc101e10 0x7f8ffc1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.655 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.651+0000 7f90037ff700 1 --2- 172.21.15.8:0/1184506535 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ffc100290 0x7f8ffc1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.651+0000 7f90037ff700 1 --2- 172.21.15.8:0/1184506535 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc101480 0x7f8ffc1018d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.651+0000 7f90037ff700 1 -- 172.21.15.8:0/1184506535 >> 172.21.15.8:0/1184506535 conn(0x7f8ffc0fb870 msgr2=0x7f8ffc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:28.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.652+0000 7f90037ff700 1 -- 172.21.15.8:0/1184506535 shutdown_connections 2023-11-27T15:25:28.656 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.652+0000 7f90037ff700 1 -- 172.21.15.8:0/1184506535 wait complete. 2023-11-27T15:25:28.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f90037ff700 1 Processor -- start 2023-11-27T15:25:28.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f90037ff700 1 -- start start 2023-11-27T15:25:28.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f90037ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc100290 0x7f8ffc0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:28.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f90037ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ffc101480 0x7f8ffc072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:28.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f90037ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ffc101e10 0x7f8ffc073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:28.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f90037ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc073bc0 con 0x7f8ffc101480 2023-11-27T15:25:28.657 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f90037ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc073d30 con 0x7f8ffc101e10 2023-11-27T15:25:28.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f90037ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1a00d0 con 0x7f8ffc100290 2023-11-27T15:25:28.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f9000d9a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ffc101480 0x7f8ffc072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:28.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f9001d9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ffc101e10 0x7f8ffc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:28.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f9000d9a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ffc101480 0x7f8ffc072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43532/0 (socket says 172.21.15.8:43532) 2023-11-27T15:25:28.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f9001d9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ffc101e10 0x7f8ffc073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:34600/0 (socket says 172.21.15.8:34600) 2023-11-27T15:25:28.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f900159b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc100290 0x7f8ffc0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:28.658 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.653+0000 7f9000d9a700 1 -- 172.21.15.8:0/425047608 learned_addr learned my addr 172.21.15.8:0/425047608 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:28.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.654+0000 7f9000d9a700 1 -- 172.21.15.8:0/425047608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc100290 msgr2=0x7f8ffc0748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:28.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.654+0000 7f9000d9a700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc100290 0x7f8ffc0748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.654+0000 7f9000d9a700 1 -- 172.21.15.8:0/425047608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ffc101e10 msgr2=0x7f8ffc073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:28.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.654+0000 7f9000d9a700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ffc101e10 0x7f8ffc073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.654+0000 7f9000d9a700 1 -- 172.21.15.8:0/425047608 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4009d00 con 0x7f8ffc101480 2023-11-27T15:25:28.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.654+0000 7f9000d9a700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ffc101480 0x7f8ffc072ee0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009fd0 tx=0x7f8fec00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:28.659 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.654+0000 7f8ff27fc700 1 -- 172.21.15.8:0/425047608 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec01c070 con 0x7f8ffc101480 2023-11-27T15:25:28.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.655+0000 7f90037ff700 1 -- 172.21.15.8:0/425047608 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec009520 con 0x7f8ffc101480 2023-11-27T15:25:28.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.655+0000 7f8ff27fc700 1 -- 172.21.15.8:0/425047608 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fec00f7d0 con 0x7f8ffc101480 2023-11-27T15:25:28.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.655+0000 7f8ff27fc700 1 -- 172.21.15.8:0/425047608 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec004ad0 con 0x7f8ffc101480 2023-11-27T15:25:28.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.655+0000 7f90037ff700 1 -- 172.21.15.8:0/425047608 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc10bbf0 con 0x7f8ffc101480 2023-11-27T15:25:28.660 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.656+0000 7f90037ff700 1 -- 172.21.15.8:0/425047608 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ffc107980 con 0x7f8ffc101480 2023-11-27T15:25:28.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.657+0000 7f8ff27fc700 1 -- 172.21.15.8:0/425047608 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8fec020070 con 0x7f8ffc101480 2023-11-27T15:25:28.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.659+0000 7f8ff27fc700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8fe80a11d0 0x7f8fe80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:28.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.659+0000 7f8ff27fc700 1 -- 172.21.15.8:0/425047608 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8fec0cd840 con 0x7f8ffc101480 2023-11-27T15:25:28.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.659+0000 7f900159b700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8fe80a11d0 0x7f8fe80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:28.663 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.660+0000 7f900159b700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8fe80a11d0 0x7f8fe80a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc1012e0 tx=0x7f8ff8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:28.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.663+0000 7f8ff27fc700 1 -- 172.21.15.8:0/425047608 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8fec05c730 con 0x7f8ffc101480 2023-11-27T15:25:28.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:28 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/82784668' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-27T15:25:28.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:28 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3975905548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:28.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:28 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/82784668' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-27T15:25:28.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:28 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3975905548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:28.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.872+0000 7f90037ff700 1 -- 172.21.15.8:0/425047608 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ffc067a50 con 0x7f8ffc101480 2023-11-27T15:25:28.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.873+0000 7f8ff27fc700 1 -- 172.21.15.8:0/425047608 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8fec027020 con 0x7f8ffc101480 2023-11-27T15:25:28.877 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:25:28.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 -- 172.21.15.8:0/425047608 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8fe80a11d0 msgr2=0x7f8fe80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:28.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8fe80a11d0 0x7f8fe80a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc1012e0 tx=0x7f8ff8009260 comp rx=0 tx=0).stop 2023-11-27T15:25:28.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 -- 172.21.15.8:0/425047608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ffc101480 msgr2=0x7f8ffc072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:28.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ffc101480 0x7f8ffc072ee0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009fd0 tx=0x7f8fec00b790 comp rx=0 tx=0).stop 2023-11-27T15:25:28.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 -- 172.21.15.8:0/425047608 shutdown_connections 2023-11-27T15:25:28.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f8fe80a11d0 0x7f8fe80a3680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f8ffc100290 0x7f8ffc0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f8ffc101e10 0x7f8ffc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 --2- 172.21.15.8:0/425047608 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ffc101480 0x7f8ffc072ee0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:28.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 -- 172.21.15.8:0/425047608 >> 172.21.15.8:0/425047608 conn(0x7f8ffc0fb870 msgr2=0x7f8ffc104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:28.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 -- 172.21.15.8:0/425047608 shutdown_connections 2023-11-27T15:25:28.880 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:28.875+0000 7f90037ff700 1 -- 172.21.15.8:0/425047608 wait complete. 2023-11-27T15:25:28.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:28 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/82784668' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-27T15:25:28.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:28 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3975905548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:28.898 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:25:29.105 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.102+0000 7f7906dff700 1 -- 172.21.15.8:0/273589875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7900105fd0 msgr2=0x7f79001083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.102+0000 7f7906dff700 1 --2- 172.21.15.8:0/273589875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7900105fd0 0x7f79001083b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f78f800b0a0 tx=0x7f78f800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.102+0000 7f7906dff700 1 -- 172.21.15.8:0/273589875 shutdown_connections 2023-11-27T15:25:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.102+0000 7f7906dff700 1 --2- 172.21.15.8:0/273589875 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7900100d90 0x7f7900103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.102+0000 7f7906dff700 1 --2- 172.21.15.8:0/273589875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7900105fd0 0x7f79001083b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.102+0000 7f7906dff700 1 --2- 172.21.15.8:0/273589875 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f79001036b0 0x7f7900105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.102+0000 7f7906dff700 1 -- 172.21.15.8:0/273589875 >> 172.21.15.8:0/273589875 conn(0x7f79000fa750 msgr2=0x7f79000fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.102+0000 7f7906dff700 1 -- 172.21.15.8:0/273589875 shutdown_connections 2023-11-27T15:25:29.106 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.102+0000 7f7906dff700 1 -- 172.21.15.8:0/273589875 wait complete. 2023-11-27T15:25:29.108 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.103+0000 7f7906dff700 1 Processor -- start 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.103+0000 7f7906dff700 1 -- start start 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.103+0000 7f7906dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7900100d90 0x7f7900100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.103+0000 7f7906dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79001036b0 0x7f79000fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.103+0000 7f7906dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7900105fd0 0x7f79000ff280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.103+0000 7f7906dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79000ffa20 con 0x7f79001036b0 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.103+0000 7f7906dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79000ffb90 con 0x7f7900105fd0 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.103+0000 7f7906dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79001a0160 con 0x7f7900100d90 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f78fffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79001036b0 0x7f79000fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f78fffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79001036b0 0x7f79000fed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43546/0 (socket says 172.21.15.8:43546) 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f78fffff700 1 -- 172.21.15.8:0/1773402051 learned_addr learned my addr 172.21.15.8:0/1773402051 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f7904b9b700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7900100d90 0x7f7900100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f790539c700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7900105fd0 0x7f79000ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:29.109 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f78fffff700 1 -- 172.21.15.8:0/1773402051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7900100d90 msgr2=0x7f7900100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f78fffff700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7900100d90 0x7f7900100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f78fffff700 1 -- 172.21.15.8:0/1773402051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7900105fd0 msgr2=0x7f79000ff280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f78fffff700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7900105fd0 0x7f79000ff280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f78fffff700 1 -- 172.21.15.8:0/1773402051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f8009d00 con 0x7f79001036b0 2023-11-27T15:25:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f78fffff700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79001036b0 0x7f79000fed40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00e9a0 tx=0x7f78ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.104+0000 7f78fdffb700 1 -- 172.21.15.8:0/1773402051 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec009790 con 0x7f79001036b0 2023-11-27T15:25:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.105+0000 7f78fdffb700 1 -- 172.21.15.8:0/1773402051 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78ec010430 con 0x7f79001036b0 2023-11-27T15:25:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.105+0000 7f7906dff700 1 -- 172.21.15.8:0/1773402051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790010bc60 con 0x7f79001036b0 2023-11-27T15:25:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.105+0000 7f78fdffb700 1 -- 172.21.15.8:0/1773402051 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec018680 con 0x7f79001036b0 2023-11-27T15:25:29.110 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.105+0000 7f7906dff700 1 -- 172.21.15.8:0/1773402051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790010bad0 con 0x7f79001036b0 2023-11-27T15:25:29.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.106+0000 7f7906dff700 1 -- 172.21.15.8:0/1773402051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79000fc2c0 con 0x7f79001036b0 2023-11-27T15:25:29.111 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.108+0000 7f78fdffb700 1 -- 172.21.15.8:0/1773402051 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f78ec0105a0 con 0x7f79001036b0 2023-11-27T15:25:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.109+0000 7f78fdffb700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78f00a11d0 0x7f78f00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.109+0000 7f78fdffb700 1 -- 172.21.15.8:0/1773402051 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f78ec014070 con 0x7f79001036b0 2023-11-27T15:25:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.109+0000 7f7904b9b700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78f00a11d0 0x7f78f00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.110+0000 7f7904b9b700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78f00a11d0 0x7f78f00a3680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7900068880 tx=0x7f78f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.113+0000 7f78fdffb700 1 -- 172.21.15.8:0/1773402051 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f78ec051730 con 0x7f79001036b0 2023-11-27T15:25:29.328 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.325+0000 7f7906dff700 1 -- 172.21.15.8:0/1773402051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7900061c40 con 0x7f79001036b0 2023-11-27T15:25:29.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.325+0000 7f78fdffb700 1 -- 172.21.15.8:0/1773402051 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78ec0961d0 con 0x7f79001036b0 2023-11-27T15:25:29.329 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 -- 172.21.15.8:0/1773402051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78f00a11d0 msgr2=0x7f78f00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78f00a11d0 0x7f78f00a3680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7900068880 tx=0x7f78f4009260 comp rx=0 tx=0).stop 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 -- 172.21.15.8:0/1773402051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79001036b0 msgr2=0x7f79000fed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79001036b0 0x7f79000fed40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00e9a0 tx=0x7f78ec00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 -- 172.21.15.8:0/1773402051 shutdown_connections 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f78f00a11d0 0x7f78f00a3680 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7900100d90 0x7f7900100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7900105fd0 0x7f79000ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 --2- 172.21.15.8:0/1773402051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f79001036b0 0x7f79000fed40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 -- 172.21.15.8:0/1773402051 >> 172.21.15.8:0/1773402051 conn(0x7f79000fa750 msgr2=0x7f7900101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 -- 172.21.15.8:0/1773402051 shutdown_connections 2023-11-27T15:25:29.332 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.328+0000 7f7906dff700 1 -- 172.21.15.8:0/1773402051 wait complete. 2023-11-27T15:25:29.351 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:25:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.556+0000 7fae0a9ff700 1 -- 172.21.15.8:0/3911309228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae04101480 msgr2=0x7fae041018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.556+0000 7fae0a9ff700 1 --2- 172.21.15.8:0/3911309228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae04101480 0x7fae041018d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009990 tx=0x7fadf8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.556+0000 7fae0a9ff700 1 -- 172.21.15.8:0/3911309228 shutdown_connections 2023-11-27T15:25:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.556+0000 7fae0a9ff700 1 --2- 172.21.15.8:0/3911309228 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae04101e10 0x7fae041086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.556+0000 7fae0a9ff700 1 --2- 172.21.15.8:0/3911309228 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae04100290 0x7fae041006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.556+0000 7fae0a9ff700 1 --2- 172.21.15.8:0/3911309228 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae04101480 0x7fae041018d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.556+0000 7fae0a9ff700 1 -- 172.21.15.8:0/3911309228 >> 172.21.15.8:0/3911309228 conn(0x7fae040fb870 msgr2=0x7fae040fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:29.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.556+0000 7fae0a9ff700 1 -- 172.21.15.8:0/3911309228 shutdown_connections 2023-11-27T15:25:29.561 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.556+0000 7fae0a9ff700 1 -- 172.21.15.8:0/3911309228 wait complete. 2023-11-27T15:25:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.557+0000 7fae0a9ff700 1 Processor -- start 2023-11-27T15:25:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.557+0000 7fae0a9ff700 1 -- start start 2023-11-27T15:25:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.557+0000 7fae0a9ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae04100290 0x7fae04198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.557+0000 7fae0a9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae04101480 0x7fae041986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.557+0000 7fae0a9ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae04101e10 0x7fae0419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae0a9ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419fce0 con 0x7fae04100290 2023-11-27T15:25:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae0a9ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419fe20 con 0x7fae04101e10 2023-11-27T15:25:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae0a9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0419ffb0 con 0x7fae04101480 2023-11-27T15:25:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae03fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae04100290 0x7fae04198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:29.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae03fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae04100290 0x7fae04198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43564/0 (socket says 172.21.15.8:43564) 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae03fff700 1 -- 172.21.15.8:0/344737090 learned_addr learned my addr 172.21.15.8:0/344737090 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae037fe700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae04101480 0x7fae041986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae08f9c700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae04101e10 0x7fae0419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae03fff700 1 -- 172.21.15.8:0/344737090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae04101480 msgr2=0x7fae041986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae03fff700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae04101480 0x7fae041986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae03fff700 1 -- 172.21.15.8:0/344737090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae04101e10 msgr2=0x7fae0419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae03fff700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae04101e10 0x7fae0419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae03fff700 1 -- 172.21.15.8:0/344737090 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80095f0 con 0x7fae04100290 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.558+0000 7fae03fff700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae04100290 0x7fae04198160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fadf400e970 tx=0x7fadf400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.559+0000 7fae017fa700 1 -- 172.21.15.8:0/344737090 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4009790 con 0x7fae04100290 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.559+0000 7fae017fa700 1 -- 172.21.15.8:0/344737090 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf40044e0 con 0x7fae04100290 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.559+0000 7fae017fa700 1 -- 172.21.15.8:0/344737090 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4003e30 con 0x7fae04100290 2023-11-27T15:25:29.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.559+0000 7fae0a9ff700 1 -- 172.21.15.8:0/344737090 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0410bbf0 con 0x7fae04100290 2023-11-27T15:25:29.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.559+0000 7fae0a9ff700 1 -- 172.21.15.8:0/344737090 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0410ba60 con 0x7fae04100290 2023-11-27T15:25:29.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.560+0000 7fae0a9ff700 1 -- 172.21.15.8:0/344737090 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae04192880 con 0x7fae04100290 2023-11-27T15:25:29.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.562+0000 7fae017fa700 1 -- 172.21.15.8:0/344737090 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fadf4003f90 con 0x7fae04100290 2023-11-27T15:25:29.567 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.564+0000 7fae017fa700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fadec0a1150 0x7fadec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:29.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.564+0000 7fae017fa700 1 -- 172.21.15.8:0/344737090 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fadf4014070 con 0x7fae04100290 2023-11-27T15:25:29.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.564+0000 7fae037fe700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fadec0a1150 0x7fadec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:29.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.565+0000 7fae037fe700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fadec0a1150 0x7fadec0a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fadf800b2e0 tx=0x7fadf8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:29.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.568+0000 7fae017fa700 1 -- 172.21.15.8:0/344737090 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fadf40d0030 con 0x7fae04100290 2023-11-27T15:25:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:29 smithi008 ceph-mon[128788]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-27T15:25:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:29 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/425047608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:25:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:29 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1773402051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.779+0000 7fae0a9ff700 1 -- 172.21.15.8:0/344737090 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae04061700 con 0x7fae04100290 2023-11-27T15:25:29.783 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.780+0000 7fae017fa700 1 -- 172.21.15.8:0/344737090 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fadf4096350 con 0x7fae04100290 2023-11-27T15:25:29.783 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-11-27T15:25:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 -- 172.21.15.8:0/344737090 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fadec0a1150 msgr2=0x7fadec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fadec0a1150 0x7fadec0a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fadf800b2e0 tx=0x7fadf8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 -- 172.21.15.8:0/344737090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae04100290 msgr2=0x7fae04198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae04100290 0x7fae04198160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fadf400e970 tx=0x7fadf400ed30 comp rx=0 tx=0).stop 2023-11-27T15:25:29.786 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 -- 172.21.15.8:0/344737090 shutdown_connections 2023-11-27T15:25:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fae04101e10 0x7fae0419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae04100290 0x7fae04198160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fadec0a1150 0x7fadec0a3600 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 --2- 172.21.15.8:0/344737090 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fae04101480 0x7fae041986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 -- 172.21.15.8:0/344737090 >> 172.21.15.8:0/344737090 conn(0x7fae040fb870 msgr2=0x7fae04106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 -- 172.21.15.8:0/344737090 shutdown_connections 2023-11-27T15:25:29.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:29.782+0000 7fae0a9ff700 1 -- 172.21.15.8:0/344737090 wait complete. 2023-11-27T15:25:29.805 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:25:29.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:29 smithi060 ceph-mon[131518]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-27T15:25:29.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:29 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/425047608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:25:29.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:29 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1773402051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:29.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:29 smithi029 ceph-mon[131166]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2023-11-27T15:25:29.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:29 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/425047608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:25:29.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:29 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1773402051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:30.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.008+0000 7f93079ff700 1 -- 172.21.15.8:0/418111740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93001014d0 msgr2=0x7f9300101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.008+0000 7f93079ff700 1 --2- 172.21.15.8:0/418111740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93001014d0 0x7f9300101920 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009990 tx=0x7f92f0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.008+0000 7f93079ff700 1 -- 172.21.15.8:0/418111740 shutdown_connections 2023-11-27T15:25:30.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.008+0000 7f93079ff700 1 --2- 172.21.15.8:0/418111740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9300101e60 0x7f9300108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.008+0000 7f93079ff700 1 --2- 172.21.15.8:0/418111740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93001002e0 0x7f93001006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.008+0000 7f93079ff700 1 --2- 172.21.15.8:0/418111740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93001014d0 0x7f9300101920 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.008+0000 7f93079ff700 1 -- 172.21.15.8:0/418111740 >> 172.21.15.8:0/418111740 conn(0x7f93000fb900 msgr2=0x7f93000fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:30.012 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.008+0000 7f93079ff700 1 -- 172.21.15.8:0/418111740 shutdown_connections 2023-11-27T15:25:30.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.008+0000 7f93079ff700 1 -- 172.21.15.8:0/418111740 wait complete. 2023-11-27T15:25:30.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.009+0000 7f93079ff700 1 Processor -- start 2023-11-27T15:25:30.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.009+0000 7f93079ff700 1 -- start start 2023-11-27T15:25:30.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f93079ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93001002e0 0x7f93001981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f93079ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93001014d0 0x7f9300198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.013 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f93079ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9300101e60 0x7f930019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f93079ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930019fd70 con 0x7f9300101e60 2023-11-27T15:25:30.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f93079ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930019feb0 con 0x7f93001002e0 2023-11-27T15:25:30.014 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f93079ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93001a0040 con 0x7f93001014d0 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f9305f9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9300101e60 0x7f930019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f9304f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93001014d0 0x7f9300198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f9304f9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93001014d0 0x7f9300198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:59286/0 (socket says 172.21.15.8:59286) 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f930579b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93001002e0 0x7f93001981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f9304f9a700 1 -- 172.21.15.8:0/3756829369 learned_addr learned my addr 172.21.15.8:0/3756829369 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f9305f9c700 1 -- 172.21.15.8:0/3756829369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93001014d0 msgr2=0x7f9300198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f9305f9c700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93001014d0 0x7f9300198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f9305f9c700 1 -- 172.21.15.8:0/3756829369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93001002e0 msgr2=0x7f93001981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f9305f9c700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93001002e0 0x7f93001981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f9305f9c700 1 -- 172.21.15.8:0/3756829369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00095f0 con 0x7f9300101e60 2023-11-27T15:25:30.015 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.010+0000 7f9304f9a700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93001014d0 0x7f9300198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:30.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.011+0000 7f9305f9c700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9300101e60 0x7f930019f7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f92f800efc0 tx=0x7f92f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:30.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.011+0000 7f92f67fc700 1 -- 172.21.15.8:0/3756829369 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8018070 con 0x7f9300101e60 2023-11-27T15:25:30.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.011+0000 7f92f67fc700 1 -- 172.21.15.8:0/3756829369 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f800f040 con 0x7f9300101e60 2023-11-27T15:25:30.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.011+0000 7f92f67fc700 1 -- 172.21.15.8:0/3756829369 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f800cd40 con 0x7f9300101e60 2023-11-27T15:25:30.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.011+0000 7f93079ff700 1 -- 172.21.15.8:0/3756829369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930010bc90 con 0x7f9300101e60 2023-11-27T15:25:30.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.011+0000 7f93079ff700 1 -- 172.21.15.8:0/3756829369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930010bb00 con 0x7f9300101e60 2023-11-27T15:25:30.016 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.012+0000 7f93079ff700 1 -- 172.21.15.8:0/3756829369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93001a0680 con 0x7f9300101e60 2023-11-27T15:25:30.017 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.014+0000 7f92f67fc700 1 -- 172.21.15.8:0/3756829369 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f92f8003a00 con 0x7f9300101e60 2023-11-27T15:25:30.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.016+0000 7f92f67fc700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f92ec0a11d0 0x7f92ec0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.017+0000 7f92f67fc700 1 -- 172.21.15.8:0/3756829369 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f92f80cc750 con 0x7f9300101e60 2023-11-27T15:25:30.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.017+0000 7f930579b700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f92ec0a11d0 0x7f92ec0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.017+0000 7f930579b700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f92ec0a11d0 0x7f92ec0a3680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9300101330 tx=0x7f92fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:30.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.020+0000 7f92f67fc700 1 -- 172.21.15.8:0/3756829369 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f92f8095bc0 con 0x7f9300101e60 2023-11-27T15:25:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.229+0000 7f93079ff700 1 -- 172.21.15.8:0/3756829369 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f93001a0c30 con 0x7f9300101e60 2023-11-27T15:25:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.229+0000 7f92f67fc700 1 -- 172.21.15.8:0/3756829369 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f92f80991e0 con 0x7f9300101e60 2023-11-27T15:25:30.233 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:25:30.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 -- 172.21.15.8:0/3756829369 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f92ec0a11d0 msgr2=0x7f92ec0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.235 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f92ec0a11d0 0x7f92ec0a3680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9300101330 tx=0x7f92fc009260 comp rx=0 tx=0).stop 2023-11-27T15:25:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 -- 172.21.15.8:0/3756829369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9300101e60 msgr2=0x7f930019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9300101e60 0x7f930019f7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f92f800efc0 tx=0x7f92f800c3a0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 -- 172.21.15.8:0/3756829369 shutdown_connections 2023-11-27T15:25:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9300101e60 0x7f930019f7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f93001002e0 0x7f93001981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f92ec0a11d0 0x7f92ec0a3680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 --2- 172.21.15.8:0/3756829369 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f93001014d0 0x7f9300198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 -- 172.21.15.8:0/3756829369 >> 172.21.15.8:0/3756829369 conn(0x7f93000fb900 msgr2=0x7f93000fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 -- 172.21.15.8:0/3756829369 shutdown_connections 2023-11-27T15:25:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.232+0000 7f93079ff700 1 -- 172.21.15.8:0/3756829369 wait complete. 2023-11-27T15:25:30.254 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:25:30.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.458+0000 7fe2065ff700 1 -- 172.21.15.8:0/3014017772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe200074a90 msgr2=0x7fe200072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.458+0000 7fe2065ff700 1 --2- 172.21.15.8:0/3014017772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe200074a90 0x7fe200072f80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0009990 tx=0x7fe1f0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.458+0000 7fe2065ff700 1 -- 172.21.15.8:0/3014017772 shutdown_connections 2023-11-27T15:25:30.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.458+0000 7fe2065ff700 1 --2- 172.21.15.8:0/3014017772 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe200103d60 0x7fe20010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.458+0000 7fe2065ff700 1 --2- 172.21.15.8:0/3014017772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe200074a90 0x7fe200072f80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.458+0000 7fe2065ff700 1 --2- 172.21.15.8:0/3014017772 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe2000734c0 0x7fe200073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.458+0000 7fe2065ff700 1 -- 172.21.15.8:0/3014017772 >> 172.21.15.8:0/3014017772 conn(0x7fe2000fb930 msgr2=0x7fe2000fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:30.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.458+0000 7fe2065ff700 1 -- 172.21.15.8:0/3014017772 shutdown_connections 2023-11-27T15:25:30.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.458+0000 7fe2065ff700 1 -- 172.21.15.8:0/3014017772 wait complete. 2023-11-27T15:25:30.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.459+0000 7fe2065ff700 1 Processor -- start 2023-11-27T15:25:30.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.460+0000 7fe2065ff700 1 -- start start 2023-11-27T15:25:30.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.460+0000 7fe2065ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2000734c0 0x7fe20019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.460+0000 7fe2065ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe200074a90 0x7fe20019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.460+0000 7fe2065ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe200103d60 0x7fe2001a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.460+0000 7fe2065ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2001a4230 con 0x7fe200103d60 2023-11-27T15:25:30.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.460+0000 7fe2065ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2001a4370 con 0x7fe2000734c0 2023-11-27T15:25:30.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.460+0000 7fe2065ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2001a4500 con 0x7fe200074a90 2023-11-27T15:25:30.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.460+0000 7fe204b9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe200103d60 0x7fe2001a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.460+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2000734c0 0x7fe20019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1ff7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe200074a90 0x7fe20019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe204b9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe200103d60 0x7fe2001a3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39038/0 (socket says 172.21.15.8:39038) 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1fffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2000734c0 0x7fe20019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:46736/0 (socket says 172.21.15.8:46736) 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe204b9c700 1 -- 172.21.15.8:0/3487365359 learned_addr learned my addr 172.21.15.8:0/3487365359 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1fffff700 1 -- 172.21.15.8:0/3487365359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe200074a90 msgr2=0x7fe20019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1fffff700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe200074a90 0x7fe20019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1fffff700 1 -- 172.21.15.8:0/3487365359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe200103d60 msgr2=0x7fe2001a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1fffff700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe200103d60 0x7fe2001a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1fffff700 1 -- 172.21.15.8:0/3487365359 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f00095f0 con 0x7fe2000734c0 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe204b9c700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe200103d60 0x7fe2001a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1fffff700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2000734c0 0x7fe20019c6b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000b680 tx=0x7fe1f0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1fd7fa700 1 -- 172.21.15.8:0/3487365359 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f001d070 con 0x7fe2000734c0 2023-11-27T15:25:30.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1fd7fa700 1 -- 172.21.15.8:0/3487365359 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1f0021410 con 0x7fe2000734c0 2023-11-27T15:25:30.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe2065ff700 1 -- 172.21.15.8:0/3487365359 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2001122b0 con 0x7fe2000734c0 2023-11-27T15:25:30.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.461+0000 7fe1fd7fa700 1 -- 172.21.15.8:0/3487365359 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0017650 con 0x7fe2000734c0 2023-11-27T15:25:30.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.462+0000 7fe2065ff700 1 -- 172.21.15.8:0/3487365359 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe200112120 con 0x7fe2000734c0 2023-11-27T15:25:30.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.462+0000 7fe2065ff700 1 -- 172.21.15.8:0/3487365359 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2001024b0 con 0x7fe2000734c0 2023-11-27T15:25:30.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.464+0000 7fe1fd7fa700 1 -- 172.21.15.8:0/3487365359 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe1f0021810 con 0x7fe2000734c0 2023-11-27T15:25:30.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.466+0000 7fe1fd7fa700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe1e80a11d0 0x7fe1e80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.466+0000 7fe1fd7fa700 1 -- 172.21.15.8:0/3487365359 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe1f00cdfb0 con 0x7fe2000734c0 2023-11-27T15:25:30.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.466+0000 7fe1ff7fe700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe1e80a11d0 0x7fe1e80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.467+0000 7fe1ff7fe700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe1e80a11d0 0x7fe1e80a3680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009c60 tx=0x7fe1f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:30.474 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.470+0000 7fe1fd7fa700 1 -- 172.21.15.8:0/3487365359 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe1f0097420 con 0x7fe2000734c0 2023-11-27T15:25:30.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:30 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/344737090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:30.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:30 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3756829369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:30.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.731+0000 7fe2065ff700 1 -- 172.21.15.8:0/3487365359 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe20002cfe0 con 0x7fe2000734c0 2023-11-27T15:25:30.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.732+0000 7fe1fd7fa700 1 -- 172.21.15.8:0/3487365359 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10768 (secure 0 0 0) 0x7fe1f009aa40 con 0x7fe2000734c0 2023-11-27T15:25:30.736 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:30.736 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:23:27.167939+0000","modified":"2023-11-27T15:23:59.241472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24595,"mds_1":24593,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24593":{"gid":24593,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24595":{"gid":24595,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:25:30.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 -- 172.21.15.8:0/3487365359 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe1e80a11d0 msgr2=0x7fe1e80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe1e80a11d0 0x7fe1e80a3680 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009c60 tx=0x7fe1f4009260 comp rx=0 tx=0).stop 2023-11-27T15:25:30.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 -- 172.21.15.8:0/3487365359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2000734c0 msgr2=0x7fe20019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2000734c0 0x7fe20019c6b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000b680 tx=0x7fe1f0009580 comp rx=0 tx=0).stop 2023-11-27T15:25:30.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 -- 172.21.15.8:0/3487365359 shutdown_connections 2023-11-27T15:25:30.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe200103d60 0x7fe2001a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe2000734c0 0x7fe20019c6b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe1e80a11d0 0x7fe1e80a3680 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 --2- 172.21.15.8:0/3487365359 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe200074a90 0x7fe20019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 -- 172.21.15.8:0/3487365359 >> 172.21.15.8:0/3487365359 conn(0x7fe2000fb930 msgr2=0x7fe2000fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:30.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 -- 172.21.15.8:0/3487365359 shutdown_connections 2023-11-27T15:25:30.740 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.735+0000 7fe2065ff700 1 -- 172.21.15.8:0/3487365359 wait complete. 2023-11-27T15:25:30.740 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2023-11-27T15:25:30.758 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:25:30.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:30 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/344737090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:30.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:30 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3756829369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:30.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:30 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/344737090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:30.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:30 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3756829369' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:30.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.959+0000 7f58c2dff700 1 -- 172.21.15.8:0/992517703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58bc1014c0 msgr2=0x7f58bc101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.959+0000 7f58c2dff700 1 --2- 172.21.15.8:0/992517703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58bc1014c0 0x7f58bc101910 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009990 tx=0x7f58a8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.960+0000 7f58c2dff700 1 -- 172.21.15.8:0/992517703 shutdown_connections 2023-11-27T15:25:30.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.960+0000 7f58c2dff700 1 --2- 172.21.15.8:0/992517703 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58bc101e50 0x7f58bc108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.963 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.960+0000 7f58c2dff700 1 --2- 172.21.15.8:0/992517703 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc1002d0 0x7f58bc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.960+0000 7f58c2dff700 1 --2- 172.21.15.8:0/992517703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58bc1014c0 0x7f58bc101910 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.960+0000 7f58c2dff700 1 -- 172.21.15.8:0/992517703 >> 172.21.15.8:0/992517703 conn(0x7f58bc0fb8b0 msgr2=0x7f58bc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:30.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.960+0000 7f58c2dff700 1 -- 172.21.15.8:0/992517703 shutdown_connections 2023-11-27T15:25:30.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.960+0000 7f58c2dff700 1 -- 172.21.15.8:0/992517703 wait complete. 2023-11-27T15:25:30.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.961+0000 7f58c2dff700 1 Processor -- start 2023-11-27T15:25:30.964 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.961+0000 7f58c2dff700 1 -- start start 2023-11-27T15:25:30.965 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.961+0000 7f58c2dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58bc1002d0 0x7f58bc198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.961+0000 7f58c2dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc1014c0 0x7f58bc198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.961+0000 7f58c2dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58bc101e50 0x7f58bc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.961+0000 7f58c2dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc19fdd0 con 0x7f58bc1002d0 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.961+0000 7f58c2dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc19ff10 con 0x7f58bc1014c0 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.961+0000 7f58c2dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1a00a0 con 0x7f58bc101e50 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.961+0000 7f58c0b9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58bc1002d0 0x7f58bc198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.961+0000 7f58c0b9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58bc1002d0 0x7f58bc198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39042/0 (socket says 172.21.15.8:39042) 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58c0b9b700 1 -- 172.21.15.8:0/533389103 learned_addr learned my addr 172.21.15.8:0/533389103 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58bbfff700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc1014c0 0x7f58bc198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58c139c700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58bc101e50 0x7f58bc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58c0b9b700 1 -- 172.21.15.8:0/533389103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58bc101e50 msgr2=0x7f58bc19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.966 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58c0b9b700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58bc101e50 0x7f58bc19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58c0b9b700 1 -- 172.21.15.8:0/533389103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc1014c0 msgr2=0x7f58bc198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:30.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58c0b9b700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc1014c0 0x7f58bc198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:30.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58c0b9b700 1 -- 172.21.15.8:0/533389103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80095f0 con 0x7f58bc1002d0 2023-11-27T15:25:30.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58c0b9b700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58bc1002d0 0x7f58bc198250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e970 tx=0x7f58b000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:30.967 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58b9ffb700 1 -- 172.21.15.8:0/533389103 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0009790 con 0x7f58bc1002d0 2023-11-27T15:25:30.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.962+0000 7f58b9ffb700 1 -- 172.21.15.8:0/533389103 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58b000c7e0 con 0x7f58bc1002d0 2023-11-27T15:25:30.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.963+0000 7f58c2dff700 1 -- 172.21.15.8:0/533389103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc10bc80 con 0x7f58bc1002d0 2023-11-27T15:25:30.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.963+0000 7f58b9ffb700 1 -- 172.21.15.8:0/533389103 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0003c70 con 0x7f58bc1002d0 2023-11-27T15:25:30.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.963+0000 7f58c2dff700 1 -- 172.21.15.8:0/533389103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc10baf0 con 0x7f58bc1002d0 2023-11-27T15:25:30.968 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.964+0000 7f58c2dff700 1 -- 172.21.15.8:0/533389103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc1929e0 con 0x7f58bc1002d0 2023-11-27T15:25:30.969 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.966+0000 7f58b9ffb700 1 -- 172.21.15.8:0/533389103 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f58b000cc50 con 0x7f58bc1002d0 2023-11-27T15:25:30.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.967+0000 7f58b9ffb700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f58ac0a1150 0x7f58ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:30.970 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.967+0000 7f58b9ffb700 1 -- 172.21.15.8:0/533389103 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f58b0014070 con 0x7f58bc1002d0 2023-11-27T15:25:30.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.967+0000 7f58bbfff700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f58ac0a1150 0x7f58ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:30.971 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.968+0000 7f58bbfff700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f58ac0a1150 0x7f58ac0a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58a800b2e0 tx=0x7f58a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:30.974 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:30.971+0000 7f58b9ffb700 1 -- 172.21.15.8:0/533389103 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f58b00961f0 con 0x7f58bc1002d0 2023-11-27T15:25:31.188 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.185+0000 7f58c2dff700 1 -- 172.21.15.8:0/533389103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58bc02cfe0 con 0x7f58bc1002d0 2023-11-27T15:25:31.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.187+0000 7f58b9ffb700 1 -- 172.21.15.8:0/533389103 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24344 (secure 0 0 0) 0x7f58b0003dd0 con 0x7f58bc1002d0 2023-11-27T15:25:31.191 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:31.191 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":90,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:24:40.249579+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:25:31.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.189+0000 7f58c2dff700 1 -- 172.21.15.8:0/533389103 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f58ac0a1150 msgr2=0x7f58ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:31.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.189+0000 7f58c2dff700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f58ac0a1150 0x7f58ac0a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58a800b2e0 tx=0x7f58a8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.189+0000 7f58c2dff700 1 -- 172.21.15.8:0/533389103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58bc1002d0 msgr2=0x7f58bc198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:31.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.189+0000 7f58c2dff700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58bc1002d0 0x7f58bc198250 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e970 tx=0x7f58b000ec80 comp rx=0 tx=0).stop 2023-11-27T15:25:31.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.189+0000 7f58c2dff700 1 -- 172.21.15.8:0/533389103 shutdown_connections 2023-11-27T15:25:31.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.189+0000 7f58c2dff700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f58ac0a1150 0x7f58ac0a3600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.189+0000 7f58c2dff700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f58bc101e50 0x7f58bc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.189+0000 7f58c2dff700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f58bc1002d0 0x7f58bc198250 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.189+0000 7f58c2dff700 1 --2- 172.21.15.8:0/533389103 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f58bc1014c0 0x7f58bc198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.189+0000 7f58c2dff700 1 -- 172.21.15.8:0/533389103 >> 172.21.15.8:0/533389103 conn(0x7f58bc0fb8b0 msgr2=0x7f58bc106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:31.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.190+0000 7f58c2dff700 1 -- 172.21.15.8:0/533389103 shutdown_connections 2023-11-27T15:25:31.194 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.190+0000 7f58c2dff700 1 -- 172.21.15.8:0/533389103 wait complete. 2023-11-27T15:25:31.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:23:25.166148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:23:26.584539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:23:30.130965+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:25:31.212 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:25:31.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.416+0000 7fe92f3ff700 1 -- 172.21.15.8:0/3129517187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe928073550 msgr2=0x7fe9280739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:31.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.416+0000 7fe92f3ff700 1 --2- 172.21.15.8:0/3129517187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe928073550 0x7fe9280739c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9180098f0 tx=0x7fe918009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:31.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.416+0000 7fe92f3ff700 1 -- 172.21.15.8:0/3129517187 shutdown_connections 2023-11-27T15:25:31.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.416+0000 7fe92f3ff700 1 --2- 172.21.15.8:0/3129517187 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe92810c860 0x7fe92810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.416+0000 7fe92f3ff700 1 --2- 172.21.15.8:0/3129517187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe928074b30 0x7fe928072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.416+0000 7fe92f3ff700 1 --2- 172.21.15.8:0/3129517187 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe928073550 0x7fe9280739c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.420 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.416+0000 7fe92f3ff700 1 -- 172.21.15.8:0/3129517187 >> 172.21.15.8:0/3129517187 conn(0x7fe9280fb870 msgr2=0x7fe9280fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:31.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.417+0000 7fe92f3ff700 1 -- 172.21.15.8:0/3129517187 shutdown_connections 2023-11-27T15:25:31.421 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.417+0000 7fe92f3ff700 1 -- 172.21.15.8:0/3129517187 wait complete. 2023-11-27T15:25:31.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.417+0000 7fe92f3ff700 1 Processor -- start 2023-11-27T15:25:31.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92f3ff700 1 -- start start 2023-11-27T15:25:31.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92f3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe928073550 0x7fe92819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:31.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92f3ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe928074b30 0x7fe92819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:31.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92f3ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe92810c860 0x7fe9281a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:31.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92f3ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9281a41e0 con 0x7fe928074b30 2023-11-27T15:25:31.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92f3ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9281a4320 con 0x7fe92810c860 2023-11-27T15:25:31.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92f3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9281a44b0 con 0x7fe928073550 2023-11-27T15:25:31.422 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92c99a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe928074b30 0x7fe92819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:31.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92c99a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe928074b30 0x7fe92819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39066/0 (socket says 172.21.15.8:39066) 2023-11-27T15:25:31.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92c99a700 1 -- 172.21.15.8:0/1856278685 learned_addr learned my addr 172.21.15.8:0/1856278685 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:31.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92d19b700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe928073550 0x7fe92819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:31.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.418+0000 7fe92d99c700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe92810c860 0x7fe9281a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:31.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe92c99a700 1 -- 172.21.15.8:0/1856278685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe928073550 msgr2=0x7fe92819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:31.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe92c99a700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe928073550 0x7fe92819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe92c99a700 1 -- 172.21.15.8:0/1856278685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe92810c860 msgr2=0x7fe9281a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:31.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe92c99a700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe92810c860 0x7fe9281a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.423 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe92c99a700 1 -- 172.21.15.8:0/1856278685 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9240095f0 con 0x7fe928074b30 2023-11-27T15:25:31.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe92c99a700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe928074b30 0x7fe92819cba0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe918009fd0 tx=0x7fe91800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:31.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe91e7fc700 1 -- 172.21.15.8:0/1856278685 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91801c070 con 0x7fe928074b30 2023-11-27T15:25:31.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe92f3ff700 1 -- 172.21.15.8:0/1856278685 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe918009520 con 0x7fe928074b30 2023-11-27T15:25:31.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe91e7fc700 1 -- 172.21.15.8:0/1856278685 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe91800f7d0 con 0x7fe928074b30 2023-11-27T15:25:31.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe91e7fc700 1 -- 172.21.15.8:0/1856278685 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918003a40 con 0x7fe928074b30 2023-11-27T15:25:31.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.419+0000 7fe92f3ff700 1 -- 172.21.15.8:0/1856278685 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe928112270 con 0x7fe928074b30 2023-11-27T15:25:31.424 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.420+0000 7fe92f3ff700 1 -- 172.21.15.8:0/1856278685 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9281023b0 con 0x7fe928074b30 2023-11-27T15:25:31.426 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.422+0000 7fe91e7fc700 1 -- 172.21.15.8:0/1856278685 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe918020030 con 0x7fe928074b30 2023-11-27T15:25:31.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.424+0000 7fe91e7fc700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9140a1150 0x7fe9140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:31.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.424+0000 7fe91e7fc700 1 -- 172.21.15.8:0/1856278685 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe9180cdbd0 con 0x7fe928074b30 2023-11-27T15:25:31.427 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.424+0000 7fe92d19b700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9140a1150 0x7fe9140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:31.428 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.425+0000 7fe92d19b700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9140a1150 0x7fe9140a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe928074f00 tx=0x7fe924009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:31.431 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.427+0000 7fe91e7fc700 1 -- 172.21.15.8:0/1856278685 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe918097040 con 0x7fe928074b30 2023-11-27T15:25:31.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:31 smithi008 ceph-mon[128788]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:25:31.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:31 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3487365359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:31.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:31 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/533389103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:31.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.638+0000 7fe92f3ff700 1 -- 172.21.15.8:0/1856278685 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe928067a50 con 0x7fe928074b30 2023-11-27T15:25:31.641 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.638+0000 7fe91e7fc700 1 -- 172.21.15.8:0/1856278685 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe91809a660 con 0x7fe928074b30 2023-11-27T15:25:31.642 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:25:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.640+0000 7fe92f3ff700 1 -- 172.21.15.8:0/1856278685 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9140a1150 msgr2=0x7fe9140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:31.644 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.640+0000 7fe92f3ff700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9140a1150 0x7fe9140a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe928074f00 tx=0x7fe924009310 comp rx=0 tx=0).stop 2023-11-27T15:25:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.640+0000 7fe92f3ff700 1 -- 172.21.15.8:0/1856278685 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe928074b30 msgr2=0x7fe92819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.640+0000 7fe92f3ff700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe928074b30 0x7fe92819cba0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe918009fd0 tx=0x7fe91800b790 comp rx=0 tx=0).stop 2023-11-27T15:25:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.641+0000 7fe92f3ff700 1 -- 172.21.15.8:0/1856278685 shutdown_connections 2023-11-27T15:25:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.641+0000 7fe92f3ff700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9140a1150 0x7fe9140a3600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.641+0000 7fe92f3ff700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe928073550 0x7fe92819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.641+0000 7fe92f3ff700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe92810c860 0x7fe9281a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.641+0000 7fe92f3ff700 1 --2- 172.21.15.8:0/1856278685 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe928074b30 0x7fe92819cba0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.641+0000 7fe92f3ff700 1 -- 172.21.15.8:0/1856278685 >> 172.21.15.8:0/1856278685 conn(0x7fe9280fb870 msgr2=0x7fe9280fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.641+0000 7fe92f3ff700 1 -- 172.21.15.8:0/1856278685 shutdown_connections 2023-11-27T15:25:31.645 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.641+0000 7fe92f3ff700 1 -- 172.21.15.8:0/1856278685 wait complete. 2023-11-27T15:25:31.665 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:25:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:31 smithi060 ceph-mon[131518]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:25:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:31 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3487365359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:31 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/533389103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:31.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.868+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1898633944 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee64101e10 msgr2=0x7fee641086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:31.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.868+0000 7fee6c7ff700 1 --2- 172.21.15.8:0/1898633944 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee64101e10 0x7fee641086d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fee5c00b0a0 tx=0x7fee5c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.868+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1898633944 shutdown_connections 2023-11-27T15:25:31.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.868+0000 7fee6c7ff700 1 --2- 172.21.15.8:0/1898633944 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee64100290 0x7fee641006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.868+0000 7fee6c7ff700 1 --2- 172.21.15.8:0/1898633944 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee64101e10 0x7fee641086d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.868+0000 7fee6c7ff700 1 --2- 172.21.15.8:0/1898633944 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee64101480 0x7fee641018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.868+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1898633944 >> 172.21.15.8:0/1898633944 conn(0x7fee640fb870 msgr2=0x7fee640fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:31.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.868+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1898633944 shutdown_connections 2023-11-27T15:25:31.872 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.868+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1898633944 wait complete. 2023-11-27T15:25:31.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.869+0000 7fee6c7ff700 1 Processor -- start 2023-11-27T15:25:31.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.869+0000 7fee6c7ff700 1 -- start start 2023-11-27T15:25:31.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee6c7ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee64100290 0x7fee641982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee6c7ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee64101480 0x7fee64198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee6c7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee64101e10 0x7fee6419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee6c7ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6419fe70 con 0x7fee64101480 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee6c7ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6419ffb0 con 0x7fee64100290 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee6c7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641a0140 con 0x7fee64101e10 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee69d9a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee64101480 0x7fee64198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee6ad9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee64101e10 0x7fee6419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee69d9a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee64101480 0x7fee64198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39086/0 (socket says 172.21.15.8:39086) 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee69d9a700 1 -- 172.21.15.8:0/1545609932 learned_addr learned my addr 172.21.15.8:0/1545609932 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee6a59b700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee64100290 0x7fee641982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee69d9a700 1 -- 172.21.15.8:0/1545609932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee64101e10 msgr2=0x7fee6419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee69d9a700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee64101e10 0x7fee6419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee69d9a700 1 -- 172.21.15.8:0/1545609932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee64100290 msgr2=0x7fee641982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee69d9a700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee64100290 0x7fee641982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.870+0000 7fee69d9a700 1 -- 172.21.15.8:0/1545609932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c009d00 con 0x7fee64101480 2023-11-27T15:25:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.871+0000 7fee69d9a700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee64101480 0x7fee64198830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fee5400e9a0 tx=0x7fee5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.871+0000 7fee5b7fe700 1 -- 172.21.15.8:0/1545609932 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54009790 con 0x7fee64101480 2023-11-27T15:25:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.871+0000 7fee5b7fe700 1 -- 172.21.15.8:0/1545609932 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee54010430 con 0x7fee64101480 2023-11-27T15:25:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.871+0000 7fee5b7fe700 1 -- 172.21.15.8:0/1545609932 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54018680 con 0x7fee64101480 2023-11-27T15:25:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.871+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1545609932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6410bc70 con 0x7fee64101480 2023-11-27T15:25:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.871+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1545609932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6410bae0 con 0x7fee64101480 2023-11-27T15:25:31.876 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.872+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1545609932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee641928e0 con 0x7fee64101480 2023-11-27T15:25:31.877 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.874+0000 7fee5b7fe700 1 -- 172.21.15.8:0/1545609932 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fee540105a0 con 0x7fee64101480 2023-11-27T15:25:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.875+0000 7fee5b7fe700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee500a11d0 0x7fee500a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.875+0000 7fee5b7fe700 1 -- 172.21.15.8:0/1545609932 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fee54014070 con 0x7fee64101480 2023-11-27T15:25:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.875+0000 7fee6a59b700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee500a11d0 0x7fee500a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:31.879 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.876+0000 7fee6a59b700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee500a11d0 0x7fee500a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fee641012e0 tx=0x7fee60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:31.882 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:31.879+0000 7fee5b7fe700 1 -- 172.21.15.8:0/1545609932 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fee540cee60 con 0x7fee64101480 2023-11-27T15:25:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:31 smithi029 ceph-mon[131166]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:25:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:31 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3487365359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:31 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/533389103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:32.091 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.088+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1545609932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee64061700 con 0x7fee64101480 2023-11-27T15:25:32.092 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.088+0000 7fee5b7fe700 1 -- 172.21.15.8:0/1545609932 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee540960a0 con 0x7fee64101480 2023-11-27T15:25:32.092 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:25:32.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1545609932 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee500a11d0 msgr2=0x7fee500a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee500a11d0 0x7fee500a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fee641012e0 tx=0x7fee60009260 comp rx=0 tx=0).stop 2023-11-27T15:25:32.094 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1545609932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee64101480 msgr2=0x7fee64198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee64101480 0x7fee64198830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fee5400e9a0 tx=0x7fee5400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1545609932 shutdown_connections 2023-11-27T15:25:32.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fee500a11d0 0x7fee500a3680 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fee64101e10 0x7fee6419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fee64100290 0x7fee641982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 --2- 172.21.15.8:0/1545609932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fee64101480 0x7fee64198830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1545609932 >> 172.21.15.8:0/1545609932 conn(0x7fee640fb870 msgr2=0x7fee64106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:32.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1545609932 shutdown_connections 2023-11-27T15:25:32.095 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.091+0000 7fee6c7ff700 1 -- 172.21.15.8:0/1545609932 wait complete. 2023-11-27T15:25:32.114 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:25:32.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.317+0000 7fd1551ff700 1 -- 172.21.15.8:0/1949169630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd150074b30 msgr2=0x7fd150072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.317+0000 7fd1551ff700 1 --2- 172.21.15.8:0/1949169630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd150074b30 0x7fd150072f80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd140009f90 tx=0x7fd14000b330 comp rx=0 tx=0).stop 2023-11-27T15:25:32.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.317+0000 7fd1551ff700 1 -- 172.21.15.8:0/1949169630 shutdown_connections 2023-11-27T15:25:32.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.317+0000 7fd1551ff700 1 --2- 172.21.15.8:0/1949169630 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd150074b30 0x7fd150072f80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.317+0000 7fd1551ff700 1 --2- 172.21.15.8:0/1949169630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd150108150 0x7fd15010a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.317+0000 7fd1551ff700 1 --2- 172.21.15.8:0/1949169630 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd150073550 0x7fd1500739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.317+0000 7fd1551ff700 1 -- 172.21.15.8:0/1949169630 >> 172.21.15.8:0/1949169630 conn(0x7fd1500fb870 msgr2=0x7fd1500fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:32.321 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.317+0000 7fd1551ff700 1 -- 172.21.15.8:0/1949169630 shutdown_connections 2023-11-27T15:25:32.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.317+0000 7fd1551ff700 1 -- 172.21.15.8:0/1949169630 wait complete. 2023-11-27T15:25:32.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.319+0000 7fd1551ff700 1 Processor -- start 2023-11-27T15:25:32.322 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.319+0000 7fd1551ff700 1 -- start start 2023-11-27T15:25:32.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.319+0000 7fd1551ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd150073550 0x7fd150198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:32.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.319+0000 7fd1551ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd150074b30 0x7fd1501988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:32.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.319+0000 7fd1551ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd150108150 0x7fd15019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:32.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.319+0000 7fd1551ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15019ff00 con 0x7fd150108150 2023-11-27T15:25:32.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.319+0000 7fd1551ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1501a0040 con 0x7fd150073550 2023-11-27T15:25:32.324 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.319+0000 7fd1551ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1501a01d0 con 0x7fd150074b30 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14f59e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd150108150 0x7fd15019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14f59e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd150108150 0x7fd15019f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39110/0 (socket says 172.21.15.8:39110) 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14f59e700 1 -- 172.21.15.8:0/2098660951 learned_addr learned my addr 172.21.15.8:0/2098660951 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14ed9d700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd150073550 0x7fd150198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14659c700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd150074b30 0x7fd1501988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14f59e700 1 -- 172.21.15.8:0/2098660951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd150074b30 msgr2=0x7fd1501988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14f59e700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd150074b30 0x7fd1501988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14f59e700 1 -- 172.21.15.8:0/2098660951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd150073550 msgr2=0x7fd150198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14f59e700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd150073550 0x7fd150198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14f59e700 1 -- 172.21.15.8:0/2098660951 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480095f0 con 0x7fd150108150 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.320+0000 7fd14f59e700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd150108150 0x7fd15019f930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd14800eb80 tx=0x7fd14800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:32.325 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.321+0000 7fd14cd99700 1 -- 172.21.15.8:0/2098660951 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1480099f0 con 0x7fd150108150 2023-11-27T15:25:32.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.321+0000 7fd14cd99700 1 -- 172.21.15.8:0/2098660951 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1480044f0 con 0x7fd150108150 2023-11-27T15:25:32.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.321+0000 7fd1551ff700 1 -- 172.21.15.8:0/2098660951 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd140009c40 con 0x7fd150108150 2023-11-27T15:25:32.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.321+0000 7fd14cd99700 1 -- 172.21.15.8:0/2098660951 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148003e90 con 0x7fd150108150 2023-11-27T15:25:32.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.321+0000 7fd1551ff700 1 -- 172.21.15.8:0/2098660951 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15010de50 con 0x7fd150108150 2023-11-27T15:25:32.326 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.322+0000 7fd1551ff700 1 -- 172.21.15.8:0/2098660951 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15010dfe0 con 0x7fd150108150 2023-11-27T15:25:32.327 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.324+0000 7fd14cd99700 1 -- 172.21.15.8:0/2098660951 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd148004000 con 0x7fd150108150 2023-11-27T15:25:32.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.326+0000 7fd14cd99700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd13c0a1150 0x7fd13c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:32.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.326+0000 7fd14cd99700 1 -- 172.21.15.8:0/2098660951 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd148014070 con 0x7fd150108150 2023-11-27T15:25:32.329 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.326+0000 7fd14ed9d700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd13c0a1150 0x7fd13c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:32.330 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.326+0000 7fd14ed9d700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd13c0a1150 0x7fd13c0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd140007900 tx=0x7fd1400096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:32.333 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.329+0000 7fd14cd99700 1 -- 172.21.15.8:0/2098660951 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd1480964f0 con 0x7fd150108150 2023-11-27T15:25:32.544 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.541+0000 7fd1551ff700 1 -- 172.21.15.8:0/2098660951 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd150061840 con 0x7fd150108150 2023-11-27T15:25:32.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.541+0000 7fd14cd99700 1 -- 172.21.15.8:0/2098660951 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd148099b10 con 0x7fd150108150 2023-11-27T15:25:32.545 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-11-27T15:25:32.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 -- 172.21.15.8:0/2098660951 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd13c0a1150 msgr2=0x7fd13c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd13c0a1150 0x7fd13c0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd140007900 tx=0x7fd1400096c0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 -- 172.21.15.8:0/2098660951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd150108150 msgr2=0x7fd15019f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd150108150 0x7fd15019f930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd14800eb80 tx=0x7fd14800ee90 comp rx=0 tx=0).stop 2023-11-27T15:25:32.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 -- 172.21.15.8:0/2098660951 shutdown_connections 2023-11-27T15:25:32.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd150108150 0x7fd15019f930 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fd150073550 0x7fd150198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fd13c0a1150 0x7fd13c0a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.548 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 --2- 172.21.15.8:0/2098660951 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fd150074b30 0x7fd1501988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 -- 172.21.15.8:0/2098660951 >> 172.21.15.8:0/2098660951 conn(0x7fd1500fb870 msgr2=0x7fd1500fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:32.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 -- 172.21.15.8:0/2098660951 shutdown_connections 2023-11-27T15:25:32.549 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.544+0000 7fd1551ff700 1 -- 172.21.15.8:0/2098660951 wait complete. 2023-11-27T15:25:32.567 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:25:32.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.770+0000 7fa248dff700 1 -- 172.21.15.8:0/3713190119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa244074b30 msgr2=0x7fa244072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.770+0000 7fa248dff700 1 --2- 172.21.15.8:0/3713190119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa244074b30 0x7fa244072f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa234009990 tx=0x7fa234009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.770+0000 7fa248dff700 1 -- 172.21.15.8:0/3713190119 shutdown_connections 2023-11-27T15:25:32.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.770+0000 7fa248dff700 1 --2- 172.21.15.8:0/3713190119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa244108150 0x7fa24410a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.770+0000 7fa248dff700 1 --2- 172.21.15.8:0/3713190119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa244074b30 0x7fa244072f80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.770+0000 7fa248dff700 1 --2- 172.21.15.8:0/3713190119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa244073550 0x7fa2440739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.770+0000 7fa248dff700 1 -- 172.21.15.8:0/3713190119 >> 172.21.15.8:0/3713190119 conn(0x7fa2440fb870 msgr2=0x7fa2440fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:32.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.770+0000 7fa248dff700 1 -- 172.21.15.8:0/3713190119 shutdown_connections 2023-11-27T15:25:32.774 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.770+0000 7fa248dff700 1 -- 172.21.15.8:0/3713190119 wait complete. 2023-11-27T15:25:32.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.771+0000 7fa248dff700 1 Processor -- start 2023-11-27T15:25:32.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.771+0000 7fa248dff700 1 -- start start 2023-11-27T15:25:32.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.771+0000 7fa248dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa244073550 0x7fa2441981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:32.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.771+0000 7fa248dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa244074b30 0x7fa244198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:32.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa248dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa244108150 0x7fa24419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:32.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa248dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24419fd50 con 0x7fa244073550 2023-11-27T15:25:32.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa248dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24419fe90 con 0x7fa244108150 2023-11-27T15:25:32.776 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa248dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2441a0020 con 0x7fa244074b30 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa24259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa244073550 0x7fa2441981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa23bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa244074b30 0x7fa244198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa23bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa244074b30 0x7fa244198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:59404/0 (socket says 172.21.15.8:59404) 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa23bfff700 1 -- 172.21.15.8:0/367530887 learned_addr learned my addr 172.21.15.8:0/367530887 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa24259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa244073550 0x7fa2441981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39138/0 (socket says 172.21.15.8:39138) 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa242d9d700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa244108150 0x7fa24419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa23bfff700 1 -- 172.21.15.8:0/367530887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa244108150 msgr2=0x7fa24419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa23bfff700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa244108150 0x7fa24419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa23bfff700 1 -- 172.21.15.8:0/367530887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa244073550 msgr2=0x7fa2441981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa23bfff700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa244073550 0x7fa2441981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa23bfff700 1 -- 172.21.15.8:0/367530887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340095f0 con 0x7fa244074b30 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.772+0000 7fa24259c700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa244073550 0x7fa2441981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:32.777 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.773+0000 7fa23bfff700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa244074b30 0x7fa244198710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00e970 tx=0x7fa22c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:32.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.773+0000 7fa23b7fe700 1 -- 172.21.15.8:0/367530887 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c009790 con 0x7fa244074b30 2023-11-27T15:25:32.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.773+0000 7fa248dff700 1 -- 172.21.15.8:0/367530887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24410de10 con 0x7fa244074b30 2023-11-27T15:25:32.778 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.773+0000 7fa248dff700 1 -- 172.21.15.8:0/367530887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24410dc80 con 0x7fa244074b30 2023-11-27T15:25:32.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.773+0000 7fa23b7fe700 1 -- 172.21.15.8:0/367530887 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22c00c7e0 con 0x7fa244074b30 2023-11-27T15:25:32.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.773+0000 7fa23b7fe700 1 -- 172.21.15.8:0/367530887 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c003b60 con 0x7fa244074b30 2023-11-27T15:25:32.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.774+0000 7fa248dff700 1 -- 172.21.15.8:0/367530887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa244106800 con 0x7fa244074b30 2023-11-27T15:25:32.779 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.776+0000 7fa23b7fe700 1 -- 172.21.15.8:0/367530887 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa22c004220 con 0x7fa244074b30 2023-11-27T15:25:32.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.777+0000 7fa23b7fe700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2300a11d0 0x7fa2300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:32.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.777+0000 7fa23b7fe700 1 -- 172.21.15.8:0/367530887 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa22c014070 con 0x7fa244074b30 2023-11-27T15:25:32.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.777+0000 7fa24259c700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2300a11d0 0x7fa2300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:32.781 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.778+0000 7fa24259c700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2300a11d0 0x7fa2300a3680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa23400b680 tx=0x7fa234005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:32.785 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.781+0000 7fa23b7fe700 1 -- 172.21.15.8:0/367530887 <== mon.1 v2:172.21.15.60:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa22c096200 con 0x7fa244074b30 2023-11-27T15:25:32.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:32 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1856278685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:25:32.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:32 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1545609932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:32 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1856278685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:25:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:32 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1545609932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:32 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1856278685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:25:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:32 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1545609932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:25:32.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.992+0000 7fa248dff700 1 -- 172.21.15.8:0/367530887 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa244061700 con 0x7fa244074b30 2023-11-27T15:25:32.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.993+0000 7fa23b7fe700 1 -- 172.21.15.8:0/367530887 <== mon.1 v2:172.21.15.60:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa22c099820 con 0x7fa244074b30 2023-11-27T15:25:32.996 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:25:32.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 -- 172.21.15.8:0/367530887 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2300a11d0 msgr2=0x7fa2300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2300a11d0 0x7fa2300a3680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa23400b680 tx=0x7fa234005fd0 comp rx=0 tx=0).stop 2023-11-27T15:25:32.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 -- 172.21.15.8:0/367530887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa244074b30 msgr2=0x7fa244198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:32.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa244074b30 0x7fa244198710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00e970 tx=0x7fa22c00ec80 comp rx=0 tx=0).stop 2023-11-27T15:25:32.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 -- 172.21.15.8:0/367530887 shutdown_connections 2023-11-27T15:25:32.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa244108150 0x7fa24419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa244073550 0x7fa2441981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2300a11d0 0x7fa2300a3680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 --2- 172.21.15.8:0/367530887 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa244074b30 0x7fa244198710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 -- 172.21.15.8:0/367530887 >> 172.21.15.8:0/367530887 conn(0x7fa2440fb870 msgr2=0x7fa2440fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:33.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 -- 172.21.15.8:0/367530887 shutdown_connections 2023-11-27T15:25:33.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:32.995+0000 7fa248dff700 1 -- 172.21.15.8:0/367530887 wait complete. 2023-11-27T15:25:33.018 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:25:33.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.218+0000 7f1899dff700 1 -- 172.21.15.8:0/2835258132 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18941014c0 msgr2=0x7f1894101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:33.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.218+0000 7f1899dff700 1 --2- 172.21.15.8:0/2835258132 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18941014c0 0x7f1894101910 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f187c009990 tx=0x7f187c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.219+0000 7f1899dff700 1 -- 172.21.15.8:0/2835258132 shutdown_connections 2023-11-27T15:25:33.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.219+0000 7f1899dff700 1 --2- 172.21.15.8:0/2835258132 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18941002d0 0x7f18941006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.219+0000 7f1899dff700 1 --2- 172.21.15.8:0/2835258132 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1894101e50 0x7f1894108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.219+0000 7f1899dff700 1 --2- 172.21.15.8:0/2835258132 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18941014c0 0x7f1894101910 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.219+0000 7f1899dff700 1 -- 172.21.15.8:0/2835258132 >> 172.21.15.8:0/2835258132 conn(0x7f18940fb8b0 msgr2=0x7f18940fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:33.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.219+0000 7f1899dff700 1 -- 172.21.15.8:0/2835258132 shutdown_connections 2023-11-27T15:25:33.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.219+0000 7f1899dff700 1 -- 172.21.15.8:0/2835258132 wait complete. 2023-11-27T15:25:33.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f1899dff700 1 Processor -- start 2023-11-27T15:25:33.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f1899dff700 1 -- start start 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f1899dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18941002d0 0x7f1894198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f1899dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18941014c0 0x7f1894198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f1899dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1894101e50 0x7f189419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f1899dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419fdd0 con 0x7f18941002d0 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f1899dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419ff10 con 0x7f1894101e50 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f1899dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18941a00a0 con 0x7f18941014c0 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f18937fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18941002d0 0x7f1894198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f18937fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18941002d0 0x7f1894198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39166/0 (socket says 172.21.15.8:39166) 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.220+0000 7f18937fe700 1 -- 172.21.15.8:0/1771110096 learned_addr learned my addr 172.21.15.8:0/1771110096 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f188bfff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18941014c0 0x7f1894198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f1893fff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1894101e50 0x7f189419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f18937fe700 1 -- 172.21.15.8:0/1771110096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18941014c0 msgr2=0x7f1894198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f18937fe700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18941014c0 0x7f1894198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f18937fe700 1 -- 172.21.15.8:0/1771110096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1894101e50 msgr2=0x7f189419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:33.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f18937fe700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1894101e50 0x7f189419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f18937fe700 1 -- 172.21.15.8:0/1771110096 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0095f0 con 0x7f18941002d0 2023-11-27T15:25:33.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f1893fff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1894101e50 0x7f189419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:33.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f18937fe700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18941002d0 0x7f1894198250 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f188400e970 tx=0x7f188400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:33.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f18917fa700 1 -- 172.21.15.8:0/1771110096 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884009790 con 0x7f18941002d0 2023-11-27T15:25:33.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f18917fa700 1 -- 172.21.15.8:0/1771110096 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f188400eec0 con 0x7f18941002d0 2023-11-27T15:25:33.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f18917fa700 1 -- 172.21.15.8:0/1771110096 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884003cf0 con 0x7f18941002d0 2023-11-27T15:25:33.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.221+0000 7f1899dff700 1 -- 172.21.15.8:0/1771110096 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189410bc80 con 0x7f18941002d0 2023-11-27T15:25:33.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.222+0000 7f1899dff700 1 -- 172.21.15.8:0/1771110096 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189410baf0 con 0x7f18941002d0 2023-11-27T15:25:33.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.223+0000 7f1899dff700 1 -- 172.21.15.8:0/1771110096 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18941929e0 con 0x7f18941002d0 2023-11-27T15:25:33.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.225+0000 7f18917fa700 1 -- 172.21.15.8:0/1771110096 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f188400ccd0 con 0x7f18941002d0 2023-11-27T15:25:33.229 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.226+0000 7f18917fa700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18800a1150 0x7f18800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:33.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.226+0000 7f18917fa700 1 -- 172.21.15.8:0/1771110096 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1884014070 con 0x7f18941002d0 2023-11-27T15:25:33.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.226+0000 7f188bfff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18800a1150 0x7f18800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:33.230 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.227+0000 7f188bfff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18800a1150 0x7f18800a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f187c00b2e0 tx=0x7f187c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:33.233 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.230+0000 7f18917fa700 1 -- 172.21.15.8:0/1771110096 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f18840d2020 con 0x7f18941002d0 2023-11-27T15:25:33.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.498+0000 7f1899dff700 1 -- 172.21.15.8:0/1771110096 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f189402cfe0 con 0x7f18941002d0 2023-11-27T15:25:33.502 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.499+0000 7f18917fa700 1 -- 172.21.15.8:0/1771110096 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10768 (secure 0 0 0) 0x7f18840962b0 con 0x7f18941002d0 2023-11-27T15:25:33.503 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:33.503 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/789896869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":789896869},{"type":"v1","addr":"172.21.15.29:6835","nonce":789896869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:23:27.167939+0000","modified":"2023-11-27T15:23:59.241472+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24595,"mds_1":24593,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24565":{"gid":24565,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/3185908756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":3185908756},{"type":"v1","addr":"172.21.15.60:6839","nonce":3185908756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24593":{"gid":24593,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24595":{"gid":24595,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:25:33.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.501+0000 7f1899dff700 1 -- 172.21.15.8:0/1771110096 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18800a1150 msgr2=0x7f18800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:33.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.501+0000 7f1899dff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18800a1150 0x7f18800a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f187c00b2e0 tx=0x7f187c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.501+0000 7f1899dff700 1 -- 172.21.15.8:0/1771110096 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18941002d0 msgr2=0x7f1894198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:33.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.501+0000 7f1899dff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18941002d0 0x7f1894198250 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f188400e970 tx=0x7f188400ec80 comp rx=0 tx=0).stop 2023-11-27T15:25:33.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.501+0000 7f1899dff700 1 -- 172.21.15.8:0/1771110096 shutdown_connections 2023-11-27T15:25:33.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.501+0000 7f1899dff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1894101e50 0x7f189419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.505 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.501+0000 7f1899dff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18941002d0 0x7f1894198250 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.501+0000 7f1899dff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f18800a1150 0x7f18800a3600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.501+0000 7f1899dff700 1 --2- 172.21.15.8:0/1771110096 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f18941014c0 0x7f1894198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.501+0000 7f1899dff700 1 -- 172.21.15.8:0/1771110096 >> 172.21.15.8:0/1771110096 conn(0x7f18940fb8b0 msgr2=0x7f1894106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:33.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.502+0000 7f1899dff700 1 -- 172.21.15.8:0/1771110096 shutdown_connections 2023-11-27T15:25:33.506 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.502+0000 7f1899dff700 1 -- 172.21.15.8:0/1771110096 wait complete. 2023-11-27T15:25:33.506 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2023-11-27T15:25:33.525 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:25:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.725+0000 7f95849ff700 1 -- 172.21.15.8:0/2386940422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9580101e50 msgr2=0x7f9580108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.725+0000 7f95849ff700 1 --2- 172.21.15.8:0/2386940422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9580101e50 0x7f9580108710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f957800b0a0 tx=0x7f957800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.725+0000 7f95849ff700 1 -- 172.21.15.8:0/2386940422 shutdown_connections 2023-11-27T15:25:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.725+0000 7f95849ff700 1 --2- 172.21.15.8:0/2386940422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9580101e50 0x7f9580108710 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.725+0000 7f95849ff700 1 --2- 172.21.15.8:0/2386940422 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95801002d0 0x7f95801006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.725+0000 7f95849ff700 1 --2- 172.21.15.8:0/2386940422 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801014c0 0x7f9580101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.725+0000 7f95849ff700 1 -- 172.21.15.8:0/2386940422 >> 172.21.15.8:0/2386940422 conn(0x7f95800fb8b0 msgr2=0x7f95800fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:33.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.725+0000 7f95849ff700 1 -- 172.21.15.8:0/2386940422 shutdown_connections 2023-11-27T15:25:33.730 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.725+0000 7f95849ff700 1 -- 172.21.15.8:0/2386940422 wait complete. 2023-11-27T15:25:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.726+0000 7f95849ff700 1 Processor -- start 2023-11-27T15:25:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.726+0000 7f95849ff700 1 -- start start 2023-11-27T15:25:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f95849ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95801002d0 0x7f9580198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f95849ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801014c0 0x7f95801988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f95849ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9580101e50 0x7f958019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f95849ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958019fef0 con 0x7f9580101e50 2023-11-27T15:25:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f95849ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801a0030 con 0x7f95801002d0 2023-11-27T15:25:33.732 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f95849ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801a01c0 con 0x7f95801014c0 2023-11-27T15:25:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f957e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95801002d0 0x7f9580198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f957ed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9580101e50 0x7f958019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f9577fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801014c0 0x7f95801988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f957e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95801002d0 0x7f9580198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:46842/0 (socket says 172.21.15.8:46842) 2023-11-27T15:25:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f957e59c700 1 -- 172.21.15.8:0/2248810662 learned_addr learned my addr 172.21.15.8:0/2248810662 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f957e59c700 1 -- 172.21.15.8:0/2248810662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801014c0 msgr2=0x7f95801988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f957e59c700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801014c0 0x7f95801988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f957e59c700 1 -- 172.21.15.8:0/2248810662 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9580101e50 msgr2=0x7f958019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f957e59c700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9580101e50 0x7f958019f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f957e59c700 1 -- 172.21.15.8:0/2248810662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578009d00 con 0x7f95801002d0 2023-11-27T15:25:33.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.727+0000 7f957ed9d700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9580101e50 0x7f958019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:33.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.728+0000 7f957e59c700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95801002d0 0x7f9580198370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f957000e970 tx=0x7f957000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:33.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.728+0000 7f95777fe700 1 -- 172.21.15.8:0/2248810662 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570009790 con 0x7f95801002d0 2023-11-27T15:25:33.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.728+0000 7f95849ff700 1 -- 172.21.15.8:0/2248810662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958010bca0 con 0x7f95801002d0 2023-11-27T15:25:33.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.728+0000 7f95777fe700 1 -- 172.21.15.8:0/2248810662 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f957000c7e0 con 0x7f95801002d0 2023-11-27T15:25:33.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.728+0000 7f95777fe700 1 -- 172.21.15.8:0/2248810662 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570003c70 con 0x7f95801002d0 2023-11-27T15:25:33.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.728+0000 7f95849ff700 1 -- 172.21.15.8:0/2248810662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958010bb10 con 0x7f95801002d0 2023-11-27T15:25:33.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.729+0000 7f95849ff700 1 -- 172.21.15.8:0/2248810662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95801a09c0 con 0x7f95801002d0 2023-11-27T15:25:33.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.731+0000 7f95777fe700 1 -- 172.21.15.8:0/2248810662 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f957000cc50 con 0x7f95801002d0 2023-11-27T15:25:33.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.732+0000 7f95777fe700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f956c0a1150 0x7f956c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:33.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.732+0000 7f95777fe700 1 -- 172.21.15.8:0/2248810662 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9570014070 con 0x7f95801002d0 2023-11-27T15:25:33.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.733+0000 7f9577fff700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f956c0a1150 0x7f956c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:33.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.733+0000 7f9577fff700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f956c0a1150 0x7f956c0a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9568009c60 tx=0x7f9568009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:33.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.734+0000 7f95777fe700 1 -- 172.21.15.8:0/2248810662 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f95700967f0 con 0x7f95801002d0 2023-11-27T15:25:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:33 smithi060 ceph-mon[131518]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:25:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:33 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2098660951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:33 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/367530887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:33.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:33 smithi008 ceph-mon[128788]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:25:33.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:33 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2098660951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:33.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:33 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/367530887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:33 smithi029 ceph-mon[131166]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:25:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:33 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2098660951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:25:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:33 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/367530887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:25:33.947 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.943+0000 7f95849ff700 1 -- 172.21.15.8:0/2248810662 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95801a0380 con 0x7f95801002d0 2023-11-27T15:25:33.949 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.945+0000 7f95777fe700 1 -- 172.21.15.8:0/2248810662 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24344 (secure 0 0 0) 0x7f9570003dd0 con 0x7f95801002d0 2023-11-27T15:25:33.949 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:25:33.950 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":90,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:24:40.249579+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:25:33.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 -- 172.21.15.8:0/2248810662 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f956c0a1150 msgr2=0x7f956c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:33.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f956c0a1150 0x7f956c0a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9568009c60 tx=0x7f9568009260 comp rx=0 tx=0).stop 2023-11-27T15:25:33.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 -- 172.21.15.8:0/2248810662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95801002d0 msgr2=0x7f9580198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:33.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95801002d0 0x7f9580198370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f957000e970 tx=0x7f957000ec80 comp rx=0 tx=0).stop 2023-11-27T15:25:33.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 -- 172.21.15.8:0/2248810662 shutdown_connections 2023-11-27T15:25:33.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9580101e50 0x7f958019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.952 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95801002d0 0x7f9580198370 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f956c0a1150 0x7f956c0a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 --2- 172.21.15.8:0/2248810662 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95801014c0 0x7f95801988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:33.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 -- 172.21.15.8:0/2248810662 >> 172.21.15.8:0/2248810662 conn(0x7f95800fb8b0 msgr2=0x7f9580106d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:33.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 -- 172.21.15.8:0/2248810662 shutdown_connections 2023-11-27T15:25:33.953 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:33.948+0000 7f95849ff700 1 -- 172.21.15.8:0/2248810662 wait complete. 2023-11-27T15:25:33.972 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-27T15:23:27.167939+0000', 'modified': '2023-11-27T15:23:59.241472+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24595, 'mds_1': 24593, 'mds_2': 24539}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24535': {'gid': 24535, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6835/612130958', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6834', 'nonce': 612130958}, {'type': 'v1', 'addr': '172.21.15.60:6835', 'nonce': 612130958}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.29:6837/1651468308', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6836', 'nonce': 1651468308}, {'type': 'v1', 'addr': '172.21.15.29:6837', 'nonce': 1651468308}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24547': {'gid': 24547, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6837/287097286', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6836', 'nonce': 287097286}, {'type': 'v1', 'addr': '172.21.15.60:6837', 'nonce': 287097286}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24565': {'gid': 24565, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.60:6839/3185908756', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6838', 'nonce': 3185908756}, {'type': 'v1', 'addr': '172.21.15.60:6839', 'nonce': 3185908756}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24593': {'gid': 24593, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.29:6839/2619830143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.29:6838', 'nonce': 2619830143}, {'type': 'v1', 'addr': '172.21.15.29:6839', 'nonce': 2619830143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24595': {'gid': 24595, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.8:6841/110161953', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.8:6840', 'nonce': 110161953}, {'type': 'v1', 'addr': '172.21.15.8:6841', 'nonce': 110161953}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-27T15:25:33.972 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-27T15:25:33.972 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-11-27T15:25:34.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.175+0000 7f2122fff700 1 -- 172.21.15.8:0/3561627961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f211c100230 msgr2=0x7f211c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.175+0000 7f2122fff700 1 --2- 172.21.15.8:0/3561627961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f211c100230 0x7f211c100640 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2110009990 tx=0x7f2110009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.175+0000 7f2122fff700 1 -- 172.21.15.8:0/3561627961 shutdown_connections 2023-11-27T15:25:34.179 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.175+0000 7f2122fff700 1 --2- 172.21.15.8:0/3561627961 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c101db0 0x7f211c108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.175+0000 7f2122fff700 1 --2- 172.21.15.8:0/3561627961 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f211c100230 0x7f211c100640 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.175+0000 7f2122fff700 1 --2- 172.21.15.8:0/3561627961 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c101420 0x7f211c101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.175+0000 7f2122fff700 1 -- 172.21.15.8:0/3561627961 >> 172.21.15.8:0/3561627961 conn(0x7f211c0fb810 msgr2=0x7f211c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.175+0000 7f2122fff700 1 -- 172.21.15.8:0/3561627961 shutdown_connections 2023-11-27T15:25:34.180 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.175+0000 7f2122fff700 1 -- 172.21.15.8:0/3561627961 wait complete. 2023-11-27T15:25:34.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.177+0000 7f2122fff700 1 Processor -- start 2023-11-27T15:25:34.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.177+0000 7f2122fff700 1 -- start start 2023-11-27T15:25:34.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.177+0000 7f2122fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c100230 0x7f211c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.183 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.177+0000 7f2122fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c101420 0x7f211c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.177+0000 7f2122fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f211c101db0 0x7f211c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.177+0000 7f2122fff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c19fd90 con 0x7f211c100230 2023-11-27T15:25:34.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.177+0000 7f2122fff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c19fed0 con 0x7f211c101db0 2023-11-27T15:25:34.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.177+0000 7f2122fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c1a0060 con 0x7f211c101420 2023-11-27T15:25:34.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f2120d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c100230 0x7f211c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f2120d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c100230 0x7f211c198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39200/0 (socket says 172.21.15.8:39200) 2023-11-27T15:25:34.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f2120d9b700 1 -- 172.21.15.8:0/2073184279 learned_addr learned my addr 172.21.15.8:0/2073184279 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f212159c700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f211c101db0 0x7f211c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.184 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f211bfff700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c101420 0x7f211c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f2120d9b700 1 -- 172.21.15.8:0/2073184279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c101420 msgr2=0x7f211c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f2120d9b700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c101420 0x7f211c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f2120d9b700 1 -- 172.21.15.8:0/2073184279 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f211c101db0 msgr2=0x7f211c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f2120d9b700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f211c101db0 0x7f211c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f2120d9b700 1 -- 172.21.15.8:0/2073184279 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21100095f0 con 0x7f211c100230 2023-11-27T15:25:34.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f211bfff700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c101420 0x7f211c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:34.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f212159c700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f211c101db0 0x7f211c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:34.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f2120d9b700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c100230 0x7f211c198210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2110004730 tx=0x7f2110009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.178+0000 7f2119ffb700 1 -- 172.21.15.8:0/2073184279 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211001d070 con 0x7f211c100230 2023-11-27T15:25:34.185 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.179+0000 7f2119ffb700 1 -- 172.21.15.8:0/2073184279 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2110004cd0 con 0x7f211c100230 2023-11-27T15:25:34.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.179+0000 7f2119ffb700 1 -- 172.21.15.8:0/2073184279 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110005530 con 0x7f211c100230 2023-11-27T15:25:34.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.179+0000 7f2122fff700 1 -- 172.21.15.8:0/2073184279 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c10bbe0 con 0x7f211c100230 2023-11-27T15:25:34.186 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.179+0000 7f2122fff700 1 -- 172.21.15.8:0/2073184279 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c10ba50 con 0x7f211c100230 2023-11-27T15:25:34.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.182+0000 7f2119ffb700 1 -- 172.21.15.8:0/2073184279 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f211000fbc0 con 0x7f211c100230 2023-11-27T15:25:34.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.183+0000 7f213685cb80 1 Processor -- start 2023-11-27T15:25:34.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.183+0000 7f2119ffb700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f210c0a1150 0x7f210c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.183+0000 7f213685cb80 1 -- start start 2023-11-27T15:25:34.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.183+0000 7f2119ffb700 1 -- 172.21.15.8:0/2073184279 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f21100cdf80 con 0x7f211c100230 2023-11-27T15:25:34.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f212159c700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f210c0a1150 0x7f210c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f213685cb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f80efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.190 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f213685cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80f500 0x55ca4f80fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f213685cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f8100c0 0x55ca4f810530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f213685cb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f714010 con 0x55ca4f80ebb0 2023-11-27T15:25:34.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f213685cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f766850 con 0x55ca4f8100c0 2023-11-27T15:25:34.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f213685cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f75fce0 con 0x55ca4f80f500 2023-11-27T15:25:34.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f2120d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f80efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f2120d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f80efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39204/0 (socket says 172.21.15.8:39204) 2023-11-27T15:25:34.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f2120d9b700 1 -- 172.21.15.8:0/2172104035 learned_addr learned my addr 172.21.15.8:0/2172104035 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f212159c700 1 --2- 172.21.15.8:0/2172104035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f8100c0 0x55ca4f810530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.191 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f211bfff700 1 --2- 172.21.15.8:0/2172104035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80f500 0x55ca4f80fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f2120d9b700 1 -- 172.21.15.8:0/2172104035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80f500 msgr2=0x55ca4f80fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f2120d9b700 1 --2- 172.21.15.8:0/2172104035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80f500 0x55ca4f80fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f2120d9b700 1 -- 172.21.15.8:0/2172104035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f8100c0 msgr2=0x55ca4f810530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f2120d9b700 1 --2- 172.21.15.8:0/2172104035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f8100c0 0x55ca4f810530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f2120d9b700 1 -- 172.21.15.8:0/2172104035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca4f5e15e0 con 0x55ca4f80ebb0 2023-11-27T15:25:34.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.184+0000 7f212159c700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f210c0a1150 0x7f210c0a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f21140093c0 tx=0x7f2114009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.185+0000 7f2120d9b700 1 --2- 172.21.15.8:0/2172104035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f80efc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f211009adb0 tx=0x7f211000ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a43ac81790aefb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.185+0000 7f2106ffd700 1 -- 172.21.15.8:0/2172104035 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21100a3470 con 0x55ca4f80ebb0 2023-11-27T15:25:34.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.185+0000 7f2106ffd700 1 -- 172.21.15.8:0/2172104035 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2110005690 con 0x55ca4f80ebb0 2023-11-27T15:25:34.192 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.185+0000 7f2106ffd700 1 -- 172.21.15.8:0/2172104035 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21100cbbd0 con 0x55ca4f80ebb0 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.185+0000 7f213685cb80 1 -- 172.21.15.8:0/2172104035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 msgr2=0x55ca4f80efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.185+0000 7f213685cb80 1 --2- 172.21.15.8:0/2172104035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f80efc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f211009adb0 tx=0x7f211000ff60 comp rx=0 tx=0).stop 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.186+0000 7f213685cb80 1 -- 172.21.15.8:0/2172104035 shutdown_connections 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.186+0000 7f213685cb80 1 --2- 172.21.15.8:0/2172104035 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f8100c0 0x55ca4f810530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.186+0000 7f213685cb80 1 --2- 172.21.15.8:0/2172104035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f80efc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.186+0000 7f213685cb80 1 --2- 172.21.15.8:0/2172104035 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80f500 0x55ca4f80fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.186+0000 7f213685cb80 1 -- 172.21.15.8:0/2172104035 >> 172.21.15.8:0/2172104035 conn(0x55ca4f804310 msgr2=0x55ca4f809620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.186+0000 7f213685cb80 1 -- 172.21.15.8:0/2172104035 shutdown_connections 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.186+0000 7f213685cb80 1 -- 172.21.15.8:0/2172104035 wait complete. 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.186+0000 7f213685cb80 1 -- start start 2023-11-27T15:25:34.193 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.186+0000 7f213685cb80 1 Processor -- start 2023-11-27T15:25:34.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f213685cb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f818270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.197 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f213685cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 0x55ca4f8187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f213685cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f8100c0 0x55ca4f81f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f213685cb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f701890 con 0x55ca4f80ebb0 2023-11-27T15:25:34.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f213685cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f701530 con 0x55ca4f80f500 2023-11-27T15:25:34.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f213685cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f5e38d0 con 0x55ca4f8100c0 2023-11-27T15:25:34.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f213685cb80 10 client.0 _ll_register_callbacks cb 0x55ca4f71f730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:34.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f2120d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f818270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f2120d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f818270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39212/0 (socket says 172.21.15.8:39212) 2023-11-27T15:25:34.198 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f2120d9b700 1 -- 172.21.15.8:0/3642338617 learned_addr learned my addr 172.21.15.8:0/3642338617 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f211bfff700 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 0x55ca4f8187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f212159c700 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f8100c0 0x55ca4f81f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f2120d9b700 1 -- 172.21.15.8:0/3642338617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f8100c0 msgr2=0x55ca4f81f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.187+0000 7f2120d9b700 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f8100c0 0x55ca4f81f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f2120d9b700 1 -- 172.21.15.8:0/3642338617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 msgr2=0x55ca4f8187b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f2120d9b700 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 0x55ca4f8187b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f2120d9b700 1 -- 172.21.15.8:0/3642338617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c10ba50 con 0x55ca4f80ebb0 2023-11-27T15:25:34.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f2120d9b700 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f818270 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2110097b20 tx=0x7f211009ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df34f6f89fb4446c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f20fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:25:34.199 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f20fbfff700 1 -- 172.21.15.8:0/3642338617 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21100a31e0 con 0x55ca4f80ebb0 2023-11-27T15:25:34.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f20fbfff700 1 -- 172.21.15.8:0/3642338617 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f21100a3470 con 0x55ca4f80ebb0 2023-11-27T15:25:34.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f213685cb80 1 -- 172.21.15.8:0/3642338617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca4f801ad0 con 0x55ca4f80ebb0 2023-11-27T15:25:34.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f20fbfff700 1 -- 172.21.15.8:0/3642338617 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2110004810 con 0x55ca4f80ebb0 2023-11-27T15:25:34.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f213685cb80 10 client.15297 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:34.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f213685cb80 1 -- 172.21.15.8:0/3642338617 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ca4f726140 con 0x55ca4f80ebb0 2023-11-27T15:25:34.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.188+0000 7f20fbfff700 1 -- 172.21.15.8:0/3642338617 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca4f801ad0 con 0x55ca4f80ebb0 2023-11-27T15:25:34.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.189+0000 7f20fbfff700 1 -- 172.21.15.8:0/3642338617 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f21100aed20 con 0x55ca4f80ebb0 2023-11-27T15:25:34.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.189+0000 7f213685cb80 10 client.15297 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:34.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.189+0000 7f213685cb80 10 client.15297 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:25:34.200 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.189+0000 7f213685cb80 20 client.15297 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.189+0000 7f213685cb80 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55ca4f8235f0 0x55ca4f822720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.189+0000 7f213685cb80 4 client.15297 mds_command: new command op to 24539 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.189+0000 7f213685cb80 1 -- 172.21.15.8:0/3642338617 --> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca4f722050 con 0x55ca4f8235f0 2023-11-27T15:25:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.189+0000 7f211bfff700 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55ca4f8235f0 0x55ca4f822720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.190+0000 7f211bfff700 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55ca4f8235f0 0x55ca4f822720 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff4835cb3e6a8751 server_cookie=7d717b6054f1d450 in_seq=0 out_seq=0 2023-11-27T15:25:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.190+0000 7f20fbfff700 10 client.15297 ms_handle_connect on v2:172.21.15.29:6836/1651468308 2023-11-27T15:25:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.190+0000 7f20fbfff700 1 -- 172.21.15.8:0/3642338617 <== mds.2 v2:172.21.15.29:6836/1651468308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca4f722050 con 0x55ca4f8235f0 2023-11-27T15:25:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.190+0000 7f20fbfff700 10 client.15297 handle_command_reply: tid=0 2023-11-27T15:25:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.190+0000 7f213685cb80 1 client.15297 shutdown 2023-11-27T15:25:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 1 -- 172.21.15.8:0/3642338617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 msgr2=0x55ca4f818270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f818270 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2110097b20 tx=0x7f211009ab40 comp rx=0 tx=0).stop 2023-11-27T15:25:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 1 -- 172.21.15.8:0/3642338617 shutdown_connections 2023-11-27T15:25:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55ca4f8235f0 0x55ca4f822720 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f8100c0 0x55ca4f81f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f818270 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 1 --2- 172.21.15.8:0/3642338617 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 0x55ca4f8187b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f20fbfff700 0 client.15297 ms_handle_reset on v2:172.21.15.29:6836/1651468308 2023-11-27T15:25:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 1 -- 172.21.15.8:0/3642338617 >> 172.21.15.8:0/3642338617 conn(0x55ca4f804310 msgr2=0x55ca4f808d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 1 -- 172.21.15.8:0/3642338617 shutdown_connections 2023-11-27T15:25:34.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 1 -- 172.21.15.8:0/3642338617 wait complete. 2023-11-27T15:25:34.203 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.191+0000 7f213685cb80 20 client.15297 trim_cache size 0 max 16384 2023-11-27T15:25:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.210+0000 7f213685cb80 1 Processor -- start 2023-11-27T15:25:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.210+0000 7f213685cb80 1 -- start start 2023-11-27T15:25:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.210+0000 7f213685cb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f82bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.210+0000 7f213685cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 0x55ca4f81f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.210+0000 7f213685cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f8100c0 0x55ca4f822720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.210+0000 7f213685cb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f81fcc0 con 0x55ca4f80ebb0 2023-11-27T15:25:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.210+0000 7f213685cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f80f8d0 con 0x55ca4f80f500 2023-11-27T15:25:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.210+0000 7f213685cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f722050 con 0x55ca4f8100c0 2023-11-27T15:25:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f2120d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f82bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f211bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 0x55ca4f81f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f211bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 0x55ca4f81f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:46876/0 (socket says 172.21.15.8:46876) 2023-11-27T15:25:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f211bfff700 1 -- 172.21.15.8:0/1472408233 learned_addr learned my addr 172.21.15.8:0/1472408233 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f212159c700 1 --2- 172.21.15.8:0/1472408233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f8100c0 0x55ca4f822720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f211bfff700 1 -- 172.21.15.8:0/1472408233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f8100c0 msgr2=0x55ca4f822720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f211bfff700 1 --2- 172.21.15.8:0/1472408233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f8100c0 0x55ca4f822720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f211bfff700 1 -- 172.21.15.8:0/1472408233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 msgr2=0x55ca4f82bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f211bfff700 1 --2- 172.21.15.8:0/1472408233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f82bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f211bfff700 1 -- 172.21.15.8:0/1472408233 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca4f725e90 con 0x55ca4f80f500 2023-11-27T15:25:34.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f2120d9b700 1 --2- 172.21.15.8:0/1472408233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f82bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:34.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f211bfff700 1 --2- 172.21.15.8:0/1472408233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 0x55ca4f81f780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55ca4f824200 tx=0x7f210801dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85097cc24f97aa15 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.211+0000 7f21077fe700 1 -- 172.21.15.8:0/1472408233 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108018040 con 0x55ca4f80f500 2023-11-27T15:25:34.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f21077fe700 1 -- 172.21.15.8:0/1472408233 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21080044f0 con 0x55ca4f80f500 2023-11-27T15:25:34.217 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f21077fe700 1 -- 172.21.15.8:0/1472408233 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108003aa0 con 0x55ca4f80f500 2023-11-27T15:25:34.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f213685cb80 1 -- 172.21.15.8:0/1472408233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 msgr2=0x55ca4f81f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f213685cb80 1 --2- 172.21.15.8:0/1472408233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 0x55ca4f81f780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55ca4f824200 tx=0x7f210801dc30 comp rx=0 tx=0).stop 2023-11-27T15:25:34.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f213685cb80 1 -- 172.21.15.8:0/1472408233 shutdown_connections 2023-11-27T15:25:34.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f213685cb80 1 --2- 172.21.15.8:0/1472408233 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f8100c0 0x55ca4f822720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f213685cb80 1 --2- 172.21.15.8:0/1472408233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80ebb0 0x55ca4f82bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f213685cb80 1 --2- 172.21.15.8:0/1472408233 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f80f500 0x55ca4f81f780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f213685cb80 1 -- 172.21.15.8:0/1472408233 >> 172.21.15.8:0/1472408233 conn(0x55ca4f804310 msgr2=0x55ca4f8095f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f213685cb80 1 -- 172.21.15.8:0/1472408233 shutdown_connections 2023-11-27T15:25:34.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.212+0000 7f213685cb80 1 -- 172.21.15.8:0/1472408233 wait complete. 2023-11-27T15:25:34.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.213+0000 7f213685cb80 1 -- start start 2023-11-27T15:25:34.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.213+0000 7f213685cb80 1 Processor -- start 2023-11-27T15:25:34.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.213+0000 7f213685cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80ebb0 0x55ca4f814a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.213+0000 7f213685cb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80f500 0x55ca4f814fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.213+0000 7f213685cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f8100c0 0x55ca4f876e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.213+0000 7f213685cb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f81f810 con 0x55ca4f80f500 2023-11-27T15:25:34.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.213+0000 7f213685cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f81f980 con 0x55ca4f8100c0 2023-11-27T15:25:34.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.213+0000 7f213685cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca4f81fb40 con 0x55ca4f80ebb0 2023-11-27T15:25:34.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.213+0000 7f213685cb80 10 client.0 _ll_register_callbacks cb 0x7f211009abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:34.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f2120d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80ebb0 0x55ca4f814a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f2120d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80ebb0 0x55ca4f814a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:59504/0 (socket says 172.21.15.8:59504) 2023-11-27T15:25:34.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f211bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80f500 0x55ca4f814fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f2120d9b700 1 -- 172.21.15.8:0/1546424559 learned_addr learned my addr 172.21.15.8:0/1546424559 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f212159c700 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f8100c0 0x55ca4f876e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f2120d9b700 1 -- 172.21.15.8:0/1546424559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f8100c0 msgr2=0x55ca4f876e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f2120d9b700 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f8100c0 0x55ca4f876e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f2120d9b700 1 -- 172.21.15.8:0/1546424559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80f500 msgr2=0x55ca4f814fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f2120d9b700 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80f500 0x55ca4f814fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f2120d9b700 1 -- 172.21.15.8:0/1546424559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca4f726140 con 0x55ca4f80ebb0 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f2120d9b700 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80ebb0 0x55ca4f814a80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2110000c00 tx=0x7f2110097520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2eeed5e9c8b5142 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f20fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f20fbfff700 1 -- 172.21.15.8:0/1546424559 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211009a5a0 con 0x55ca4f80ebb0 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f20fbfff700 1 -- 172.21.15.8:0/1546424559 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f21100a3470 con 0x55ca4f80ebb0 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.214+0000 7f213685cb80 1 -- 172.21.15.8:0/1546424559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2110005690 con 0x55ca4f80ebb0 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f20fbfff700 1 -- 172.21.15.8:0/1546424559 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2110099510 con 0x55ca4f80ebb0 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f213685cb80 10 client.25348 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f213685cb80 1 -- 172.21.15.8:0/1546424559 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f2108009af0 con 0x55ca4f80ebb0 2023-11-27T15:25:34.225 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f20fbfff700 1 -- 172.21.15.8:0/1546424559 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2110005690 con 0x55ca4f80ebb0 2023-11-27T15:25:34.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f20fbfff700 1 -- 172.21.15.8:0/1546424559 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f211009b310 con 0x55ca4f80ebb0 2023-11-27T15:25:34.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f213685cb80 10 client.25348 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:34.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f213685cb80 10 client.25348 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:25:34.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f213685cb80 20 client.25348 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:34.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f213685cb80 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55ca4f87bb10 0x55ca4f878460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f213685cb80 4 client.25348 mds_command: new command op to 24539 tid=0[{"prefix": "status"}] 2023-11-27T15:25:34.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.215+0000 7f213685cb80 1 -- 172.21.15.8:0/1546424559 --> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ca4f72e610 con 0x55ca4f87bb10 2023-11-27T15:25:34.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.216+0000 7f211bfff700 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55ca4f87bb10 0x55ca4f878460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.226 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.216+0000 7f211bfff700 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55ca4f87bb10 0x55ca4f878460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4c8aaa151546bc1 server_cookie=c6a63912fb4494bf in_seq=0 out_seq=0 2023-11-27T15:25:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.216+0000 7f20fbfff700 10 client.25348 ms_handle_connect on v2:172.21.15.29:6836/1651468308 2023-11-27T15:25:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.216+0000 7f20fbfff700 1 -- 172.21.15.8:0/1546424559 <== mds.2 v2:172.21.15.29:6836/1651468308 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ca4f72e610 con 0x55ca4f87bb10 2023-11-27T15:25:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f20fbfff700 10 client.25348 handle_command_reply: tid=0 2023-11-27T15:25:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 client.25348 shutdown 2023-11-27T15:25:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 -- 172.21.15.8:0/1546424559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80ebb0 msgr2=0x55ca4f814a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80ebb0 0x55ca4f814a80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2110000c00 tx=0x7f2110097520 comp rx=0 tx=0).stop 2023-11-27T15:25:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 -- 172.21.15.8:0/1546424559 shutdown_connections 2023-11-27T15:25:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.29:6836/1651468308,v1:172.21.15.29:6837/1651468308] conn(0x55ca4f87bb10 0x55ca4f878460 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ca4f80ebb0 0x55ca4f814a80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ca4f8100c0 0x55ca4f876e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 --2- 172.21.15.8:0/1546424559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ca4f80f500 0x55ca4f814fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f20fbfff700 0 client.25348 ms_handle_reset on v2:172.21.15.29:6836/1651468308 2023-11-27T15:25:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 -- 172.21.15.8:0/1546424559 >> 172.21.15.8:0/1546424559 conn(0x55ca4f804310 msgr2=0x55ca4f822720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 -- 172.21.15.8:0/1546424559 shutdown_connections 2023-11-27T15:25:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 1 -- 172.21.15.8:0/1546424559 wait complete. 2023-11-27T15:25:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.217+0000 7f213685cb80 20 client.25348 trim_cache size 0 max 16384 2023-11-27T15:25:34.228 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:25:34.229 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:25:34.229 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 2, 2023-11-27T15:25:34.229 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24539, 2023-11-27T15:25:34.229 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2023-11-27T15:25:34.229 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2023-11-27T15:25:34.229 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2023-11-27T15:25:34.229 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 95.979676253999997, 2023-11-27T15:25:34.229 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 24, 2023-11-27T15:25:34.229 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 90, 2023-11-27T15:25:34.230 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 88, 2023-11-27T15:25:34.230 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 145.406995502 2023-11-27T15:25:34.230 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:25:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 -- 172.21.15.8:0/2073184279 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f210c0a1150 msgr2=0x7f210c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f210c0a1150 0x7f210c0a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f21140093c0 tx=0x7f2114009920 comp rx=0 tx=0).stop 2023-11-27T15:25:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 -- 172.21.15.8:0/2073184279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c100230 msgr2=0x7f211c198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c100230 0x7f211c198210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2110004730 tx=0x7f2110009510 comp rx=0 tx=0).stop 2023-11-27T15:25:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 -- 172.21.15.8:0/2073184279 shutdown_connections 2023-11-27T15:25:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f211c101db0 0x7f211c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f211c100230 0x7f211c198210 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f210c0a1150 0x7f210c0a3600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 --2- 172.21.15.8:0/2073184279 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f211c101420 0x7f211c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 -- 172.21.15.8:0/2073184279 >> 172.21.15.8:0/2073184279 conn(0x7f211c0fb810 msgr2=0x7f211c106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 -- 172.21.15.8:0/2073184279 shutdown_connections 2023-11-27T15:25:34.232 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.218+0000 7f2122fff700 1 -- 172.21.15.8:0/2073184279 wait complete. 2023-11-27T15:25:34.240 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-27T15:25:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.446+0000 7feeeefff700 1 -- 172.21.15.8:0/3697619212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feee8100230 msgr2=0x7feee8100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.446+0000 7feeeefff700 1 --2- 172.21.15.8:0/3697619212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feee8100230 0x7feee8100640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feedc009990 tx=0x7feedc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.446+0000 7feeeefff700 1 -- 172.21.15.8:0/3697619212 shutdown_connections 2023-11-27T15:25:34.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.446+0000 7feeeefff700 1 --2- 172.21.15.8:0/3697619212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feee8101db0 0x7feee8108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.446+0000 7feeeefff700 1 --2- 172.21.15.8:0/3697619212 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feee8100230 0x7feee8100640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.446+0000 7feeeefff700 1 --2- 172.21.15.8:0/3697619212 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8101420 0x7feee8101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.446+0000 7feeeefff700 1 -- 172.21.15.8:0/3697619212 >> 172.21.15.8:0/3697619212 conn(0x7feee80fb810 msgr2=0x7feee80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.446+0000 7feeeefff700 1 -- 172.21.15.8:0/3697619212 shutdown_connections 2023-11-27T15:25:34.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.446+0000 7feeeefff700 1 -- 172.21.15.8:0/3697619212 wait complete. 2023-11-27T15:25:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.447+0000 7feeeefff700 1 Processor -- start 2023-11-27T15:25:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.447+0000 7feeeefff700 1 -- start start 2023-11-27T15:25:34.453 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feeeefff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8100230 0x7feee8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feeeefff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feee8101420 0x7feee8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feeeefff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feee8101db0 0x7feee819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feeeefff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee819fd80 con 0x7feee8101db0 2023-11-27T15:25:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feeeefff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee819fec0 con 0x7feee8101420 2023-11-27T15:25:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feeeefff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee81a0050 con 0x7feee8100230 2023-11-27T15:25:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feeecd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8100230 0x7feee8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feeecd9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8100230 0x7feee8198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:59528/0 (socket says 172.21.15.8:59528) 2023-11-27T15:25:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feeecd9b700 1 -- 172.21.15.8:0/1655686080 learned_addr learned my addr 172.21.15.8:0/1655686080 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feee7fff700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feee8101420 0x7feee8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feeed59c700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feee8101db0 0x7feee819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feee7fff700 1 -- 172.21.15.8:0/1655686080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8100230 msgr2=0x7feee8198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.448+0000 7feee7fff700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8100230 0x7feee8198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.449+0000 7feee7fff700 1 -- 172.21.15.8:0/1655686080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feee8101db0 msgr2=0x7feee819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.449+0000 7feee7fff700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feee8101db0 0x7feee819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.449+0000 7feee7fff700 1 -- 172.21.15.8:0/1655686080 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc0095f0 con 0x7feee8101420 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.449+0000 7feeecd9b700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8100230 0x7feee8198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.449+0000 7feee7fff700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feee8101420 0x7feee8198740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feed400e9f0 tx=0x7feed400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.449+0000 7feee5ffb700 1 -- 172.21.15.8:0/1655686080 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed400c9c0 con 0x7feee8101420 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.449+0000 7feee5ffb700 1 -- 172.21.15.8:0/1655686080 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feed4010430 con 0x7feee8101420 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.449+0000 7feeeefff700 1 -- 172.21.15.8:0/1655686080 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee810bbe0 con 0x7feee8101420 2023-11-27T15:25:34.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.449+0000 7feee5ffb700 1 -- 172.21.15.8:0/1655686080 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4018680 con 0x7feee8101420 2023-11-27T15:25:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.449+0000 7feeeefff700 1 -- 172.21.15.8:0/1655686080 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee810ba50 con 0x7feee8101420 2023-11-27T15:25:34.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.452+0000 7feee5ffb700 1 -- 172.21.15.8:0/1655686080 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7feed4010860 con 0x7feee8101420 2023-11-27T15:25:34.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.453+0000 7feee5ffb700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feed80a1150 0x7feed80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.454+0000 7feee5ffb700 1 -- 172.21.15.8:0/1655686080 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feed4014070 con 0x7feee8101420 2023-11-27T15:25:34.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.454+0000 7fef028f5b80 1 Processor -- start 2023-11-27T15:25:34.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.454+0000 7feeecd9b700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feed80a1150 0x7feed80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.454+0000 7fef028f5b80 1 -- start start 2023-11-27T15:25:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.454+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df53fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.454+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df54500 0x562d3df54ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.454+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df550c0 0x562d3df55530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.454+0000 7fef028f5b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3de59010 con 0x562d3df54500 2023-11-27T15:25:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.454+0000 7fef028f5b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3deab850 con 0x562d3df53bb0 2023-11-27T15:25:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.454+0000 7fef028f5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3dea4ce0 con 0x562d3df550c0 2023-11-27T15:25:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feeecd9b700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feed80a1150 0x7feed80a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feedc00b2e0 tx=0x7feedc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feee7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df54500 0x562d3df54ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.461 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feeecd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df53fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feee7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df54500 0x562d3df54ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39272/0 (socket says 172.21.15.8:39272) 2023-11-27T15:25:34.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feee7fff700 1 -- 172.21.15.8:0/683363138 learned_addr learned my addr 172.21.15.8:0/683363138 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feeecd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df53fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:46904/0 (socket says 172.21.15.8:46904) 2023-11-27T15:25:34.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feeed59c700 1 --2- 172.21.15.8:0/683363138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df550c0 0x562d3df55530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feee7fff700 1 -- 172.21.15.8:0/683363138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df550c0 msgr2=0x562d3df55530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feee7fff700 1 --2- 172.21.15.8:0/683363138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df550c0 0x562d3df55530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feee7fff700 1 -- 172.21.15.8:0/683363138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 msgr2=0x562d3df53fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feee7fff700 1 --2- 172.21.15.8:0/683363138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df53fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feee7fff700 1 -- 172.21.15.8:0/683363138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d3dd265e0 con 0x562d3df54500 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feeecd9b700 1 --2- 172.21.15.8:0/683363138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df53fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.455+0000 7feee7fff700 1 --2- 172.21.15.8:0/683363138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df54500 0x562d3df54ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7feed4099e10 tx=0x7feed4099860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db34e90e70913b08 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.456+0000 7feed2ffd700 1 -- 172.21.15.8:0/683363138 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed409deb0 con 0x562d3df54500 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.456+0000 7feed2ffd700 1 -- 172.21.15.8:0/683363138 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feed40a22d0 con 0x562d3df54500 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.456+0000 7feed2ffd700 1 -- 172.21.15.8:0/683363138 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40a1e40 con 0x562d3df54500 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.456+0000 7fef028f5b80 1 -- 172.21.15.8:0/683363138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df54500 msgr2=0x562d3df54ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.456+0000 7fef028f5b80 1 --2- 172.21.15.8:0/683363138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df54500 0x562d3df54ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7feed4099e10 tx=0x7feed4099860 comp rx=0 tx=0).stop 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.457+0000 7fef028f5b80 1 -- 172.21.15.8:0/683363138 shutdown_connections 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.457+0000 7fef028f5b80 1 --2- 172.21.15.8:0/683363138 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df550c0 0x562d3df55530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.457+0000 7fef028f5b80 1 --2- 172.21.15.8:0/683363138 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df53fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.463 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.457+0000 7fef028f5b80 1 --2- 172.21.15.8:0/683363138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df54500 0x562d3df54ae0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.457+0000 7fef028f5b80 1 -- 172.21.15.8:0/683363138 >> 172.21.15.8:0/683363138 conn(0x562d3df49310 msgr2=0x562d3df4e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.457+0000 7fef028f5b80 1 -- 172.21.15.8:0/683363138 shutdown_connections 2023-11-27T15:25:34.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.457+0000 7fef028f5b80 1 -- 172.21.15.8:0/683363138 wait complete. 2023-11-27T15:25:34.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.457+0000 7fef028f5b80 1 -- start start 2023-11-27T15:25:34.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.457+0000 7fef028f5b80 1 Processor -- start 2023-11-27T15:25:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df5d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df54500 0x562d3df5dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3df64b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7fef028f5b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3de46890 con 0x562d3df550c0 2023-11-27T15:25:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7fef028f5b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3de46530 con 0x562d3df53bb0 2023-11-27T15:25:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7fef028f5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3dd288d0 con 0x562d3df54500 2023-11-27T15:25:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7fef028f5b80 10 client.0 _ll_register_callbacks cb 0x562d3de64730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7feeed59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3df64b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7feeecd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df5d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.468 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7feee7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df54500 0x562d3df5dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7feeed59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3df64b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39286/0 (socket says 172.21.15.8:39286) 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7feeecd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df5d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:46910/0 (socket says 172.21.15.8:46910) 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7feeed59c700 1 -- 172.21.15.8:0/548890112 learned_addr learned my addr 172.21.15.8:0/548890112 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.458+0000 7feeecd9b700 1 -- 172.21.15.8:0/548890112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df54500 msgr2=0x562d3df5dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7feeecd9b700 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df54500 0x562d3df5dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7feeecd9b700 1 -- 172.21.15.8:0/548890112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 msgr2=0x562d3df64b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7feeecd9b700 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3df64b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7feeecd9b700 1 -- 172.21.15.8:0/548890112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee810ba50 con 0x562d3df53bb0 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7feeed59c700 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3df64b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7feeecd9b700 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df5d590 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feedc000c00 tx=0x7feedc01a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e028f55941672ecb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7feec7fff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7feec7fff700 1 -- 172.21.15.8:0/548890112 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc028030 con 0x562d3df53bb0 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7fef028f5b80 1 -- 172.21.15.8:0/548890112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d3df46ad0 con 0x562d3df53bb0 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7feec7fff700 1 -- 172.21.15.8:0/548890112 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feedc0199f0 con 0x562d3df53bb0 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7feec7fff700 1 -- 172.21.15.8:0/548890112 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feedc019bb0 con 0x562d3df53bb0 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7fef028f5b80 10 client.25340 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.459+0000 7fef028f5b80 1 -- 172.21.15.8:0/548890112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562d3de6b140 con 0x562d3df53bb0 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.460+0000 7feec7fff700 1 -- 172.21.15.8:0/548890112 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562d3df46ad0 con 0x562d3df53bb0 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.460+0000 7feec7fff700 1 -- 172.21.15.8:0/548890112 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feedc02fbd0 con 0x562d3df53bb0 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.460+0000 7fef028f5b80 10 client.25340 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.460+0000 7fef028f5b80 10 client.25340 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:25:34.470 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.460+0000 7fef028f5b80 20 client.25340 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.460+0000 7fef028f5b80 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x562d3df68860 0x562d3df67c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.460+0000 7fef028f5b80 4 client.25340 mds_command: new command op to 24593 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.460+0000 7fef028f5b80 1 -- 172.21.15.8:0/548890112 --> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d3de67050 con 0x562d3df68860 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.461+0000 7feee7fff700 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x562d3df68860 0x562d3df67c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.461+0000 7feee7fff700 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x562d3df68860 0x562d3df67c60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9845025b5429e8af server_cookie=13f90e603859df89 in_seq=0 out_seq=0 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.461+0000 7feec7fff700 10 client.25340 ms_handle_connect on v2:172.21.15.29:6838/2619830143 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7feec7fff700 1 -- 172.21.15.8:0/548890112 <== mds.1 v2:172.21.15.29:6838/2619830143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562d3de67050 con 0x562d3df68860 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7feec7fff700 10 client.25340 handle_command_reply: tid=0 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 client.25340 shutdown 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 -- 172.21.15.8:0/548890112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 msgr2=0x562d3df5d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.471 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df5d590 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feedc000c00 tx=0x7feedc01a050 comp rx=0 tx=0).stop 2023-11-27T15:25:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 -- 172.21.15.8:0/548890112 shutdown_connections 2023-11-27T15:25:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x562d3df68860 0x562d3df67c60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3df64b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df53bb0 0x562d3df5d590 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7feec7fff700 0 client.25340 ms_handle_reset on v2:172.21.15.29:6838/2619830143 2023-11-27T15:25:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 --2- 172.21.15.8:0/548890112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df54500 0x562d3df5dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 -- 172.21.15.8:0/548890112 >> 172.21.15.8:0/548890112 conn(0x562d3df49310 msgr2=0x562d3df4dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 -- 172.21.15.8:0/548890112 shutdown_connections 2023-11-27T15:25:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.462+0000 7fef028f5b80 1 -- 172.21.15.8:0/548890112 wait complete. 2023-11-27T15:25:34.472 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.463+0000 7fef028f5b80 20 client.25340 trim_cache size 0 max 16384 2023-11-27T15:25:34.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.481+0000 7fef028f5b80 1 Processor -- start 2023-11-27T15:25:34.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.481+0000 7fef028f5b80 1 -- start start 2023-11-27T15:25:34.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.481+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df53bb0 0x562d3df67c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.481+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df54500 0x562d3df681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.481+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df550c0 0x562d3df64ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.481+0000 7fef028f5b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3df66800 con 0x562d3df53bb0 2023-11-27T15:25:34.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.481+0000 7fef028f5b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3ddb0ed0 con 0x562d3df550c0 2023-11-27T15:25:34.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.481+0000 7fef028f5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3de62fa0 con 0x562d3df54500 2023-11-27T15:25:34.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeecd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df53bb0 0x562d3df67c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeed59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df550c0 0x562d3df64ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeed59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df550c0 0x562d3df64ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:46918/0 (socket says 172.21.15.8:46918) 2023-11-27T15:25:34.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeed59c700 1 -- 172.21.15.8:0/797521002 learned_addr learned my addr 172.21.15.8:0/797521002 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeecd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df53bb0 0x562d3df67c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39292/0 (socket says 172.21.15.8:39292) 2023-11-27T15:25:34.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feee7fff700 1 --2- 172.21.15.8:0/797521002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df54500 0x562d3df681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeed59c700 1 -- 172.21.15.8:0/797521002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df54500 msgr2=0x562d3df681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.487 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeed59c700 1 --2- 172.21.15.8:0/797521002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df54500 0x562d3df681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeed59c700 1 -- 172.21.15.8:0/797521002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df53bb0 msgr2=0x562d3df67c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeed59c700 1 --2- 172.21.15.8:0/797521002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df53bb0 0x562d3df67c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeed59c700 1 -- 172.21.15.8:0/797521002 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d3de6ae90 con 0x562d3df550c0 2023-11-27T15:25:34.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeecd9b700 1 --2- 172.21.15.8:0/797521002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df53bb0 0x562d3df67c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:34.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.482+0000 7feeed59c700 1 --2- 172.21.15.8:0/797521002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df550c0 0x562d3df64ad0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feee0011cb0 tx=0x7feee0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a9f264e82c308a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7feed37fe700 1 -- 172.21.15.8:0/797521002 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee000e040 con 0x562d3df550c0 2023-11-27T15:25:34.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7feed37fe700 1 -- 172.21.15.8:0/797521002 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feee000f040 con 0x562d3df550c0 2023-11-27T15:25:34.488 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7feed37fe700 1 -- 172.21.15.8:0/797521002 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0004050 con 0x562d3df550c0 2023-11-27T15:25:34.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7fef028f5b80 1 -- 172.21.15.8:0/797521002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df550c0 msgr2=0x562d3df64ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.492 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7fef028f5b80 1 --2- 172.21.15.8:0/797521002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df550c0 0x562d3df64ad0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feee0011cb0 tx=0x7feee0009920 comp rx=0 tx=0).stop 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7fef028f5b80 1 -- 172.21.15.8:0/797521002 shutdown_connections 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7fef028f5b80 1 --2- 172.21.15.8:0/797521002 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df550c0 0x562d3df64ad0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7fef028f5b80 1 --2- 172.21.15.8:0/797521002 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df53bb0 0x562d3df67c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7fef028f5b80 1 --2- 172.21.15.8:0/797521002 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df54500 0x562d3df681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7fef028f5b80 1 -- 172.21.15.8:0/797521002 >> 172.21.15.8:0/797521002 conn(0x562d3df49310 msgr2=0x562d3df5c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7fef028f5b80 1 -- 172.21.15.8:0/797521002 shutdown_connections 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.483+0000 7fef028f5b80 1 -- 172.21.15.8:0/797521002 wait complete. 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.484+0000 7fef028f5b80 1 -- start start 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.484+0000 7fef028f5b80 1 Processor -- start 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.484+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df53bb0 0x562d3df5f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.484+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df54500 0x562d3df5fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.484+0000 7fef028f5b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3dfbc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.493 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.484+0000 7fef028f5b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3df64cb0 con 0x562d3df550c0 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.484+0000 7fef028f5b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3df64e20 con 0x562d3df54500 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.484+0000 7fef028f5b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d3dfbcc20 con 0x562d3df53bb0 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.484+0000 7fef028f5b80 10 client.0 _ll_register_callbacks cb 0x7feedc00f810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeed59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3dfbc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feee7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df54500 0x562d3df5fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeed59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3dfbc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39308/0 (socket says 172.21.15.8:39308) 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeed59c700 1 -- 172.21.15.8:0/252248848 learned_addr learned my addr 172.21.15.8:0/252248848 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeecd9b700 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df53bb0 0x562d3df5f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeed59c700 1 -- 172.21.15.8:0/252248848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df53bb0 msgr2=0x562d3df5f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeed59c700 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df53bb0 0x562d3df5f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeed59c700 1 -- 172.21.15.8:0/252248848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df54500 msgr2=0x562d3df5fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.494 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeed59c700 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df54500 0x562d3df5fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeed59c700 1 -- 172.21.15.8:0/252248848 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d3de6b140 con 0x562d3df550c0 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeecd9b700 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df53bb0 0x562d3df5f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feeed59c700 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3dfbc6e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feee0011c80 tx=0x7feee0003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd3263b219b47fc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feec7fff700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.485+0000 7feec7fff700 1 -- 172.21.15.8:0/252248848 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee000e050 con 0x562d3df550c0 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7fef028f5b80 1 -- 172.21.15.8:0/252248848 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feed40a22d0 con 0x562d3df550c0 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7feec7fff700 1 -- 172.21.15.8:0/252248848 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feee00093a0 con 0x562d3df550c0 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7feec7fff700 1 -- 172.21.15.8:0/252248848 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feee0004460 con 0x562d3df550c0 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7fef028f5b80 10 client.15321 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7fef028f5b80 1 -- 172.21.15.8:0/252248848 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7feee0011960 con 0x562d3df550c0 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7feec7fff700 1 -- 172.21.15.8:0/252248848 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feed40a22d0 con 0x562d3df550c0 2023-11-27T15:25:34.495 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7feec7fff700 1 -- 172.21.15.8:0/252248848 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feee0007ae0 con 0x562d3df550c0 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7fef028f5b80 10 client.15321 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7fef028f5b80 10 client.15321 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7fef028f5b80 20 client.15321 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7fef028f5b80 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x562d3dfc18e0 0x562d3dfbe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7fef028f5b80 4 client.15321 mds_command: new command op to 24593 tid=0[{"prefix": "status"}] 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.486+0000 7fef028f5b80 1 -- 172.21.15.8:0/252248848 --> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d3de73610 con 0x562d3dfc18e0 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.487+0000 7feeecd9b700 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x562d3dfc18e0 0x562d3dfbe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.487+0000 7feeecd9b700 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x562d3dfc18e0 0x562d3dfbe250 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=883e218c2d22adc3 server_cookie=7a34d00959bac4d8 in_seq=0 out_seq=0 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.487+0000 7feec7fff700 10 client.15321 ms_handle_connect on v2:172.21.15.29:6838/2619830143 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7feec7fff700 1 -- 172.21.15.8:0/252248848 <== mds.1 v2:172.21.15.29:6838/2619830143 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x562d3de73610 con 0x562d3dfc18e0 2023-11-27T15:25:34.496 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7feec7fff700 10 client.15321 handle_command_reply: tid=0 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 client.15321 shutdown 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 -- 172.21.15.8:0/252248848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 msgr2=0x562d3dfbc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3dfbc6e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7feee0011c80 tx=0x7feee0003c30 comp rx=0 tx=0).stop 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 -- 172.21.15.8:0/252248848 shutdown_connections 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.29:6838/2619830143,v1:172.21.15.29:6839/2619830143] conn(0x562d3dfc18e0 0x562d3dfbe250 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x562d3df53bb0 0x562d3df5f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7feec7fff700 0 client.15321 ms_handle_reset on v2:172.21.15.29:6838/2619830143 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x562d3df550c0 0x562d3dfbc6e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 --2- 172.21.15.8:0/252248848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x562d3df54500 0x562d3df5fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 -- 172.21.15.8:0/252248848 >> 172.21.15.8:0/252248848 conn(0x562d3df49310 msgr2=0x562d3df57fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 -- 172.21.15.8:0/252248848 shutdown_connections 2023-11-27T15:25:34.497 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 1 -- 172.21.15.8:0/252248848 wait complete. 2023-11-27T15:25:34.498 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.488+0000 7fef028f5b80 20 client.15321 trim_cache size 0 max 16384 2023-11-27T15:25:34.498 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:25:34.498 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:25:34.498 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 1, 2023-11-27T15:25:34.498 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24593, 2023-11-27T15:25:34.498 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2023-11-27T15:25:34.498 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2023-11-27T15:25:34.498 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2023-11-27T15:25:34.498 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 97.055596382000004, 2023-11-27T15:25:34.499 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 24, 2023-11-27T15:25:34.499 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 90, 2023-11-27T15:25:34.499 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 88, 2023-11-27T15:25:34.499 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 134.146342213 2023-11-27T15:25:34.499 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 -- 172.21.15.8:0/1655686080 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feed80a1150 msgr2=0x7feed80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feed80a1150 0x7feed80a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7feedc00b2e0 tx=0x7feedc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 -- 172.21.15.8:0/1655686080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feee8101420 msgr2=0x7feee8198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feee8101420 0x7feee8198740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feed400e9f0 tx=0x7feed400ed00 comp rx=0 tx=0).stop 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 -- 172.21.15.8:0/1655686080 shutdown_connections 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7feed80a1150 0x7feed80a3600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7feee8100230 0x7feee8198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feee8101db0 0x7feee819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 --2- 172.21.15.8:0/1655686080 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7feee8101420 0x7feee8198740 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 -- 172.21.15.8:0/1655686080 >> 172.21.15.8:0/1655686080 conn(0x7feee80fb810 msgr2=0x7feee80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.500 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 -- 172.21.15.8:0/1655686080 shutdown_connections 2023-11-27T15:25:34.501 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.489+0000 7feeeefff700 1 -- 172.21.15.8:0/1655686080 wait complete. 2023-11-27T15:25:34.508 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:34 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1771110096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:34.508 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:34 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2248810662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:34.510 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-27T15:25:34.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.714+0000 7fca583ff700 1 -- 172.21.15.8:0/4124396749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca50101db0 msgr2=0x7fca50108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.714+0000 7fca583ff700 1 --2- 172.21.15.8:0/4124396749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca50101db0 0x7fca50108670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fca4800b0a0 tx=0x7fca4800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.714+0000 7fca583ff700 1 -- 172.21.15.8:0/4124396749 shutdown_connections 2023-11-27T15:25:34.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.714+0000 7fca583ff700 1 --2- 172.21.15.8:0/4124396749 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca50100230 0x7fca50100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.714+0000 7fca583ff700 1 --2- 172.21.15.8:0/4124396749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca50101db0 0x7fca50108670 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.714+0000 7fca583ff700 1 --2- 172.21.15.8:0/4124396749 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca50101420 0x7fca50101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.718 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.714+0000 7fca583ff700 1 -- 172.21.15.8:0/4124396749 >> 172.21.15.8:0/4124396749 conn(0x7fca500fb810 msgr2=0x7fca500fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.714+0000 7fca583ff700 1 -- 172.21.15.8:0/4124396749 shutdown_connections 2023-11-27T15:25:34.719 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.714+0000 7fca583ff700 1 -- 172.21.15.8:0/4124396749 wait complete. 2023-11-27T15:25:34.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.715+0000 7fca583ff700 1 Processor -- start 2023-11-27T15:25:34.720 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.715+0000 7fca583ff700 1 -- start start 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.715+0000 7fca583ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca50100230 0x7fca500748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.715+0000 7fca583ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca50101420 0x7fca50072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.715+0000 7fca583ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca50101db0 0x7fca50073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.715+0000 7fca583ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca50073bc0 con 0x7fca50101420 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.715+0000 7fca583ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca50073d30 con 0x7fca50100230 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.715+0000 7fca583ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca501a0100 con 0x7fca50101db0 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5619b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca50100230 0x7fca500748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5619b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca50100230 0x7fca500748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:46956/0 (socket says 172.21.15.8:46956) 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5599a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca50101420 0x7fca50072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5619b700 1 -- 172.21.15.8:0/2408352704 learned_addr learned my addr 172.21.15.8:0/2408352704 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.721 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5699c700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca50101db0 0x7fca50073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5619b700 1 -- 172.21.15.8:0/2408352704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca50101db0 msgr2=0x7fca50073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5619b700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca50101db0 0x7fca50073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5619b700 1 -- 172.21.15.8:0/2408352704 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca50101420 msgr2=0x7fca50072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5619b700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca50101420 0x7fca50072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5619b700 1 -- 172.21.15.8:0/2408352704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca48009d00 con 0x7fca50100230 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.716+0000 7fca5619b700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca50100230 0x7fca500748a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00e970 tx=0x7fca4c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.717+0000 7fca477fe700 1 -- 172.21.15.8:0/2408352704 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c009790 con 0x7fca50100230 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.717+0000 7fca477fe700 1 -- 172.21.15.8:0/2408352704 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca4c00c7e0 con 0x7fca50100230 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.717+0000 7fca477fe700 1 -- 172.21.15.8:0/2408352704 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c003c70 con 0x7fca50100230 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.717+0000 7fca583ff700 1 -- 172.21.15.8:0/2408352704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5010bc00 con 0x7fca50100230 2023-11-27T15:25:34.722 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.717+0000 7fca583ff700 1 -- 172.21.15.8:0/2408352704 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5010ba70 con 0x7fca50100230 2023-11-27T15:25:34.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.722+0000 7fca477fe700 1 -- 172.21.15.8:0/2408352704 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fca4c00cc50 con 0x7fca50100230 2023-11-27T15:25:34.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.722+0000 7fca6bdc0b80 1 Processor -- start 2023-11-27T15:25:34.726 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.722+0000 7fca6bdc0b80 1 -- start start 2023-11-27T15:25:34.727 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 0x55723fc3cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 0x55723fc3dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3e0c0 0x55723fc3e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fb42010 con 0x55723fc3d500 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fb94850 con 0x55723fc3e0c0 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fb8dce0 con 0x55723fc3cbb0 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca5699c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 0x55723fc3cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca5699c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 0x55723fc3cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:59594/0 (socket says 172.21.15.8:59594) 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca5699c700 1 -- 172.21.15.8:0/841923572 learned_addr learned my addr 172.21.15.8:0/841923572 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca5619b700 1 --2- 172.21.15.8:0/841923572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 0x55723fc3dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca5599a700 1 --2- 172.21.15.8:0/841923572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3e0c0 0x55723fc3e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca5599a700 1 -- 172.21.15.8:0/841923572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 msgr2=0x55723fc3cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca5599a700 1 --2- 172.21.15.8:0/841923572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 0x55723fc3cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca477fe700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fca3c0a1150 0x7fca3c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca5599a700 1 -- 172.21.15.8:0/841923572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 msgr2=0x55723fc3dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca5599a700 1 --2- 172.21.15.8:0/841923572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 0x55723fc3dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.723+0000 7fca5599a700 1 -- 172.21.15.8:0/841923572 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55723fa0f5e0 con 0x55723fc3e0c0 2023-11-27T15:25:34.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.724+0000 7fca477fe700 1 -- 172.21.15.8:0/2408352704 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fca4c014070 con 0x7fca50100230 2023-11-27T15:25:34.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.724+0000 7fca5599a700 1 --2- 172.21.15.8:0/841923572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3e0c0 0x55723fc3e530 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca40009cb0 tx=0x7fca40011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf30fac4f47eb0f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.724+0000 7fca5699c700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fca3c0a1150 0x7fca3c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.724+0000 7fca44bfe700 1 -- 172.21.15.8:0/841923572 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40011d20 con 0x55723fc3e0c0 2023-11-27T15:25:34.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.724+0000 7fca44bfe700 1 -- 172.21.15.8:0/841923572 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca40011e80 con 0x55723fc3e0c0 2023-11-27T15:25:34.729 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.724+0000 7fca44bfe700 1 -- 172.21.15.8:0/841923572 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40005280 con 0x55723fc3e0c0 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/841923572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3e0c0 msgr2=0x55723fc3e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca5699c700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fca3c0a1150 0x7fca3c0a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fca480062a0 tx=0x7fca48006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/841923572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3e0c0 0x55723fc3e530 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca40009cb0 tx=0x7fca40011440 comp rx=0 tx=0).stop 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/841923572 shutdown_connections 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/841923572 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 0x55723fc3cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/841923572 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3e0c0 0x55723fc3e530 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/841923572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 0x55723fc3dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/841923572 >> 172.21.15.8:0/841923572 conn(0x55723fc32310 msgr2=0x55723fc37620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/841923572 shutdown_connections 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/841923572 wait complete. 2023-11-27T15:25:34.733 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.725+0000 7fca6bdc0b80 1 -- start start 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.726+0000 7fca6bdc0b80 1 Processor -- start 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.726+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3cbb0 0x55723fc46550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.726+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 0x55723fc46a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.726+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 0x55723fc4dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.726+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fb2f890 con 0x55723fc3d500 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.726+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fb2f530 con 0x55723fc3cbb0 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.726+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fa118d0 con 0x55723fc3e0c0 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.726+0000 7fca6bdc0b80 10 client.0 _ll_register_callbacks cb 0x55723fb4d730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.726+0000 7fca5599a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 0x55723fc46a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5599a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 0x55723fc46a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39344/0 (socket says 172.21.15.8:39344) 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5599a700 1 -- 172.21.15.8:0/277128333 learned_addr learned my addr 172.21.15.8:0/277128333 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.734 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5619b700 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3cbb0 0x55723fc46550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5699c700 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 0x55723fc4dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5599a700 1 -- 172.21.15.8:0/277128333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 msgr2=0x55723fc4dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5599a700 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 0x55723fc4dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5599a700 1 -- 172.21.15.8:0/277128333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3cbb0 msgr2=0x55723fc46550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5599a700 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3cbb0 0x55723fc46550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5599a700 1 -- 172.21.15.8:0/277128333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca400098e0 con 0x55723fc3d500 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5699c700 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 0x55723fc4dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca5599a700 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 0x55723fc46a90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca40005c50 tx=0x7fca40011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79edc117e88ade40 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.727+0000 7fca39ffb700 1 -- 172.21.15.8:0/277128333 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400051b0 con 0x55723fc3d500 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.728+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/277128333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55723fb50050 con 0x55723fc3d500 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.728+0000 7fca39ffb700 1 -- 172.21.15.8:0/277128333 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fca40005310 con 0x55723fc3d500 2023-11-27T15:25:34.735 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.728+0000 7fca39ffb700 1 -- 172.21.15.8:0/277128333 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca400054d0 con 0x55723fc3d500 2023-11-27T15:25:34.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.728+0000 7fca6bdc0b80 10 client.15333 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:34.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.728+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/277128333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55723fb54140 con 0x55723fc3d500 2023-11-27T15:25:34.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.728+0000 7fca39ffb700 1 -- 172.21.15.8:0/277128333 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55723fb50050 con 0x55723fc3d500 2023-11-27T15:25:34.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.728+0000 7fca39ffb700 1 -- 172.21.15.8:0/277128333 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca400193f0 con 0x55723fc3d500 2023-11-27T15:25:34.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.728+0000 7fca6bdc0b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:34.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.728+0000 7fca6bdc0b80 10 client.15333 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-27T15:25:34.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.728+0000 7fca6bdc0b80 20 client.15333 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:34.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.729+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55723fc51710 0x55723fc50ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.729+0000 7fca6bdc0b80 4 client.15333 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:34.736 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.729+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/277128333 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55723fb5c610 con 0x55723fc51710 2023-11-27T15:25:34.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.729+0000 7fca5619b700 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55723fc51710 0x55723fc50ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.729+0000 7fca39ffb700 10 client.15333 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:34.737 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.729+0000 7fca5619b700 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55723fc51710 0x55723fc50ac0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63772353b7859ee8 server_cookie=1999f859309b062 in_seq=0 out_seq=0 2023-11-27T15:25:34.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca39ffb700 1 -- 172.21.15.8:0/277128333 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55723fb5c610 con 0x55723fc51710 2023-11-27T15:25:34.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca39ffb700 10 client.15333 handle_command_reply: tid=0 2023-11-27T15:25:34.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 client.15333 shutdown 2023-11-27T15:25:34.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/277128333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 msgr2=0x55723fc46a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 0x55723fc46a90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fca40005c50 tx=0x7fca40011c50 comp rx=0 tx=0).stop 2023-11-27T15:25:34.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/277128333 shutdown_connections 2023-11-27T15:25:34.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55723fc51710 0x55723fc50ac0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.738 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 0x55723fc4dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3cbb0 0x55723fc46550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca39ffb700 0 client.15333 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:34.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/277128333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3d500 0x55723fc46a90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/277128333 >> 172.21.15.8:0/277128333 conn(0x55723fc32310 msgr2=0x55723fc36d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/277128333 shutdown_connections 2023-11-27T15:25:34.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.730+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/277128333 wait complete. 2023-11-27T15:25:34.739 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.731+0000 7fca6bdc0b80 20 client.15333 trim_cache size 0 max 16384 2023-11-27T15:25:34.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.749+0000 7fca6bdc0b80 1 Processor -- start 2023-11-27T15:25:34.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.749+0000 7fca6bdc0b80 1 -- start start 2023-11-27T15:25:34.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.749+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 0x55723fc3e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.749+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 0x55723fc4d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.749+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3e0c0 0x55723fc5a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.749+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fa99ed0 con 0x55723fc3e0c0 2023-11-27T15:25:34.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.749+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fb29e10 con 0x55723fc3d500 2023-11-27T15:25:34.753 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.749+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fb4bfa0 con 0x55723fc3cbb0 2023-11-27T15:25:34.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5699c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3e0c0 0x55723fc5a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5619b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 0x55723fc3e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5699c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3e0c0 0x55723fc5a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39346/0 (socket says 172.21.15.8:39346) 2023-11-27T15:25:34.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5699c700 1 -- 172.21.15.8:0/3546953719 learned_addr learned my addr 172.21.15.8:0/3546953719 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.755 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5599a700 1 --2- 172.21.15.8:0/3546953719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 0x55723fc4d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5619b700 1 -- 172.21.15.8:0/3546953719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 msgr2=0x55723fc4d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5619b700 1 --2- 172.21.15.8:0/3546953719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 0x55723fc4d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5619b700 1 -- 172.21.15.8:0/3546953719 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3e0c0 msgr2=0x55723fc5a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5619b700 1 --2- 172.21.15.8:0/3546953719 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3e0c0 0x55723fc5a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5619b700 1 -- 172.21.15.8:0/3546953719 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55723fb53e90 con 0x55723fc3cbb0 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca5619b700 1 --2- 172.21.15.8:0/3546953719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 0x55723fc3e490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0ca570 tx=0x7fca4c0d0650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81732fb15cb5ac7d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca453ff700 1 -- 172.21.15.8:0/3546953719 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c099810 con 0x55723fc3cbb0 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.750+0000 7fca453ff700 1 -- 172.21.15.8:0/3546953719 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca4c003dd0 con 0x55723fc3cbb0 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.751+0000 7fca453ff700 1 -- 172.21.15.8:0/3546953719 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c098370 con 0x55723fc3cbb0 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.751+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/3546953719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 msgr2=0x55723fc3e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.751+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/3546953719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 0x55723fc3e490 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0ca570 tx=0x7fca4c0d0650 comp rx=0 tx=0).stop 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.751+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/3546953719 shutdown_connections 2023-11-27T15:25:34.756 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.751+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/3546953719 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3cbb0 0x55723fc3e490 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.751+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/3546953719 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3e0c0 0x55723fc5a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.751+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/3546953719 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 0x55723fc4d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.751+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/3546953719 >> 172.21.15.8:0/3546953719 conn(0x55723fc32310 msgr2=0x55723fc375f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.751+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/3546953719 shutdown_connections 2023-11-27T15:25:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.751+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/3546953719 wait complete. 2023-11-27T15:25:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.752+0000 7fca6bdc0b80 1 -- start start 2023-11-27T15:25:34.757 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.752+0000 7fca6bdc0b80 1 Processor -- start 2023-11-27T15:25:34.760 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.752+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3cbb0 0x55723fc48340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.752+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 0x55723fc4d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.752+0000 7fca6bdc0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 0x55723fc48880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.752+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fc48dc0 con 0x55723fc3cbb0 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.752+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fc48f30 con 0x55723fc3d500 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.752+0000 7fca6bdc0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723fca5590 con 0x55723fc3e0c0 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.752+0000 7fca6bdc0b80 10 client.0 _ll_register_callbacks cb 0x7fca4c0d0520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.752+0000 7fca5619b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3cbb0 0x55723fc48340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5619b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3cbb0 0x55723fc48340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39354/0 (socket says 172.21.15.8:39354) 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5619b700 1 -- 172.21.15.8:0/1215864860 learned_addr learned my addr 172.21.15.8:0/1215864860 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5599a700 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 0x55723fc4d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5699c700 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 0x55723fc48880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5619b700 1 -- 172.21.15.8:0/1215864860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 msgr2=0x55723fc48880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.761 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5619b700 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 0x55723fc48880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5619b700 1 -- 172.21.15.8:0/1215864860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 msgr2=0x55723fc4d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5619b700 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 0x55723fc4d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5619b700 1 -- 172.21.15.8:0/1215864860 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5010ba70 con 0x55723fc3cbb0 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5599a700 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 0x55723fc4d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca5619b700 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3cbb0 0x55723fc48340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0a1dd0 tx=0x7fca4c0d03c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1301b14d5c75c9b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca39ffb700 1 -- 172.21.15.8:0/1215864860 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c0b1070 con 0x55723fc3cbb0 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca39ffb700 1 -- 172.21.15.8:0/1215864860 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fca4c09dee0 con 0x55723fc3cbb0 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.753+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/1215864860 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55723fc2fad0 con 0x55723fc3cbb0 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca39ffb700 1 -- 172.21.15.8:0/1215864860 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca4c099eb0 con 0x55723fc3cbb0 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca6bdc0b80 10 client.15339 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/1215864860 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55723f9a4c10 con 0x55723fc3cbb0 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca39ffb700 1 -- 172.21.15.8:0/1215864860 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55723fc2fad0 con 0x55723fc3cbb0 2023-11-27T15:25:34.762 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca39ffb700 1 -- 172.21.15.8:0/1215864860 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca4c0993a0 con 0x55723fc3cbb0 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca6bdc0b80 10 client.15339 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca6bdc0b80 10 client.15339 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca6bdc0b80 20 client.15339 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55723fcaa040 0x55723fca69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca6bdc0b80 4 client.15339 mds_command: new command op to 24595 tid=0[{"prefix": "status"}] 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/1215864860 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55723f9dcf20 con 0x55723fcaa040 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.754+0000 7fca5599a700 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55723fcaa040 0x55723fca69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca5599a700 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55723fcaa040 0x55723fca69f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a84b8ee1505c34 server_cookie=1c65c404c8b9973a in_seq=0 out_seq=0 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca39ffb700 10 client.15339 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca39ffb700 1 -- 172.21.15.8:0/1215864860 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55723f9dcf20 con 0x55723fcaa040 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca39ffb700 10 client.15339 handle_command_reply: tid=0 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 client.15339 shutdown 2023-11-27T15:25:34.763 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/1215864860 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3cbb0 msgr2=0x55723fc48340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3cbb0 0x55723fc48340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0a1dd0 tx=0x7fca4c0d03c0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/1215864860 shutdown_connections 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55723fcaa040 0x55723fca69f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55723fc3e0c0 0x55723fc48880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca39ffb700 0 client.15339 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55723fc3cbb0 0x55723fc48340 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 --2- 172.21.15.8:0/1215864860 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55723fc3d500 0x55723fc4d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/1215864860 >> 172.21.15.8:0/1215864860 conn(0x55723fc32310 msgr2=0x55723fc44460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/1215864860 shutdown_connections 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.755+0000 7fca6bdc0b80 1 -- 172.21.15.8:0/1215864860 wait complete. 2023-11-27T15:25:34.764 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca6bdc0b80 20 client.15339 trim_cache size 0 max 16384 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "cluster_fsid": "9f287ba8-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "whoami": 0, 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 24595, 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "want_state": "up:active", 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "state": "up:active", 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs", 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "rank_uptime": 127.584506264, 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "mdsmap_epoch": 24, 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch": 90, 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_epoch_barrier": 88, 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout: "uptime": 136.748770966 2023-11-27T15:25:34.765 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:25:34.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 -- 172.21.15.8:0/2408352704 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fca3c0a1150 msgr2=0x7fca3c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fca3c0a1150 0x7fca3c0a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fca480062a0 tx=0x7fca48006210 comp rx=0 tx=0).stop 2023-11-27T15:25:34.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 -- 172.21.15.8:0/2408352704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca50100230 msgr2=0x7fca500748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:34.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca50100230 0x7fca500748a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00e970 tx=0x7fca4c00ec80 comp rx=0 tx=0).stop 2023-11-27T15:25:34.766 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 -- 172.21.15.8:0/2408352704 shutdown_connections 2023-11-27T15:25:34.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fca3c0a1150 0x7fca3c0a3600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fca50101db0 0x7fca50073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fca50100230 0x7fca500748a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 --2- 172.21.15.8:0/2408352704 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca50101420 0x7fca50072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:34.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 -- 172.21.15.8:0/2408352704 >> 172.21.15.8:0/2408352704 conn(0x7fca500fb810 msgr2=0x7fca500fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:34.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 -- 172.21.15.8:0/2408352704 shutdown_connections 2023-11-27T15:25:34.767 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:34.756+0000 7fca583ff700 1 -- 172.21.15.8:0/2408352704 wait complete. 2023-11-27T15:25:34.778 INFO:teuthology.run_tasks:Running task kclient... 2023-11-27T15:25:34.791 INFO:tasks.kclient:Mounting kernel clients... 2023-11-27T15:25:34.791 INFO:tasks.kclient:config is {'client.0': {}} 2023-11-27T15:25:34.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:34.792 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2023-11-27T15:25:34.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:34 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1771110096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:34.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:34 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2248810662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:34.834 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-27T15:25:34.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:34.836 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-27T15:25:34.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:34 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1771110096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:34.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:34 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2248810662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:25:34.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:34.945 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link delete ceph-brx 2023-11-27T15:25:35.076 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-11-27T15:25:35.076 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-27T15:25:35.077 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-27T15:25:35.077 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph fs ls 2023-11-27T15:25:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:35 smithi060 ceph-mon[131518]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-27T15:25:35.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:35 smithi008 ceph-mon[128788]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-27T15:25:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:35 smithi029 ceph-mon[131166]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-11-27T15:25:35.948 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:25:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.458+0000 7f0e81f8c700 1 -- 172.21.15.8:0/3794338382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e7c100ee0 msgr2=0x7f0e7c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.458+0000 7f0e81f8c700 1 --2- 172.21.15.8:0/3794338382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e7c100ee0 0x7f0e7c104eb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0e700099c0 tx=0x7f0e70009cd0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.458+0000 7f0e81f8c700 1 -- 172.21.15.8:0/3794338382 shutdown_connections 2023-11-27T15:25:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.458+0000 7f0e81f8c700 1 --2- 172.21.15.8:0/3794338382 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e7c100540 0x7f0e7c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.458+0000 7f0e81f8c700 1 --2- 172.21.15.8:0/3794338382 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e7c1055e0 0x7f0e7c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.458+0000 7f0e81f8c700 1 --2- 172.21.15.8:0/3794338382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e7c100ee0 0x7f0e7c104eb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.458+0000 7f0e81f8c700 1 -- 172.21.15.8:0/3794338382 >> 172.21.15.8:0/3794338382 conn(0x7f0e7c0fbdc0 msgr2=0x7f0e7c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.458+0000 7f0e81f8c700 1 -- 172.21.15.8:0/3794338382 shutdown_connections 2023-11-27T15:25:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.458+0000 7f0e81f8c700 1 -- 172.21.15.8:0/3794338382 wait complete. 2023-11-27T15:25:37.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.459+0000 7f0e81f8c700 1 Processor -- start 2023-11-27T15:25:37.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.459+0000 7f0e81f8c700 1 -- start start 2023-11-27T15:25:37.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.459+0000 7f0e81f8c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e7c100540 0x7f0e7c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:37.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.459+0000 7f0e81f8c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e7c100ee0 0x7f0e7c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:37.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e81f8c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e7c1055e0 0x7f0e7c0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:37.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e81f8c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c1a4840 con 0x7f0e7c1055e0 2023-11-27T15:25:37.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e81f8c700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c1a49b0 con 0x7f0e7c100540 2023-11-27T15:25:37.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e81f8c700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c1a4b70 con 0x7f0e7c100ee0 2023-11-27T15:25:37.464 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e7c1055e0 0x7f0e7c0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e7c1055e0 0x7f0e7c0738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39370/0 (socket says 172.21.15.8:39370) 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7b7fe700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e7c100540 0x7f0e7c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7bfff700 1 -- 172.21.15.8:0/412087472 learned_addr learned my addr 172.21.15.8:0/412087472 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7affd700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e7c100ee0 0x7f0e7c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7b7fe700 1 -- 172.21.15.8:0/412087472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e7c100ee0 msgr2=0x7f0e7c073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7b7fe700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e7c100ee0 0x7f0e7c073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7b7fe700 1 -- 172.21.15.8:0/412087472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e7c1055e0 msgr2=0x7f0e7c0738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7b7fe700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e7c1055e0 0x7f0e7c0738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7b7fe700 1 -- 172.21.15.8:0/412087472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e700095f0 con 0x7f0e7c100540 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.460+0000 7f0e7bfff700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e7c1055e0 0x7f0e7c0738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:37.465 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.461+0000 7f0e7b7fe700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e7c100540 0x7f0e7c072df0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e9f0 tx=0x7f0e6c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:37.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.461+0000 7f0e78ff9700 1 -- 172.21.15.8:0/412087472 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00c9c0 con 0x7f0e7c100540 2023-11-27T15:25:37.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.461+0000 7f0e81f8c700 1 -- 172.21.15.8:0/412087472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c110360 con 0x7f0e7c100540 2023-11-27T15:25:37.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.461+0000 7f0e78ff9700 1 -- 172.21.15.8:0/412087472 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e6c010430 con 0x7f0e7c100540 2023-11-27T15:25:37.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.461+0000 7f0e78ff9700 1 -- 172.21.15.8:0/412087472 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c018680 con 0x7f0e7c100540 2023-11-27T15:25:37.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.461+0000 7f0e81f8c700 1 -- 172.21.15.8:0/412087472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c1101d0 con 0x7f0e7c100540 2023-11-27T15:25:37.466 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.462+0000 7f0e81f8c700 1 -- 172.21.15.8:0/412087472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c1104f0 con 0x7f0e7c100540 2023-11-27T15:25:37.467 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.464+0000 7f0e78ff9700 1 -- 172.21.15.8:0/412087472 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0e6c010860 con 0x7f0e7c100540 2023-11-27T15:25:37.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.465+0000 7f0e78ff9700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0e640a1150 0x7f0e640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:37.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.465+0000 7f0e78ff9700 1 -- 172.21.15.8:0/412087472 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0e6c014070 con 0x7f0e7c100540 2023-11-27T15:25:37.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.466+0000 7f0e7affd700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0e640a1150 0x7f0e640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:37.469 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.466+0000 7f0e7affd700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0e640a1150 0x7f0e640a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009fd0 tx=0x7f0e70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:37.473 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.470+0000 7f0e78ff9700 1 -- 172.21.15.8:0/412087472 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0e6c051730 con 0x7f0e7c100540 2023-11-27T15:25:37.724 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.721+0000 7f0e81f8c700 1 -- 172.21.15.8:0/412087472 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0e7c1a55b0 con 0x7f0e7c100540 2023-11-27T15:25:37.725 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.722+0000 7f0e78ff9700 1 -- 172.21.15.8:0/412087472 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0e6c096160 con 0x7f0e7c100540 2023-11-27T15:25:37.725 INFO:teuthology.orchestra.run.smithi008.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 -- 172.21.15.8:0/412087472 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0e640a1150 msgr2=0x7f0e640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0e640a1150 0x7f0e640a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009fd0 tx=0x7f0e70005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 -- 172.21.15.8:0/412087472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e7c100540 msgr2=0x7f0e7c072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e7c100540 0x7f0e7c072df0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e9f0 tx=0x7f0e6c00ed00 comp rx=0 tx=0).stop 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 -- 172.21.15.8:0/412087472 shutdown_connections 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0e7c1055e0 0x7f0e7c0738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f0e7c100540 0x7f0e7c072df0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f0e640a1150 0x7f0e640a3600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 --2- 172.21.15.8:0/412087472 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f0e7c100ee0 0x7f0e7c073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 -- 172.21.15.8:0/412087472 >> 172.21.15.8:0/412087472 conn(0x7f0e7c0fbdc0 msgr2=0x7f0e7c0fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 -- 172.21.15.8:0/412087472 shutdown_connections 2023-11-27T15:25:37.728 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:25:37.724+0000 7f0e81f8c700 1 -- 172.21.15.8:0/412087472 wait complete. 2023-11-27T15:25:37.750 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:37 smithi008 ceph-mon[128788]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-11-27T15:25:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:37 smithi060 ceph-mon[131518]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-11-27T15:25:37.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:37 smithi029 ceph-mon[131166]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-11-27T15:25:38.138 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-27T15:25:38.138 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-27T15:25:38.138 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi008.front.sepia.ceph.com 2023-11-27T15:25:38.138 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-27T15:25:38.139 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-27T15:25:38.139 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-27T15:25:38.139 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-27T15:25:38.139 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-27T15:25:38.139 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-27T15:25:38.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:38.139 DEBUG:teuthology.orchestra.run.smithi008:> ip addr 2023-11-27T15:25:38.183 INFO:teuthology.orchestra.run.smithi008.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout: inet 127.0.0.1/8 scope host lo 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 ::1/128 scope host 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2e brd ff:ff:ff:ff:ff:ff 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f0 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2f brd ff:ff:ff:ff:ff:ff 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f1 2023-11-27T15:25:38.184 INFO:teuthology.orchestra.run.smithi008.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-27T15:25:38.185 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:fe brd ff:ff:ff:ff:ff:ff 2023-11-27T15:25:38.185 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f0 2023-11-27T15:25:38.185 INFO:teuthology.orchestra.run.smithi008.stdout: inet 172.21.15.8/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-27T15:25:38.185 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft 41448sec preferred_lft 41448sec 2023-11-27T15:25:38.185 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 fe80::ae1f:6bff:fef8:25fe/64 scope link 2023-11-27T15:25:38.185 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:25:38.185 INFO:teuthology.orchestra.run.smithi008.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:25:38.185 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:ff brd ff:ff:ff:ff:ff:ff 2023-11-27T15:25:38.185 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f1 2023-11-27T15:25:38.186 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-27T15:25:38.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:38.186 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-11-27T15:25:38.186 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-11-27T15:25:38.186 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add name ceph-brx type bridge 2023-11-27T15:25:38.186 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr flush dev ceph-brx 2023-11-27T15:25:38.186 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set ceph-brx up 2023-11-27T15:25:38.186 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-27T15:25:38.187 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-11-27T15:25:38.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:38.439 DEBUG:teuthology.orchestra.run.smithi008:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-27T15:25:38.510 INFO:teuthology.orchestra.run.smithi008.stdout:1 2023-11-27T15:25:38.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:38.513 DEBUG:teuthology.orchestra.run.smithi008:> route 2023-11-27T15:25:38.560 INFO:teuthology.orchestra.run.smithi008.stdout:Kernel IP routing table 2023-11-27T15:25:38.560 INFO:teuthology.orchestra.run.smithi008.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-27T15:25:38.560 INFO:teuthology.orchestra.run.smithi008.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-27T15:25:38.560 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-27T15:25:38.560 INFO:teuthology.orchestra.run.smithi008.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-27T15:25:38.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:38.561 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-11-27T15:25:38.561 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-11-27T15:25:38.561 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-27T15:25:38.561 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-27T15:25:38.561 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-27T15:25:38.561 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-11-27T15:25:38.664 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:38 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/412087472' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:25:38.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:38.737 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2023-11-27T15:25:38.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:38.781 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list-id 2023-11-27T15:25:38.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:38 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/412087472' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:25:38.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:38.865 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-11-27T15:25:38.866 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-11-27T15:25:38.866 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-27T15:25:38.866 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-27T15:25:38.866 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-11-27T15:25:38.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:38 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/412087472' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:25:39.050 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-27T15:25:39.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:39.050 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-11-27T15:25:39.050 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-11-27T15:25:39.050 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-27T15:25:39.051 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-27T15:25:39.051 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-27T15:25:39.051 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-27T15:25:39.051 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-27T15:25:39.051 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-11-27T15:25:39.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:39.304 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2023-11-27T15:25:39.304 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2023-11-27T15:25:39.304 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set brx.0 up 2023-11-27T15:25:39.304 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set dev brx.0 master ceph-brx 2023-11-27T15:25:39.304 DEBUG:teuthology.orchestra.run.smithi008:> ') 2023-11-27T15:25:39.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-27T15:25:39.479 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-27T15:25:39.523 INFO:teuthology.orchestra.run.smithi008.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-27T15:25:39.523 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-27T15:25:39.523 DEBUG:teuthology.orchestra.run.smithi008:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-27T15:25:39.606 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-11-27T15:25:39.606 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-11-27T15:25:39.606 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/49db8ba1-39b2-4d18-8e42-2d890bb92a8e 2023-11-27T15:25:39.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:39.606 DEBUG:teuthology.orchestra.run.smithi008:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/49db8ba1-39b2-4d18-8e42-2d890bb92a8e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2023-11-27T15:25:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:39 smithi008 ceph-mon[128788]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:39 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-11-27T15:25:39.724 INFO:teuthology.orchestra.run.smithi008.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2023-11-27T15:25:39.724 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2023-11-27T15:25:39.724 INFO:teuthology.orchestra.run.smithi008.stdout:invalid new device string format 2023-11-27T15:25:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:39 smithi060 ceph-mon[131518]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:39 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-11-27T15:25:39.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:39 smithi029 ceph-mon[131166]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:25:39.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:39 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2023-11-27T15:25:40.240 INFO:teuthology.orchestra.run.smithi008.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2023-11-27T15:25:40.240 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2023-11-27T15:25:40.240 INFO:teuthology.orchestra.run.smithi008.stdout:invalid new device string format 2023-11-27T15:25:40.241 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: resolved to: "172.21.15.8:3300,172.21.15.29:3300,172.21.15.60:3300" 2023-11-27T15:25:40.241 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.8:3300,172.21.15.29:3300,172.21.15.60:3300:/volumes/_nogroup/sv_1/49db8ba1-39b2-4d18-8e42-2d890bb92a8e 2023-11-27T15:25:40.241 INFO:teuthology.orchestra.run.smithi008.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=9f287ba8-8d37-11ee-95a2-87774f69a715" will pass to kernel 2023-11-27T15:25:40.247 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-27T15:25:40.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:40.247 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-27T15:25:40.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:40.319 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-27T15:25:40.406 INFO:teuthology.orchestra.run.smithi008.stdout:client15348 2023-11-27T15:25:40.406 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:25:40.406 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:25:40.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:40.451 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-27T15:25:40.578 INFO:teuthology.orchestra.run.smithi008.stdout:client15348 2023-11-27T15:25:40.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:40.578 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-27T15:25:40.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:40 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-11-27T15:25:40.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:40 smithi008 ceph-mon[128788]: osdmap e91: 12 total, 12 up, 12 in 2023-11-27T15:25:40.664 INFO:teuthology.orchestra.run.smithi008.stdout:client15348 2023-11-27T15:25:40.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:25:40.665 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715.client15348/status') 2023-11-27T15:25:40.814 INFO:teuthology.orchestra.run.smithi008.stdout:instance: client.15348 (3)192.168.0.1:0/3329107675 2023-11-27T15:25:40.814 INFO:teuthology.orchestra.run.smithi008.stdout:blocklisted: false 2023-11-27T15:25:40.814 INFO:teuthology.orchestra.run.smithi008.stderr:0+1 records in 2023-11-27T15:25:40.814 INFO:teuthology.orchestra.run.smithi008.stderr:0+1 records out 2023-11-27T15:25:40.814 INFO:teuthology.orchestra.run.smithi008.stderr:70 bytes copied, 5.8374e-05 s, 1.2 MB/s 2023-11-27T15:25:40.817 INFO:teuthology.run_tasks:Running task check-counter... 2023-11-27T15:25:40.828 INFO:teuthology.run_tasks:Running task background_exec... 2023-11-27T15:25:40.837 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi008.front.sepia.ceph.com 2023-11-27T15:25:40.837 DEBUG:teuthology.orchestra.run.smithi008:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-11-27T15:25:40.838 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:40 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-11-27T15:25:40.838 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:40 smithi060 ceph-mon[131518]: osdmap e91: 12 total, 12 up, 12 in 2023-11-27T15:25:40.842 INFO:teuthology.run_tasks:Running task workunit... 2023-11-27T15:25:40.854 INFO:tasks.workunit:Pulling workunits from ref 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-27T15:25:40.854 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-11-27T15:25:40.854 INFO:tasks.workunit:timeout=3h 2023-11-27T15:25:40.854 INFO:tasks.workunit:cleanup=True 2023-11-27T15:25:40.854 DEBUG:teuthology.orchestra.run.smithi008:> stat -- /home/ubuntu/cephtest/mnt.0 2023-11-27T15:25:40.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:40 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2023-11-27T15:25:40.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:40 smithi029 ceph-mon[131166]: osdmap e91: 12 total, 12 up, 12 in 2023-11-27T15:25:40.902 INFO:teuthology.orchestra.run.smithi008.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-11-27T15:25:40.902 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-11-27T15:25:40.902 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-11-27T15:25:40.902 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:25:40.902 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:cephfs_t:s0 2023-11-27T15:25:40.902 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2023-11-27 15:24:00.758034960 +0000 2023-11-27T15:25:40.902 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2023-11-27 15:24:00.758034960 +0000 2023-11-27T15:25:40.902 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2023-11-27 15:25:40.311636438 +0000 2023-11-27T15:25:40.902 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: 2023-11-27 15:24:00.758034960 +0000 2023-11-27T15:25:40.902 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-11-27T15:25:40.903 DEBUG:teuthology.orchestra.run.smithi008:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-11-27T15:25:40.986 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/chrisphoffman/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-27T15:25:41.033 INFO:tasks.workunit.client.0.smithi008.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-11-27T15:25:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:41 smithi060 ceph-mon[131518]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-27T15:25:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:41 smithi060 ceph-mon[131518]: osdmap e92: 12 total, 12 up, 12 in 2023-11-27T15:25:41.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:41 smithi008 ceph-mon[128788]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-27T15:25:41.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:41 smithi008 ceph-mon[128788]: osdmap e92: 12 total, 12 up, 12 in 2023-11-27T15:25:41.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:41 smithi029 ceph-mon[131166]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-11-27T15:25:41.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:41 smithi029 ceph-mon[131166]: osdmap e92: 12 total, 12 up, 12 in 2023-11-27T15:25:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:43 smithi060 ceph-mon[131518]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:43.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:43 smithi008 ceph-mon[128788]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:43.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:43 smithi029 ceph-mon[131166]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 127 B/s wr, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:45 smithi060 ceph-mon[131518]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:45.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:45 smithi008 ceph-mon[128788]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:45 smithi029 ceph-mon[131166]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 127 B/s wr, 4 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:46.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:46 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:25:46.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:46 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:25:46.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:46 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:25:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:47 smithi060 ceph-mon[131518]: pgmap v356: 129 pgs: 129 active+clean; 650 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:47 smithi008 ceph-mon[128788]: pgmap v356: 129 pgs: 129 active+clean; 650 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:47 smithi029 ceph-mon[131166]: pgmap v356: 129 pgs: 129 active+clean; 650 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:49.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:49 smithi060 ceph-mon[131518]: pgmap v357: 129 pgs: 129 active+clean; 650 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:49.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:49 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:49 smithi008 ceph-mon[128788]: pgmap v357: 129 pgs: 129 active+clean; 650 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:49.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:49 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:49 smithi029 ceph-mon[131166]: pgmap v357: 129 pgs: 129 active+clean; 650 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2023-11-27T15:25:49.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:49.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:49 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:51.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:51.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:51 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:51.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:51 smithi060 ceph-mon[131518]: pgmap v358: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:25:51.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:51.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:51 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:51.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:51 smithi008 ceph-mon[128788]: pgmap v358: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:25:51.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:51.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:51 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:51.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:51 smithi029 ceph-mon[131166]: pgmap v358: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:25:52.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:25:52.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:25:52.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:52 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:25:52.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:52.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:52 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:25:52.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:52 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:25:52.575 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:52 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:25:52.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:25:52.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:25:52.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:52 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:25:52.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:52.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:52 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:25:52.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:52 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:25:52.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:52 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:25:52.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:25:52.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:25:52.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:52 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:25:52.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:25:52.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:52 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:25:52.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:52 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:25:52.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:52 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:25:53.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:53 smithi060 ceph-mon[131518]: pgmap v359: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:25:53.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:53 smithi008 ceph-mon[128788]: pgmap v359: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:25:53.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:53 smithi029 ceph-mon[131166]: pgmap v359: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-11-27T15:25:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.094+0000 7f3def59e700 1 -- 172.21.15.8:0/3102576983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3df0103120 msgr2=0x7f3df01055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.094+0000 7f3def59e700 1 --2- 172.21.15.8:0/3102576983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3df0103120 0x7f3df01055a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009990 tx=0x7f3de4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.094+0000 7f3def59e700 1 -- 172.21.15.8:0/3102576983 shutdown_connections 2023-11-27T15:25:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.094+0000 7f3def59e700 1 --2- 172.21.15.8:0/3102576983 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df0105c30 0x7f3df01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.094+0000 7f3def59e700 1 --2- 172.21.15.8:0/3102576983 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3df0068d90 0x7f3df0102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.094+0000 7f3def59e700 1 --2- 172.21.15.8:0/3102576983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3df0103120 0x7f3df01055a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.094+0000 7f3def59e700 1 -- 172.21.15.8:0/3102576983 >> 172.21.15.8:0/3102576983 conn(0x7f3df00fa550 msgr2=0x7f3df00fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.094+0000 7f3def59e700 1 -- 172.21.15.8:0/3102576983 shutdown_connections 2023-11-27T15:25:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.094+0000 7f3def59e700 1 -- 172.21.15.8:0/3102576983 wait complete. 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.095+0000 7f3def59e700 1 Processor -- start 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.095+0000 7f3def59e700 1 -- start start 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.095+0000 7f3def59e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df0068d90 0x7f3df0100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.095+0000 7f3def59e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3df0103120 0x7f3df00feb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.095+0000 7f3def59e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3df0105c30 0x7f3df00ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.095+0000 7f3def59e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00ff860 con 0x7f3df0105c30 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.095+0000 7f3def59e700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00ff9d0 con 0x7f3df0103120 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.095+0000 7f3def59e700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df019fe90 con 0x7f3df0068d90 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3dee59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df0068d90 0x7f3df0100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3dee59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df0068d90 0x7f3df0100540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:50750/0 (socket says 172.21.15.8:50750) 2023-11-27T15:25:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3dee59c700 1 -- 172.21.15.8:0/3452709524 learned_addr learned my addr 172.21.15.8:0/3452709524 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3df0103120 0x7f3df00feb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3deed9d700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3df0105c30 0x7f3df00ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3dee59c700 1 -- 172.21.15.8:0/3452709524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3df0103120 msgr2=0x7f3df00feb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3dee59c700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3df0103120 0x7f3df00feb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3dee59c700 1 -- 172.21.15.8:0/3452709524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3df0105c30 msgr2=0x7f3df00ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3dee59c700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3df0105c30 0x7f3df00ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3dee59c700 1 -- 172.21.15.8:0/3452709524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40095f0 con 0x7f3df0068d90 2023-11-27T15:25:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.096+0000 7f3dee59c700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df0068d90 0x7f3df0100540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3de000e970 tx=0x7f3de000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.097+0000 7f3ddf7fe700 1 -- 172.21.15.8:0/3452709524 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0009790 con 0x7f3df0068d90 2023-11-27T15:25:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.097+0000 7f3def59e700 1 -- 172.21.15.8:0/3452709524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df010ba20 con 0x7f3df0068d90 2023-11-27T15:25:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.097+0000 7f3ddf7fe700 1 -- 172.21.15.8:0/3452709524 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de000c7e0 con 0x7f3df0068d90 2023-11-27T15:25:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.097+0000 7f3ddf7fe700 1 -- 172.21.15.8:0/3452709524 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de00054e0 con 0x7f3df0068d90 2023-11-27T15:25:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.097+0000 7f3def59e700 1 -- 172.21.15.8:0/3452709524 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df010b890 con 0x7f3df0068d90 2023-11-27T15:25:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.101+0000 7f3ddf7fe700 1 -- 172.21.15.8:0/3452709524 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3de000c950 con 0x7f3df0068d90 2023-11-27T15:25:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.102+0000 7f3e08982b80 1 Processor -- start 2023-11-27T15:25:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.102+0000 7f3e08982b80 1 -- start start 2023-11-27T15:25:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad11a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad11fb0 0x5639bad12820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad13200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3e08982b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bac17000 con 0x5639bad12d60 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3e08982b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bac69840 con 0x5639bad11fb0 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3e08982b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bac62cd0 con 0x5639bab6d6b0 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3deed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad11a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3deed9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad11a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:50756/0 (socket says 172.21.15.8:50756) 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3deed9d700 1 -- 172.21.15.8:0/1571932095 learned_addr learned my addr 172.21.15.8:0/1571932095 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3ddf7fe700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3dd80a1150 0x7f3dd80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/1571932095 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad13200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3ddf7fe700 1 -- 172.21.15.8:0/3452709524 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3de0014070 con 0x7f3df0068d90 2023-11-27T15:25:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.103+0000 7f3dee59c700 1 --2- 172.21.15.8:0/1571932095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad11fb0 0x5639bad12820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.104+0000 7f3deed9d700 1 -- 172.21.15.8:0/1571932095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad11fb0 msgr2=0x5639bad12820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.104+0000 7f3deed9d700 1 --2- 172.21.15.8:0/1571932095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad11fb0 0x5639bad12820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.104+0000 7f3deed9d700 1 -- 172.21.15.8:0/1571932095 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 msgr2=0x5639bad13200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.104+0000 7f3deed9d700 1 --2- 172.21.15.8:0/1571932095 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad13200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.104+0000 7f3deed9d700 1 -- 172.21.15.8:0/1571932095 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639baae45d0 con 0x5639bab6d6b0 2023-11-27T15:25:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.104+0000 7f3deed9d700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3dd80a1150 0x7f3dd80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.104+0000 7f3deed9d700 1 --2- 172.21.15.8:0/1571932095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad11a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3de80099f0 tx=0x7f3de8009d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1b3c7eddd495844 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3deed9d700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3dd80a1150 0x7f3dd80a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3de8021000 tx=0x7f3de801e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3ddcff9700 1 -- 172.21.15.8:0/1571932095 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8007c70 con 0x5639bab6d6b0 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3ddcff9700 1 -- 172.21.15.8:0/1571932095 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de800f040 con 0x5639bab6d6b0 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3ddcff9700 1 -- 172.21.15.8:0/1571932095 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80173e0 con 0x5639bab6d6b0 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3e08982b80 1 -- 172.21.15.8:0/1571932095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 msgr2=0x5639bad11a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3e08982b80 1 --2- 172.21.15.8:0/1571932095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad11a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3de80099f0 tx=0x7f3de8009d00 comp rx=0 tx=0).stop 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3e08982b80 1 -- 172.21.15.8:0/1571932095 shutdown_connections 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3e08982b80 1 --2- 172.21.15.8:0/1571932095 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad11a70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3e08982b80 1 --2- 172.21.15.8:0/1571932095 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad13200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3e08982b80 1 --2- 172.21.15.8:0/1571932095 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad11fb0 0x5639bad12820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.105+0000 7f3e08982b80 1 -- 172.21.15.8:0/1571932095 >> 172.21.15.8:0/1571932095 conn(0x5639bad071a0 msgr2=0x5639bad0c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.106+0000 7f3e08982b80 1 -- 172.21.15.8:0/1571932095 shutdown_connections 2023-11-27T15:25:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.106+0000 7f3e08982b80 1 -- 172.21.15.8:0/1571932095 wait complete. 2023-11-27T15:25:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.106+0000 7f3e08982b80 1 -- start start 2023-11-27T15:25:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.107+0000 7f3e08982b80 1 Processor -- start 2023-11-27T15:25:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.107+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.107+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad11fb0 0x5639bad1b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.107+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad22470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.107+0000 7f3e08982b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bac04880 con 0x5639bad12d60 2023-11-27T15:25:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.107+0000 7f3e08982b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bac04520 con 0x5639bad11fb0 2023-11-27T15:25:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.107+0000 7f3e08982b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639baae68c0 con 0x5639bab6d6b0 2023-11-27T15:25:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.107+0000 7f3e08982b80 10 client.0 _ll_register_callbacks cb 0x5639bac22720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dee59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dedd9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad11fb0 0x5639bad1b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dee59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:50760/0 (socket says 172.21.15.8:50760) 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dee59c700 1 -- 172.21.15.8:0/715459885 learned_addr learned my addr 172.21.15.8:0/715459885 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3deed9d700 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad22470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dee59c700 1 -- 172.21.15.8:0/715459885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad11fb0 msgr2=0x5639bad1b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dee59c700 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad11fb0 0x5639bad1b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dee59c700 1 -- 172.21.15.8:0/715459885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 msgr2=0x5639bad22470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dee59c700 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad22470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dee59c700 1 -- 172.21.15.8:0/715459885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de8003c80 con 0x5639bab6d6b0 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3deed9d700 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad22470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dee59c700 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1aef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3de000cc00 tx=0x7f3de0099f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c2406275190bee8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.108+0000 7f3dd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:25:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/715459885 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de00a2380 con 0x5639bab6d6b0 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3e08982b80 1 -- 172.21.15.8:0/715459885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639bad04960 con 0x5639bab6d6b0 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/715459885 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3de0099420 con 0x5639bab6d6b0 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/715459885 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3de00b82b0 con 0x5639bab6d6b0 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3e08982b80 10 client.25402 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3e08982b80 1 -- 172.21.15.8:0/715459885 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5639bac29130 con 0x5639bab6d6b0 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/715459885 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de00ca080 con 0x5639bab6d6b0 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/715459885 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3de00a1f60 con 0x5639bab6d6b0 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3e08982b80 10 client.25402 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3e08982b80 10 client.25402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.109+0000 7f3e08982b80 20 client.25402 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.110+0000 7f3e08982b80 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5639bad26c80 0x5639bad29090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.110+0000 7f3e08982b80 4 client.25402 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.110+0000 7f3e08982b80 1 -- 172.21.15.8:0/715459885 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639bac25040 con 0x5639bad26c80 2023-11-27T15:25:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.110+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5639bad26c80 0x5639bad29090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.110+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5639bad26c80 0x5639bad29090 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c4ccff81cb4b71 server_cookie=860f5eec9e06caeb in_seq=0 out_seq=0 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.110+0000 7f3dd5ffb700 10 client.25402 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.111+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/715459885 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5639bac25040 con 0x5639bad26c80 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.111+0000 7f3dd5ffb700 10 client.25402 handle_command_reply: tid=0 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.111+0000 7f3e08982b80 1 client.25402 shutdown 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.111+0000 7f3e08982b80 1 -- 172.21.15.8:0/715459885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 msgr2=0x5639bad1aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.111+0000 7f3e08982b80 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1aef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3de000cc00 tx=0x7f3de0099f30 comp rx=0 tx=0).stop 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.112+0000 7f3e08982b80 1 -- 172.21.15.8:0/715459885 shutdown_connections 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.112+0000 7f3e08982b80 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5639bad26c80 0x5639bad29090 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.112+0000 7f3e08982b80 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1aef0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.112+0000 7f3e08982b80 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad22470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.112+0000 7f3dd5ffb700 0 client.25402 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.112+0000 7f3e08982b80 1 --2- 172.21.15.8:0/715459885 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad11fb0 0x5639bad1b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.112+0000 7f3e08982b80 1 -- 172.21.15.8:0/715459885 >> 172.21.15.8:0/715459885 conn(0x5639bad071a0 msgr2=0x5639bad0ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.112+0000 7f3e08982b80 1 -- 172.21.15.8:0/715459885 shutdown_connections 2023-11-27T15:25:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.112+0000 7f3e08982b80 1 -- 172.21.15.8:0/715459885 wait complete. 2023-11-27T15:25:54.118 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.112+0000 7f3e08982b80 20 client.25402 trim_cache size 0 max 16384 2023-11-27T15:25:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.128+0000 7f3e08982b80 1 Processor -- start 2023-11-27T15:25:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.128+0000 7f3e08982b80 1 -- start start 2023-11-27T15:25:54.132 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bab6d6b0 0x5639bad2d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bad11fb0 0x5639bad22430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad15c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3e08982b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bad199d0 con 0x5639bad12d60 2023-11-27T15:25:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3e08982b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bad06d70 con 0x5639bab6d6b0 2023-11-27T15:25:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3e08982b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bad0ddd0 con 0x5639bad11fb0 2023-11-27T15:25:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3deed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad15c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3deed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad15c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41132/0 (socket says 172.21.15.8:41132) 2023-11-27T15:25:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3deed9d700 1 -- 172.21.15.8:0/2607205922 learned_addr learned my addr 172.21.15.8:0/2607205922 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/2607205922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bad11fb0 0x5639bad22430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3dee59c700 1 --2- 172.21.15.8:0/2607205922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bab6d6b0 0x5639bad2d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3dedd9b700 1 -- 172.21.15.8:0/2607205922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bab6d6b0 msgr2=0x5639bad2d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.129+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/2607205922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bab6d6b0 0x5639bad2d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.130+0000 7f3dedd9b700 1 -- 172.21.15.8:0/2607205922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 msgr2=0x5639bad15c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.130+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/2607205922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad15c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.130+0000 7f3dedd9b700 1 -- 172.21.15.8:0/2607205922 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639bac28e80 con 0x5639bad11fb0 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.130+0000 7f3deed9d700 1 --2- 172.21.15.8:0/2607205922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad15c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.130+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/2607205922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bad11fb0 0x5639bad22430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de4005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86fb1058fdf29aba server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.130+0000 7f3ddd7fa700 1 -- 172.21.15.8:0/2607205922 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de401d040 con 0x5639bad11fb0 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.130+0000 7f3ddd7fa700 1 -- 172.21.15.8:0/2607205922 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de401e040 con 0x5639bad11fb0 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.130+0000 7f3ddd7fa700 1 -- 172.21.15.8:0/2607205922 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de40052a0 con 0x5639bad11fb0 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.131+0000 7f3e08982b80 1 -- 172.21.15.8:0/2607205922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bad11fb0 msgr2=0x5639bad22430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.131+0000 7f3e08982b80 1 --2- 172.21.15.8:0/2607205922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bad11fb0 0x5639bad22430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.131+0000 7f3e08982b80 1 -- 172.21.15.8:0/2607205922 shutdown_connections 2023-11-27T15:25:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.131+0000 7f3e08982b80 1 --2- 172.21.15.8:0/2607205922 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad12d60 0x5639bad15c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.131+0000 7f3e08982b80 1 --2- 172.21.15.8:0/2607205922 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bab6d6b0 0x5639bad2d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.131+0000 7f3e08982b80 1 --2- 172.21.15.8:0/2607205922 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bad11fb0 0x5639bad22430 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.131+0000 7f3e08982b80 1 -- 172.21.15.8:0/2607205922 >> 172.21.15.8:0/2607205922 conn(0x5639bad071a0 msgr2=0x5639bad19c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.131+0000 7f3e08982b80 1 -- 172.21.15.8:0/2607205922 shutdown_connections 2023-11-27T15:25:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.131+0000 7f3e08982b80 1 -- 172.21.15.8:0/2607205922 wait complete. 2023-11-27T15:25:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.131+0000 7f3e08982b80 1 -- start start 2023-11-27T15:25:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3e08982b80 1 Processor -- start 2023-11-27T15:25:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad11fb0 0x5639bad1ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3e08982b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad12d60 0x5639bad21a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3e08982b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bad21fa0 con 0x5639bad11fb0 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3e08982b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bad22110 con 0x5639bad12d60 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3e08982b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639bad222d0 con 0x5639bab6d6b0 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3e08982b80 10 client.0 _ll_register_callbacks cb 0x7f3de0099b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3dee59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3dedd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad11fb0 0x5639bad1ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.132+0000 7f3dee59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:50792/0 (socket says 172.21.15.8:50792) 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dee59c700 1 -- 172.21.15.8:0/2645665762 learned_addr learned my addr 172.21.15.8:0/2645665762 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3deed9d700 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad12d60 0x5639bad21a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dee59c700 1 -- 172.21.15.8:0/2645665762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad12d60 msgr2=0x5639bad21a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dee59c700 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad12d60 0x5639bad21a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dee59c700 1 -- 172.21.15.8:0/2645665762 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad11fb0 msgr2=0x5639bad1ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dee59c700 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad11fb0 0x5639bad1ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dee59c700 1 -- 172.21.15.8:0/2645665762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de8026070 con 0x5639bab6d6b0 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad11fb0 0x5639bad1ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dee59c700 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1e500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3de000cef0 tx=0x7f3de00ccf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8171c458a28869c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dd5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/2645665762 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0096270 con 0x5639bab6d6b0 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/2645665762 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3de00a2380 con 0x5639bab6d6b0 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.133+0000 7f3e08982b80 1 -- 172.21.15.8:0/2645665762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3de800f040 con 0x5639bab6d6b0 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/2645665762 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3de0099420 con 0x5639bab6d6b0 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3e08982b80 10 client.25408 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3e08982b80 1 -- 172.21.15.8:0/2645665762 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f3df0108da0 con 0x5639bab6d6b0 2023-11-27T15:25:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/2645665762 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3de800f040 con 0x5639bab6d6b0 2023-11-27T15:25:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/2645665762 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3de0095d20 con 0x5639bab6d6b0 2023-11-27T15:25:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3e08982b80 10 client.25408 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3e08982b80 10 client.25408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:25:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3e08982b80 20 client.25408 populate_metadata read hostname 'smithi008' 2023-11-27T15:25:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3e08982b80 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5639bad7b970 0x5639bad7dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3e08982b80 4 client.25408 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:25:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3e08982b80 1 -- 172.21.15.8:0/2645665762 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639bac31600 con 0x5639bad7b970 2023-11-27T15:25:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.134+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5639bad7b970 0x5639bad7dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.135+0000 7f3dd5ffb700 10 client.25408 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.135+0000 7f3dedd9b700 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5639bad7b970 0x5639bad7dd80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20e32f3fc30fe7c server_cookie=b9ac9fa10198532f in_seq=0 out_seq=0 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.151+0000 7f3dd5ffb700 1 -- 172.21.15.8:0/2645665762 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639bac31600 con 0x5639bad7b970 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.151+0000 7f3dd5ffb700 10 client.25408 handle_command_reply: tid=0 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 client.25408 shutdown 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 -- 172.21.15.8:0/2645665762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 msgr2=0x5639bad1e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1e500 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3de000cef0 tx=0x7f3de00ccf30 comp rx=0 tx=0).stop 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 -- 172.21.15.8:0/2645665762 shutdown_connections 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5639bad7b970 0x5639bad7dd80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5639bab6d6b0 0x5639bad1e500 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5639bad12d60 0x5639bad21a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 --2- 172.21.15.8:0/2645665762 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5639bad11fb0 0x5639bad1ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3dd5ffb700 0 client.25408 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 -- 172.21.15.8:0/2645665762 >> 172.21.15.8:0/2645665762 conn(0x5639bad071a0 msgr2=0x5639bad2d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 -- 172.21.15.8:0/2645665762 shutdown_connections 2023-11-27T15:25:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 1 -- 172.21.15.8:0/2645665762 wait complete. 2023-11-27T15:25:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.152+0000 7f3e08982b80 20 client.25408 trim_cache size 0 max 16384 2023-11-27T15:25:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:25:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:25:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:25:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:25:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 -- 172.21.15.8:0/3452709524 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3dd80a1150 msgr2=0x7f3dd80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3dd80a1150 0x7f3dd80a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3de8021000 tx=0x7f3de801e000 comp rx=0 tx=0).stop 2023-11-27T15:25:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 -- 172.21.15.8:0/3452709524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df0068d90 msgr2=0x7f3df0100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df0068d90 0x7f3df0100540 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3de000e970 tx=0x7f3de000ec80 comp rx=0 tx=0).stop 2023-11-27T15:25:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 -- 172.21.15.8:0/3452709524 shutdown_connections 2023-11-27T15:25:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3dd80a1150 0x7f3dd80a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3df0068d90 0x7f3df0100540 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3df0105c30 0x7f3df00ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 --2- 172.21.15.8:0/3452709524 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3df0103120 0x7f3df00feb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 -- 172.21.15.8:0/3452709524 >> 172.21.15.8:0/3452709524 conn(0x7f3df00fa550 msgr2=0x7f3df0101500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 -- 172.21.15.8:0/3452709524 shutdown_connections 2023-11-27T15:25:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:25:54.153+0000 7f3def59e700 1 -- 172.21.15.8:0/3452709524 wait complete. 2023-11-27T15:25:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:55 smithi060 ceph-mon[131518]: pgmap v360: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-27T15:25:55.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:55 smithi008 ceph-mon[128788]: pgmap v360: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-27T15:25:55.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:55 smithi029 ceph-mon[131166]: pgmap v360: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-11-27T15:25:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:57 smithi060 ceph-mon[131518]: pgmap v361: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2023-11-27T15:25:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:57 smithi008 ceph-mon[128788]: pgmap v361: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2023-11-27T15:25:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:57 smithi029 ceph-mon[131166]: pgmap v361: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2023-11-27T15:25:59.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:25:59 smithi060 ceph-mon[131518]: pgmap v362: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-11-27T15:25:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:25:59 smithi008 ceph-mon[128788]: pgmap v362: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-11-27T15:25:59.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:25:59 smithi029 ceph-mon[131166]: pgmap v362: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-11-27T15:26:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:01 smithi060 ceph-mon[131518]: pgmap v363: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-11-27T15:26:01.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:01 smithi008 ceph-mon[128788]: pgmap v363: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-11-27T15:26:01.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:01 smithi029 ceph-mon[131166]: pgmap v363: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-11-27T15:26:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:03 smithi060 ceph-mon[131518]: pgmap v364: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-27T15:26:03.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:03 smithi008 ceph-mon[128788]: pgmap v364: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-27T15:26:03.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:03 smithi029 ceph-mon[131166]: pgmap v364: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-27T15:26:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:05 smithi060 ceph-mon[131518]: pgmap v365: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-27T15:26:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:05 smithi008 ceph-mon[128788]: pgmap v365: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-27T15:26:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:05 smithi029 ceph-mon[131166]: pgmap v365: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-27T15:26:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.392+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/3493098709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d681012f0 msgr2=0x7f1d68101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.392+0000 7f1d6cb9d700 1 --2- 172.21.15.8:0/3493098709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d681012f0 0x7f1d68101760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0098f0 tx=0x7f1d5c009c00 comp rx=0 tx=0).stop 2023-11-27T15:26:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.393+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/3493098709 shutdown_connections 2023-11-27T15:26:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.393+0000 7f1d6cb9d700 1 --2- 172.21.15.8:0/3493098709 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d68101ca0 0x7f1d68108540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.393+0000 7f1d6cb9d700 1 --2- 172.21.15.8:0/3493098709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d68100060 0x7f1d68100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.393+0000 7f1d6cb9d700 1 --2- 172.21.15.8:0/3493098709 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d681012f0 0x7f1d68101760 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.393+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/3493098709 >> 172.21.15.8:0/3493098709 conn(0x7f1d680fb640 msgr2=0x7f1d680fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.393+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/3493098709 shutdown_connections 2023-11-27T15:26:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.393+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/3493098709 wait complete. 2023-11-27T15:26:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.394+0000 7f1d6cb9d700 1 Processor -- start 2023-11-27T15:26:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.395+0000 7f1d6cb9d700 1 -- start start 2023-11-27T15:26:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.395+0000 7f1d6cb9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d68100060 0x7f1d68193c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.395+0000 7f1d6cb9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d681012f0 0x7f1d68194180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.395+0000 7f1d6cb9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d68101ca0 0x7f1d6819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.395+0000 7f1d6cb9d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6819b780 con 0x7f1d68100060 2023-11-27T15:26:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.395+0000 7f1d6cb9d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6819b8f0 con 0x7f1d681012f0 2023-11-27T15:26:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.395+0000 7f1d6cb9d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6819bab0 con 0x7f1d68101ca0 2023-11-27T15:26:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d66ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d681012f0 0x7f1d68194180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d677fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d68100060 0x7f1d68193c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d66ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d681012f0 0x7f1d68194180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:60856/0 (socket says 172.21.15.8:60856) 2023-11-27T15:26:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d66ffd700 1 -- 172.21.15.8:0/2445811603 learned_addr learned my addr 172.21.15.8:0/2445811603 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d67fff700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d68101ca0 0x7f1d6819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d66ffd700 1 -- 172.21.15.8:0/2445811603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d68101ca0 msgr2=0x7f1d6819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d68101ca0 0x7f1d6819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d66ffd700 1 -- 172.21.15.8:0/2445811603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d68100060 msgr2=0x7f1d68193c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d68100060 0x7f1d68193c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d66ffd700 1 -- 172.21.15.8:0/2445811603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5800da00 con 0x7f1d681012f0 2023-11-27T15:26:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.396+0000 7f1d677fe700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d68100060 0x7f1d68193c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.397+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d681012f0 0x7f1d68194180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009fd0 tx=0x7f1d5c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.397+0000 7f1d64ff9700 1 -- 172.21.15.8:0/2445811603 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c01c070 con 0x7f1d681012f0 2023-11-27T15:26:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.397+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/2445811603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c009520 con 0x7f1d681012f0 2023-11-27T15:26:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.397+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/2445811603 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d681098a0 con 0x7f1d681012f0 2023-11-27T15:26:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.397+0000 7f1d64ff9700 1 -- 172.21.15.8:0/2445811603 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d5c00f7d0 con 0x7f1d681012f0 2023-11-27T15:26:07.402 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.397+0000 7f1d64ff9700 1 -- 172.21.15.8:0/2445811603 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c004ad0 con 0x7f1d681012f0 2023-11-27T15:26:07.406 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.403+0000 7f1d64ff9700 1 -- 172.21.15.8:0/2445811603 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1d5c020070 con 0x7f1d681012f0 2023-11-27T15:26:07.407 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.404+0000 7f1d8161fb80 1 Processor -- start 2023-11-27T15:26:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.404+0000 7f1d8161fb80 1 -- start start 2023-11-27T15:26:07.409 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 0x55e5e8884280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8885030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e8885a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8781000 con 0x55e5e88847c0 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e87d3840 con 0x55e5e86d76b0 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e87cccd0 con 0x55e5e8885570 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d67fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 0x55e5e8884280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d66ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e8885a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d64ff9700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d500a11d0 0x7f1d500a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d67fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 0x55e5e8884280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:60858/0 (socket says 172.21.15.8:60858) 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.405+0000 7f1d67fff700 1 -- 172.21.15.8:0/1054031519 learned_addr learned my addr 172.21.15.8:0/1054031519 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.406+0000 7f1d67fff700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d500a11d0 0x7f1d500a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.406+0000 7f1d64ff9700 1 -- 172.21.15.8:0/2445811603 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1d5c0cd850 con 0x7f1d681012f0 2023-11-27T15:26:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.406+0000 7f1d66ffd700 1 -- 172.21.15.8:0/1054031519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 msgr2=0x55e5e8884280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.406+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/1054031519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 0x55e5e8884280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.406+0000 7f1d66ffd700 1 -- 172.21.15.8:0/1054031519 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 msgr2=0x55e5e8885030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.406+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/1054031519 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8885030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.406+0000 7f1d66ffd700 1 -- 172.21.15.8:0/1054031519 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e864e5d0 con 0x55e5e8885570 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.406+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/1054031519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e8885a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c014ff0 tx=0x7f1d5c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de31e4d3c2a8657 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.407+0000 7f1d4dffb700 1 -- 172.21.15.8:0/1054031519 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c09a2c0 con 0x55e5e8885570 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.407+0000 7f1d4dffb700 1 -- 172.21.15.8:0/1054031519 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d5c09a960 con 0x55e5e8885570 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.407+0000 7f1d4dffb700 1 -- 172.21.15.8:0/1054031519 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c004e30 con 0x55e5e8885570 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.408+0000 7f1d8161fb80 1 -- 172.21.15.8:0/1054031519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 msgr2=0x55e5e8885a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.408+0000 7f1d67fff700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d500a11d0 0x7f1d500a3680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000fec0 tx=0x7f1d6000f490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.408+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/1054031519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e8885a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c014ff0 tx=0x7f1d5c009260 comp rx=0 tx=0).stop 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.408+0000 7f1d8161fb80 1 -- 172.21.15.8:0/1054031519 shutdown_connections 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.408+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/1054031519 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e8885a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.408+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/1054031519 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 0x55e5e8884280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.408+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/1054031519 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8885030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.408+0000 7f1d8161fb80 1 -- 172.21.15.8:0/1054031519 >> 172.21.15.8:0/1054031519 conn(0x55e5e88711a0 msgr2=0x55e5e88762e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.408+0000 7f1d8161fb80 1 -- 172.21.15.8:0/1054031519 shutdown_connections 2023-11-27T15:26:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.408+0000 7f1d8161fb80 1 -- 172.21.15.8:0/1054031519 wait complete. 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.409+0000 7f1d8161fb80 1 -- start start 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.409+0000 7f1d8161fb80 1 Processor -- start 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e86d76b0 0x55e5e888d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e88847c0 0x55e5e888d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 0x55e5e88948c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e876e880 con 0x55e5e86d76b0 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e876e520 con 0x55e5e8885570 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e86508c0 con 0x55e5e88847c0 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d8161fb80 10 client.0 _ll_register_callbacks cb 0x55e5e878c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d66ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e88847c0 0x55e5e888d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d66ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e88847c0 0x55e5e888d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:51172/0 (socket says 172.21.15.8:51172) 2023-11-27T15:26:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d66ffd700 1 -- 172.21.15.8:0/2638136495 learned_addr learned my addr 172.21.15.8:0/2638136495 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d677fe700 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e86d76b0 0x55e5e888d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.410+0000 7f1d67fff700 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 0x55e5e88948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.411+0000 7f1d66ffd700 1 -- 172.21.15.8:0/2638136495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 msgr2=0x55e5e88948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.411+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 0x55e5e88948c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.411+0000 7f1d66ffd700 1 -- 172.21.15.8:0/2638136495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e86d76b0 msgr2=0x55e5e888d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.411+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e86d76b0 0x55e5e888d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.411+0000 7f1d66ffd700 1 -- 172.21.15.8:0/2638136495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d681098a0 con 0x55e5e88847c0 2023-11-27T15:26:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.411+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e88847c0 0x55e5e888d880 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0cdb00 tx=0x7f1d5c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=157a5628cf743a89 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.411+0000 7f1d46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:26:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.411+0000 7f1d46ffd700 1 -- 172.21.15.8:0/2638136495 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c09adc0 con 0x55e5e88847c0 2023-11-27T15:26:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.412+0000 7f1d46ffd700 1 -- 172.21.15.8:0/2638136495 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1d5c0a2d20 con 0x55e5e88847c0 2023-11-27T15:26:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.412+0000 7f1d8161fb80 1 -- 172.21.15.8:0/2638136495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5e886e960 con 0x55e5e88847c0 2023-11-27T15:26:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.412+0000 7f1d46ffd700 1 -- 172.21.15.8:0/2638136495 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d5c09e990 con 0x55e5e88847c0 2023-11-27T15:26:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.412+0000 7f1d8161fb80 10 client.25420 fetch_fsmap learned FSMap version 24 2023-11-27T15:26:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.412+0000 7f1d8161fb80 1 -- 172.21.15.8:0/2638136495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e5e8793130 con 0x55e5e88847c0 2023-11-27T15:26:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.412+0000 7f1d46ffd700 1 -- 172.21.15.8:0/2638136495 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e5e886e960 con 0x55e5e88847c0 2023-11-27T15:26:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.413+0000 7f1d46ffd700 1 -- 172.21.15.8:0/2638136495 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1d5c0b4020 con 0x55e5e88847c0 2023-11-27T15:26:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.413+0000 7f1d8161fb80 10 client.25420 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:26:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.413+0000 7f1d8161fb80 10 client.25420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:26:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.413+0000 7f1d8161fb80 20 client.25420 populate_metadata read hostname 'smithi008' 2023-11-27T15:26:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.413+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e5e8898d80 0x55e5e889b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.413+0000 7f1d8161fb80 4 client.25420 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.413+0000 7f1d8161fb80 1 -- 172.21.15.8:0/2638136495 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5e878f040 con 0x55e5e8898d80 2023-11-27T15:26:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.413+0000 7f1d677fe700 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e5e8898d80 0x55e5e889b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.414+0000 7f1d677fe700 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e5e8898d80 0x55e5e889b190 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd03eb1824b0dfe server_cookie=29d5161fa1eb31d0 in_seq=0 out_seq=0 2023-11-27T15:26:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.414+0000 7f1d46ffd700 10 client.25420 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.415+0000 7f1d46ffd700 1 -- 172.21.15.8:0/2638136495 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e5e878f040 con 0x55e5e8898d80 2023-11-27T15:26:07.419 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.415+0000 7f1d46ffd700 10 client.25420 handle_command_reply: tid=0 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.415+0000 7f1d8161fb80 1 client.25420 shutdown 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d8161fb80 1 -- 172.21.15.8:0/2638136495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e88847c0 msgr2=0x55e5e888d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e88847c0 0x55e5e888d880 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0cdb00 tx=0x7f1d5c009c00 comp rx=0 tx=0).stop 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d8161fb80 1 -- 172.21.15.8:0/2638136495 shutdown_connections 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 0x55e5e88948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e86d76b0 0x55e5e888d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e5e8898d80 0x55e5e889b190 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/2638136495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e88847c0 0x55e5e888d880 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d8161fb80 1 -- 172.21.15.8:0/2638136495 >> 172.21.15.8:0/2638136495 conn(0x55e5e88711a0 msgr2=0x55e5e8875a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d46ffd700 0 client.25420 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d8161fb80 1 -- 172.21.15.8:0/2638136495 shutdown_connections 2023-11-27T15:26:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.416+0000 7f1d8161fb80 1 -- 172.21.15.8:0/2638136495 wait complete. 2023-11-27T15:26:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.417+0000 7f1d8161fb80 20 client.25420 trim_cache size 0 max 16384 2023-11-27T15:26:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.446+0000 7f1d8161fb80 1 Processor -- start 2023-11-27T15:26:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.447+0000 7f1d8161fb80 1 -- start start 2023-11-27T15:26:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.447+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 0x55e5e888d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.447+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8894880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.447+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e888d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.447+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e888dfc0 con 0x55e5e88847c0 2023-11-27T15:26:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.447+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e878f040 con 0x55e5e86d76b0 2023-11-27T15:26:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.447+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e888e530 con 0x55e5e8885570 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.447+0000 7f1d67fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e888d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.447+0000 7f1d677fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 0x55e5e888d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.448+0000 7f1d67fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e888d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:51184/0 (socket says 172.21.15.8:51184) 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.448+0000 7f1d67fff700 1 -- 172.21.15.8:0/497543611 learned_addr learned my addr 172.21.15.8:0/497543611 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.448+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/497543611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8894880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.448+0000 7f1d67fff700 1 -- 172.21.15.8:0/497543611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 msgr2=0x55e5e888d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.448+0000 7f1d67fff700 1 --2- 172.21.15.8:0/497543611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 0x55e5e888d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.448+0000 7f1d67fff700 1 -- 172.21.15.8:0/497543611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 msgr2=0x55e5e8894880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.448+0000 7f1d67fff700 1 --2- 172.21.15.8:0/497543611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8894880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.448+0000 7f1d67fff700 1 -- 172.21.15.8:0/497543611 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e8792e80 con 0x55e5e8885570 2023-11-27T15:26:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.448+0000 7f1d67fff700 1 --2- 172.21.15.8:0/497543611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e888d880 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1d6001af20 tx=0x7f1d60015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5346eb04dcc4da77 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.449+0000 7f1d4e7fc700 1 -- 172.21.15.8:0/497543611 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000e040 con 0x55e5e8885570 2023-11-27T15:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.449+0000 7f1d4e7fc700 1 -- 172.21.15.8:0/497543611 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d6001f070 con 0x55e5e8885570 2023-11-27T15:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.449+0000 7f1d4e7fc700 1 -- 172.21.15.8:0/497543611 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d60007bf0 con 0x55e5e8885570 2023-11-27T15:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.450+0000 7f1d8161fb80 1 -- 172.21.15.8:0/497543611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 msgr2=0x55e5e888d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.450+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/497543611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e888d880 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1d6001af20 tx=0x7f1d60015000 comp rx=0 tx=0).stop 2023-11-27T15:26:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.450+0000 7f1d8161fb80 1 -- 172.21.15.8:0/497543611 shutdown_connections 2023-11-27T15:26:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.450+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/497543611 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e8885570 0x55e5e888d880 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.450+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/497543611 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e86d76b0 0x55e5e888d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.450+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/497543611 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8894880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.450+0000 7f1d8161fb80 1 -- 172.21.15.8:0/497543611 >> 172.21.15.8:0/497543611 conn(0x55e5e88711a0 msgr2=0x55e5e889fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.450+0000 7f1d8161fb80 1 -- 172.21.15.8:0/497543611 shutdown_connections 2023-11-27T15:26:07.454 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.450+0000 7f1d8161fb80 1 -- 172.21.15.8:0/497543611 wait complete. 2023-11-27T15:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.451+0000 7f1d8161fb80 1 -- start start 2023-11-27T15:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.451+0000 7f1d8161fb80 1 Processor -- start 2023-11-27T15:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e86d76b0 0x55e5e8892ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8893000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d8161fb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 0x55e5e8893540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8893c50 con 0x55e5e88847c0 2023-11-27T15:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8893dc0 con 0x55e5e8885570 2023-11-27T15:26:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d8161fb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8893f80 con 0x55e5e86d76b0 2023-11-27T15:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d8161fb80 10 client.0 _ll_register_callbacks cb 0x7f1d6000f1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e86d76b0 0x55e5e8892ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d66ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8893000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d677fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e86d76b0 0x55e5e8892ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:51186/0 (socket says 172.21.15.8:51186) 2023-11-27T15:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d677fe700 1 -- 172.21.15.8:0/3736900639 learned_addr learned my addr 172.21.15.8:0/3736900639 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.452+0000 7f1d67fff700 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 0x55e5e8893540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d66ffd700 1 -- 172.21.15.8:0/3736900639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e86d76b0 msgr2=0x55e5e8892ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e86d76b0 0x55e5e8892ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d66ffd700 1 -- 172.21.15.8:0/3736900639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 msgr2=0x55e5e8893540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 0x55e5e8893540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d66ffd700 1 -- 172.21.15.8:0/3736900639 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6001ac60 con 0x55e5e88847c0 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d66ffd700 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8893000 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c09a8f0 tx=0x7f1d5c098e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd1ab676954f670 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d46ffd700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d67fff700 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 0x55e5e8893540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d46ffd700 1 -- 172.21.15.8:0/3736900639 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0a2b50 con 0x55e5e88847c0 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d46ffd700 1 -- 172.21.15.8:0/3736900639 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1d5c0a2d20 con 0x55e5e88847c0 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.453+0000 7f1d8161fb80 1 -- 172.21.15.8:0/3736900639 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d5c09a960 con 0x55e5e88847c0 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.454+0000 7f1d46ffd700 1 -- 172.21.15.8:0/3736900639 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d5c09e990 con 0x55e5e88847c0 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.454+0000 7f1d8161fb80 10 client.15381 fetch_fsmap learned FSMap version 24 2023-11-27T15:26:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.454+0000 7f1d8161fb80 1 -- 172.21.15.8:0/3736900639 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e5e85e3c00 con 0x55e5e88847c0 2023-11-27T15:26:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.454+0000 7f1d46ffd700 1 -- 172.21.15.8:0/3736900639 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d5c09a960 con 0x55e5e88847c0 2023-11-27T15:26:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.455+0000 7f1d46ffd700 1 -- 172.21.15.8:0/3736900639 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1d5c0b4070 con 0x55e5e88847c0 2023-11-27T15:26:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.455+0000 7f1d8161fb80 10 client.15381 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:26:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.455+0000 7f1d8161fb80 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:26:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.455+0000 7f1d8161fb80 20 client.15381 populate_metadata read hostname 'smithi008' 2023-11-27T15:26:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.455+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e5e88ed160 0x55e5e88ef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.456+0000 7f1d8161fb80 4 client.15381 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:26:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.456+0000 7f1d8161fb80 1 -- 172.21.15.8:0/3736900639 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e5e879b600 con 0x55e5e88ed160 2023-11-27T15:26:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.456+0000 7f1d677fe700 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e5e88ed160 0x55e5e88ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.457+0000 7f1d677fe700 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e5e88ed160 0x55e5e88ef540 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f888740fac2001 server_cookie=c5886b47e857f446 in_seq=0 out_seq=0 2023-11-27T15:26:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.457+0000 7f1d46ffd700 10 client.15381 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.461+0000 7f1d46ffd700 1 -- 172.21.15.8:0/3736900639 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e5e879b600 con 0x55e5e88ed160 2023-11-27T15:26:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.461+0000 7f1d46ffd700 10 client.15381 handle_command_reply: tid=0 2023-11-27T15:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.461+0000 7f1d8161fb80 1 client.15381 shutdown 2023-11-27T15:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.461+0000 7f1d8161fb80 1 -- 172.21.15.8:0/3736900639 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 msgr2=0x55e5e8893000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.461+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8893000 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c09a8f0 tx=0x7f1d5c098e50 comp rx=0 tx=0).stop 2023-11-27T15:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.462+0000 7f1d8161fb80 1 -- 172.21.15.8:0/3736900639 shutdown_connections 2023-11-27T15:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.462+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e5e88ed160 0x55e5e88ef540 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.462+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e5e86d76b0 0x55e5e8892ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.462+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e5e8885570 0x55e5e8893540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.462+0000 7f1d8161fb80 1 --2- 172.21.15.8:0/3736900639 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e5e88847c0 0x55e5e8893000 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.462+0000 7f1d8161fb80 1 -- 172.21.15.8:0/3736900639 >> 172.21.15.8:0/3736900639 conn(0x55e5e88711a0 msgr2=0x55e5e88882a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.462+0000 7f1d46ffd700 0 client.15381 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.462+0000 7f1d8161fb80 1 -- 172.21.15.8:0/3736900639 shutdown_connections 2023-11-27T15:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.462+0000 7f1d8161fb80 1 -- 172.21.15.8:0/3736900639 wait complete. 2023-11-27T15:26:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.463+0000 7f1d8161fb80 20 client.15381 trim_cache size 0 max 16384 2023-11-27T15:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:26:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.463+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/2445811603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d500a11d0 msgr2=0x7f1d500a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.463+0000 7f1d6cb9d700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d500a11d0 0x7f1d500a3680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000fec0 tx=0x7f1d6000f490 comp rx=0 tx=0).stop 2023-11-27T15:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.463+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/2445811603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d681012f0 msgr2=0x7f1d68194180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.463+0000 7f1d6cb9d700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d681012f0 0x7f1d68194180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009fd0 tx=0x7f1d5c00b790 comp rx=0 tx=0).stop 2023-11-27T15:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.464+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/2445811603 shutdown_connections 2023-11-27T15:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.464+0000 7f1d6cb9d700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d500a11d0 0x7f1d500a3680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.464+0000 7f1d6cb9d700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d68101ca0 0x7f1d6819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.464+0000 7f1d6cb9d700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d68100060 0x7f1d68193c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.464+0000 7f1d6cb9d700 1 --2- 172.21.15.8:0/2445811603 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d681012f0 0x7f1d68194180 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.464+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/2445811603 >> 172.21.15.8:0/2445811603 conn(0x7f1d680fb640 msgr2=0x7f1d681047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.464+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/2445811603 shutdown_connections 2023-11-27T15:26:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:07.464+0000 7f1d6cb9d700 1 -- 172.21.15.8:0/2445811603 wait complete. 2023-11-27T15:26:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:07 smithi060 ceph-mon[131518]: pgmap v366: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-11-27T15:26:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:07 smithi008 ceph-mon[128788]: pgmap v366: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-11-27T15:26:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:07 smithi029 ceph-mon[131166]: pgmap v366: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-11-27T15:26:09.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:09 smithi060 ceph-mon[131518]: pgmap v367: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 7 op/s 2023-11-27T15:26:09.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:09 smithi008 ceph-mon[128788]: pgmap v367: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 7 op/s 2023-11-27T15:26:09.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:09 smithi029 ceph-mon[131166]: pgmap v367: 129 pgs: 129 active+clean; 654 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 511 B/s wr, 7 op/s 2023-11-27T15:26:11.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:11 smithi060 ceph-mon[131518]: pgmap v368: 129 pgs: 129 active+clean; 655 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2023-11-27T15:26:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:11 smithi008 ceph-mon[128788]: pgmap v368: 129 pgs: 129 active+clean; 655 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2023-11-27T15:26:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:11 smithi029 ceph-mon[131166]: pgmap v368: 129 pgs: 129 active+clean; 655 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2023-11-27T15:26:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:13 smithi060 ceph-mon[131518]: pgmap v369: 129 pgs: 129 active+clean; 655 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-27T15:26:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:13 smithi008 ceph-mon[128788]: pgmap v369: 129 pgs: 129 active+clean; 655 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-27T15:26:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:13 smithi029 ceph-mon[131166]: pgmap v369: 129 pgs: 129 active+clean; 655 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-27T15:26:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:15 smithi060 ceph-mon[131518]: pgmap v370: 129 pgs: 129 active+clean; 655 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:26:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:15 smithi008 ceph-mon[128788]: pgmap v370: 129 pgs: 129 active+clean; 655 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:26:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:15 smithi029 ceph-mon[131166]: pgmap v370: 129 pgs: 129 active+clean; 655 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:26:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:17 smithi060 ceph-mon[131518]: pgmap v371: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-11-27T15:26:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:17 smithi008 ceph-mon[128788]: pgmap v371: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-11-27T15:26:17.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:17 smithi029 ceph-mon[131166]: pgmap v371: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-11-27T15:26:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:19 smithi060 ceph-mon[131518]: pgmap v372: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2023-11-27T15:26:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:19 smithi008 ceph-mon[128788]: pgmap v372: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2023-11-27T15:26:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:19 smithi029 ceph-mon[131166]: pgmap v372: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2023-11-27T15:26:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.657+0000 7f283cf9d700 1 -- 172.21.15.8:0/1144305106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2838101ca0 msgr2=0x7f2838108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:20.661 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.657+0000 7f283cf9d700 1 --2- 172.21.15.8:0/1144305106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2838101ca0 0x7f2838108540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2830009f90 tx=0x7f283000b330 comp rx=0 tx=0).stop 2023-11-27T15:26:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.657+0000 7f283cf9d700 1 -- 172.21.15.8:0/1144305106 shutdown_connections 2023-11-27T15:26:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.657+0000 7f283cf9d700 1 --2- 172.21.15.8:0/1144305106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2838101ca0 0x7f2838108540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.657+0000 7f283cf9d700 1 --2- 172.21.15.8:0/1144305106 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838100060 0x7f2838100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.657+0000 7f283cf9d700 1 --2- 172.21.15.8:0/1144305106 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28381012f0 0x7f2838101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.657+0000 7f283cf9d700 1 -- 172.21.15.8:0/1144305106 >> 172.21.15.8:0/1144305106 conn(0x7f28380fb640 msgr2=0x7f28380fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.657+0000 7f283cf9d700 1 -- 172.21.15.8:0/1144305106 shutdown_connections 2023-11-27T15:26:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.657+0000 7f283cf9d700 1 -- 172.21.15.8:0/1144305106 wait complete. 2023-11-27T15:26:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.659+0000 7f283cf9d700 1 Processor -- start 2023-11-27T15:26:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.659+0000 7f283cf9d700 1 -- start start 2023-11-27T15:26:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f283cf9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2838100060 0x7f28381980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f283cf9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28381012f0 0x7f28381985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f283cf9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838101ca0 0x7f283819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f283cf9d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819fc20 con 0x7f2838100060 2023-11-27T15:26:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f283cf9d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819fd60 con 0x7f2838101ca0 2023-11-27T15:26:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f283cf9d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283819fef0 con 0x7f28381012f0 2023-11-27T15:26:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f28377fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2838100060 0x7f28381980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f28377fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2838100060 0x7f28381980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55574/0 (socket says 172.21.15.8:55574) 2023-11-27T15:26:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f28377fe700 1 -- 172.21.15.8:0/4048913439 learned_addr learned my addr 172.21.15.8:0/4048913439 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f2836ffd700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28381012f0 0x7f28381985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.660+0000 7f2837fff700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838101ca0 0x7f283819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f28377fe700 1 -- 172.21.15.8:0/4048913439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28381012f0 msgr2=0x7f28381985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f28377fe700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28381012f0 0x7f28381985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f28377fe700 1 -- 172.21.15.8:0/4048913439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838101ca0 msgr2=0x7f283819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f28377fe700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838101ca0 0x7f283819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f28377fe700 1 -- 172.21.15.8:0/4048913439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28280095f0 con 0x7f2838100060 2023-11-27T15:26:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f2837fff700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838101ca0 0x7f283819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f2836ffd700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28381012f0 0x7f28381985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f28377fe700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2838100060 0x7f28381980a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f282800ec10 tx=0x7f282800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f2834ff9700 1 -- 172.21.15.8:0/4048913439 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828009a10 con 0x7f2838100060 2023-11-27T15:26:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f2834ff9700 1 -- 172.21.15.8:0/4048913439 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28280044e0 con 0x7f2838100060 2023-11-27T15:26:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f2834ff9700 1 -- 172.21.15.8:0/4048913439 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828010490 con 0x7f2838100060 2023-11-27T15:26:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.661+0000 7f283cf9d700 1 -- 172.21.15.8:0/4048913439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2830009c40 con 0x7f2838100060 2023-11-27T15:26:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.662+0000 7f283cf9d700 1 -- 172.21.15.8:0/4048913439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283810ba60 con 0x7f2838100060 2023-11-27T15:26:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.667+0000 7f2834ff9700 1 -- 172.21.15.8:0/4048913439 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2828003680 con 0x7f2838100060 2023-11-27T15:26:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.669+0000 7f2851a48b80 1 Processor -- start 2023-11-27T15:26:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.669+0000 7f2834ff9700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f28200a1150 0x7f28200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.669+0000 7f2834ff9700 1 -- 172.21.15.8:0/4048913439 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2828014070 con 0x7f2838100060 2023-11-27T15:26:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.669+0000 7f2851a48b80 1 -- start start 2023-11-27T15:26:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.669+0000 7f2837fff700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f28200a1150 0x7f28200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.669+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c91300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.669+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0c8ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c90970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f2851a48b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0b8d000 con 0x560af0c90500 2023-11-27T15:26:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f2851a48b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0bdf840 con 0x560af0c91840 2023-11-27T15:26:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f2851a48b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0bd8cd0 con 0x560af0ae36b0 2023-11-27T15:26:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f28377fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c91300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f28377fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c91300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:34278/0 (socket says 172.21.15.8:34278) 2023-11-27T15:26:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f28377fe700 1 -- 172.21.15.8:0/3368391821 learned_addr learned my addr 172.21.15.8:0/3368391821 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f2836ffd700 1 --2- 172.21.15.8:0/3368391821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0c8ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f28377fe700 1 -- 172.21.15.8:0/3368391821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 msgr2=0x560af0c8ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f28377fe700 1 --2- 172.21.15.8:0/3368391821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0c8ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f28377fe700 1 -- 172.21.15.8:0/3368391821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 msgr2=0x560af0c90970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:26:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f28377fe700 1 --2- 172.21.15.8:0/3368391821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c90970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f28377fe700 1 -- 172.21.15.8:0/3368391821 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af0a5a5d0 con 0x560af0ae36b0 2023-11-27T15:26:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.670+0000 7f2836ffd700 1 --2- 172.21.15.8:0/3368391821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0c8ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.671+0000 7f28377fe700 1 --2- 172.21.15.8:0/3368391821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c91300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f28280a1b60 tx=0x7f2828099780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e189690d827dffe2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.671+0000 7f2837fff700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f28200a1150 0x7f28200a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x560af0c830b0 tx=0x7f2830006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.671+0000 7f281dffb700 1 -- 172.21.15.8:0/3368391821 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28280b1070 con 0x560af0ae36b0 2023-11-27T15:26:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.671+0000 7f281dffb700 1 -- 172.21.15.8:0/3368391821 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f282809d070 con 0x560af0ae36b0 2023-11-27T15:26:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.671+0000 7f281dffb700 1 -- 172.21.15.8:0/3368391821 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28280b2220 con 0x560af0ae36b0 2023-11-27T15:26:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.672+0000 7f2851a48b80 1 -- 172.21.15.8:0/3368391821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 msgr2=0x560af0c91300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.672+0000 7f2851a48b80 1 --2- 172.21.15.8:0/3368391821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c91300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f28280a1b60 tx=0x7f2828099780 comp rx=0 tx=0).stop 2023-11-27T15:26:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.672+0000 7f2851a48b80 1 -- 172.21.15.8:0/3368391821 shutdown_connections 2023-11-27T15:26:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.672+0000 7f2851a48b80 1 --2- 172.21.15.8:0/3368391821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c91300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.672+0000 7f2851a48b80 1 --2- 172.21.15.8:0/3368391821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c90970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.672+0000 7f2851a48b80 1 --2- 172.21.15.8:0/3368391821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0c8ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.672+0000 7f2851a48b80 1 -- 172.21.15.8:0/3368391821 >> 172.21.15.8:0/3368391821 conn(0x560af0c7d1a0 msgr2=0x560af0c822e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.672+0000 7f2851a48b80 1 -- 172.21.15.8:0/3368391821 shutdown_connections 2023-11-27T15:26:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.672+0000 7f2851a48b80 1 -- 172.21.15.8:0/3368391821 wait complete. 2023-11-27T15:26:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.673+0000 7f2851a48b80 1 -- start start 2023-11-27T15:26:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.673+0000 7f2851a48b80 1 Processor -- start 2023-11-27T15:26:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.673+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c996e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.673+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c99c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.673+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0ca0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f2851a48b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0b7a880 con 0x560af0c90500 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f2851a48b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0b7a520 con 0x560af0c91840 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f2851a48b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0a5c8c0 con 0x560af0ae36b0 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f2851a48b80 10 client.0 _ll_register_callbacks cb 0x560af0b98720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f28377fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f28377fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:34288/0 (socket says 172.21.15.8:34288) 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f28377fe700 1 -- 172.21.15.8:0/422391474 learned_addr learned my addr 172.21.15.8:0/422391474 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f2837fff700 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0ca0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f2836ffd700 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c99c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f28377fe700 1 -- 172.21.15.8:0/422391474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 msgr2=0x560af0ca0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.674+0000 7f28377fe700 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0ca0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.675+0000 7f28377fe700 1 -- 172.21.15.8:0/422391474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 msgr2=0x560af0c99c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.675+0000 7f28377fe700 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c99c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.675+0000 7f28377fe700 1 -- 172.21.15.8:0/422391474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283810ba60 con 0x560af0ae36b0 2023-11-27T15:26:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.675+0000 7f2836ffd700 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c99c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.675+0000 7f28377fe700 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c996e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2828099940 tx=0x7f282800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a867be51c73d04 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.675+0000 7f2816ffd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:26:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.675+0000 7f2816ffd700 1 -- 172.21.15.8:0/422391474 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28280b8070 con 0x560af0ae36b0 2023-11-27T15:26:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.675+0000 7f2851a48b80 1 -- 172.21.15.8:0/422391474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560af0c7a960 con 0x560af0ae36b0 2023-11-27T15:26:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.676+0000 7f2816ffd700 1 -- 172.21.15.8:0/422391474 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f28280b2070 con 0x560af0ae36b0 2023-11-27T15:26:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.676+0000 7f2816ffd700 1 -- 172.21.15.8:0/422391474 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28280b1030 con 0x560af0ae36b0 2023-11-27T15:26:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.676+0000 7f2851a48b80 10 client.25438 fetch_fsmap learned FSMap version 24 2023-11-27T15:26:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.676+0000 7f2851a48b80 1 -- 172.21.15.8:0/422391474 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560af0b9f130 con 0x560af0ae36b0 2023-11-27T15:26:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.676+0000 7f2816ffd700 1 -- 172.21.15.8:0/422391474 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560af0c7a960 con 0x560af0ae36b0 2023-11-27T15:26:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.676+0000 7f2816ffd700 1 -- 172.21.15.8:0/422391474 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f28280a1c90 con 0x560af0ae36b0 2023-11-27T15:26:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.676+0000 7f2851a48b80 10 client.25438 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:26:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.676+0000 7f2851a48b80 10 client.25438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:26:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.677+0000 7f2851a48b80 20 client.25438 populate_metadata read hostname 'smithi008' 2023-11-27T15:26:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.677+0000 7f2851a48b80 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560af0ca5130 0x560af0ca7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.677+0000 7f2851a48b80 4 client.25438 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.677+0000 7f2851a48b80 1 -- 172.21.15.8:0/422391474 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560af0b9b040 con 0x560af0ca5130 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.677+0000 7f2836ffd700 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560af0ca5130 0x560af0ca7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.678+0000 7f2836ffd700 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560af0ca5130 0x560af0ca7540 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432df9fbf23740b4 server_cookie=d1ab9e13ed835557 in_seq=0 out_seq=0 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.678+0000 7f2816ffd700 10 client.25438 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.678+0000 7f2816ffd700 1 -- 172.21.15.8:0/422391474 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560af0b9b040 con 0x560af0ca5130 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.678+0000 7f2816ffd700 10 client.25438 handle_command_reply: tid=0 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.678+0000 7f2851a48b80 1 client.25438 shutdown 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2851a48b80 1 -- 172.21.15.8:0/422391474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 msgr2=0x560af0c996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2851a48b80 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c996e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2828099940 tx=0x7f282800c850 comp rx=0 tx=0).stop 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2851a48b80 1 -- 172.21.15.8:0/422391474 shutdown_connections 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2851a48b80 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560af0ca5130 0x560af0ca7540 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2851a48b80 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0ae36b0 0x560af0c996e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2816ffd700 0 client.25438 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2851a48b80 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0ca0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2851a48b80 1 --2- 172.21.15.8:0/422391474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c99c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2851a48b80 1 -- 172.21.15.8:0/422391474 >> 172.21.15.8:0/422391474 conn(0x560af0c7d1a0 msgr2=0x560af0c81a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2851a48b80 1 -- 172.21.15.8:0/422391474 shutdown_connections 2023-11-27T15:26:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.679+0000 7f2851a48b80 1 -- 172.21.15.8:0/422391474 wait complete. 2023-11-27T15:26:20.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.680+0000 7f2851a48b80 20 client.25438 trim_cache size 0 max 16384 2023-11-27T15:26:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.695+0000 7f2851a48b80 1 Processor -- start 2023-11-27T15:26:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.695+0000 7f2851a48b80 1 -- start start 2023-11-27T15:26:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0ae36b0 0x560af0c98d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c90500 0x560af0ca0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0c983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f2851a48b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0c98b90 con 0x560af0ae36b0 2023-11-27T15:26:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f2851a48b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0c82dc0 con 0x560af0c91840 2023-11-27T15:26:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f2851a48b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0c82be0 con 0x560af0c90500 2023-11-27T15:26:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f28377fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0ae36b0 0x560af0c98d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f2836ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c90500 0x560af0ca0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f2837fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0c983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f2836ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c90500 0x560af0ca0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:34294/0 (socket says 172.21.15.8:34294) 2023-11-27T15:26:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.696+0000 7f2836ffd700 1 -- 172.21.15.8:0/484752537 learned_addr learned my addr 172.21.15.8:0/484752537 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.697+0000 7f2836ffd700 1 -- 172.21.15.8:0/484752537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 msgr2=0x560af0c983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.697+0000 7f2836ffd700 1 --2- 172.21.15.8:0/484752537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0c983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.697+0000 7f2836ffd700 1 -- 172.21.15.8:0/484752537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0ae36b0 msgr2=0x560af0c98d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.697+0000 7f2836ffd700 1 --2- 172.21.15.8:0/484752537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0ae36b0 0x560af0c98d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.697+0000 7f2836ffd700 1 -- 172.21.15.8:0/484752537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af0b9ee80 con 0x560af0c90500 2023-11-27T15:26:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.697+0000 7f2836ffd700 1 --2- 172.21.15.8:0/484752537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c90500 0x560af0ca0c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x560af0ca7df0 tx=0x7f282c0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4773d4b3ed54e7ec server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.697+0000 7f281e7fc700 1 -- 172.21.15.8:0/484752537 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c018040 con 0x560af0c90500 2023-11-27T15:26:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.697+0000 7f281e7fc700 1 -- 172.21.15.8:0/484752537 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f282c0055a0 con 0x560af0c90500 2023-11-27T15:26:20.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.697+0000 7f281e7fc700 1 -- 172.21.15.8:0/484752537 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c004a20 con 0x560af0c90500 2023-11-27T15:26:20.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.698+0000 7f2851a48b80 1 -- 172.21.15.8:0/484752537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c90500 msgr2=0x560af0ca0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.698+0000 7f2851a48b80 1 --2- 172.21.15.8:0/484752537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c90500 0x560af0ca0c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x560af0ca7df0 tx=0x7f282c0058e0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.698+0000 7f2851a48b80 1 -- 172.21.15.8:0/484752537 shutdown_connections 2023-11-27T15:26:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.698+0000 7f2851a48b80 1 --2- 172.21.15.8:0/484752537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0c91840 0x560af0c983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.698+0000 7f2851a48b80 1 --2- 172.21.15.8:0/484752537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0ae36b0 0x560af0c98d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.698+0000 7f2851a48b80 1 --2- 172.21.15.8:0/484752537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c90500 0x560af0ca0c20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.698+0000 7f2851a48b80 1 -- 172.21.15.8:0/484752537 >> 172.21.15.8:0/484752537 conn(0x560af0c7d1a0 msgr2=0x560af0cabea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.698+0000 7f2851a48b80 1 -- 172.21.15.8:0/484752537 shutdown_connections 2023-11-27T15:26:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.698+0000 7f2851a48b80 1 -- 172.21.15.8:0/484752537 wait complete. 2023-11-27T15:26:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.699+0000 7f2851a48b80 1 -- start start 2023-11-27T15:26:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.699+0000 7f2851a48b80 1 Processor -- start 2023-11-27T15:26:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.699+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0ae36b0 0x560af0c9c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.699+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c9cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.699+0000 7f2851a48b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c91840 0x560af0c9fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.699+0000 7f2851a48b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0ca0160 con 0x560af0c90500 2023-11-27T15:26:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.699+0000 7f2851a48b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0ca02d0 con 0x560af0ae36b0 2023-11-27T15:26:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.699+0000 7f2851a48b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af0ca0490 con 0x560af0c91840 2023-11-27T15:26:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.699+0000 7f2851a48b80 10 client.0 _ll_register_callbacks cb 0x7f2828099a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f28377fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0ae36b0 0x560af0c9c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f28377fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0ae36b0 0x560af0c9c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:42482/0 (socket says 172.21.15.8:42482) 2023-11-27T15:26:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f28377fe700 1 -- 172.21.15.8:0/2992591740 learned_addr learned my addr 172.21.15.8:0/2992591740 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f2837fff700 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c91840 0x560af0c9fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f2836ffd700 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c9cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f28377fe700 1 -- 172.21.15.8:0/2992591740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c91840 msgr2=0x560af0c9fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f28377fe700 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c91840 0x560af0c9fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f28377fe700 1 -- 172.21.15.8:0/2992591740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 msgr2=0x560af0c9cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f28377fe700 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c9cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f28377fe700 1 -- 172.21.15.8:0/2992591740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c01a860 con 0x560af0ae36b0 2023-11-27T15:26:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f28377fe700 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0ae36b0 0x560af0c9c6c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28280966b0 tx=0x7f2828099ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b1bf0c7e24bb856 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.700+0000 7f2816ffd700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:26:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2816ffd700 1 -- 172.21.15.8:0/2992591740 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28280b1070 con 0x560af0ae36b0 2023-11-27T15:26:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2816ffd700 1 -- 172.21.15.8:0/2992591740 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f2828004210 con 0x560af0ae36b0 2023-11-27T15:26:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2851a48b80 1 -- 172.21.15.8:0/2992591740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f282809d070 con 0x560af0ae36b0 2023-11-27T15:26:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2816ffd700 1 -- 172.21.15.8:0/2992591740 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28280b8040 con 0x560af0ae36b0 2023-11-27T15:26:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2851a48b80 10 client.25421 fetch_fsmap learned FSMap version 24 2023-11-27T15:26:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2851a48b80 1 -- 172.21.15.8:0/2992591740 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560af09efc00 con 0x560af0ae36b0 2023-11-27T15:26:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2816ffd700 1 -- 172.21.15.8:0/2992591740 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f282809d070 con 0x560af0ae36b0 2023-11-27T15:26:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2816ffd700 1 -- 172.21.15.8:0/2992591740 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2828098ba0 con 0x560af0ae36b0 2023-11-27T15:26:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2851a48b80 10 client.25421 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:26:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2851a48b80 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:26:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2851a48b80 20 client.25421 populate_metadata read hostname 'smithi008' 2023-11-27T15:26:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2851a48b80 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560af0cf9790 0x560af0cfbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2851a48b80 4 client.25421 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:26:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.701+0000 7f2851a48b80 1 -- 172.21.15.8:0/2992591740 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560af0ba7600 con 0x560af0cf9790 2023-11-27T15:26:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.702+0000 7f2836ffd700 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560af0cf9790 0x560af0cfbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.702+0000 7f2836ffd700 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560af0cf9790 0x560af0cfbba0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cac7682e27f707c9 server_cookie=4afe64405ad74fab in_seq=0 out_seq=0 2023-11-27T15:26:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.702+0000 7f2816ffd700 10 client.25421 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2816ffd700 1 -- 172.21.15.8:0/2992591740 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560af0ba7600 con 0x560af0cf9790 2023-11-27T15:26:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2816ffd700 10 client.25421 handle_command_reply: tid=0 2023-11-27T15:26:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2851a48b80 1 client.25421 shutdown 2023-11-27T15:26:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2851a48b80 1 -- 172.21.15.8:0/2992591740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0ae36b0 msgr2=0x560af0c9c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2851a48b80 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0ae36b0 0x560af0c9c6c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f28280966b0 tx=0x7f2828099ac0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2851a48b80 1 -- 172.21.15.8:0/2992591740 shutdown_connections 2023-11-27T15:26:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2851a48b80 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560af0cf9790 0x560af0cfbba0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2851a48b80 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560af0c91840 0x560af0c9fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2851a48b80 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560af0ae36b0 0x560af0c9c6c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2816ffd700 0 client.25421 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2851a48b80 1 --2- 172.21.15.8:0/2992591740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560af0c90500 0x560af0c9cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.710+0000 7f2851a48b80 1 -- 172.21.15.8:0/2992591740 >> 172.21.15.8:0/2992591740 conn(0x560af0c7d1a0 msgr2=0x560af0c983d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.711+0000 7f2851a48b80 1 -- 172.21.15.8:0/2992591740 shutdown_connections 2023-11-27T15:26:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.711+0000 7f2851a48b80 1 -- 172.21.15.8:0/2992591740 wait complete. 2023-11-27T15:26:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.711+0000 7f2851a48b80 20 client.25421 trim_cache size 0 max 16384 2023-11-27T15:26:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:26:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:26:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:26:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:26:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.711+0000 7f283cf9d700 1 -- 172.21.15.8:0/4048913439 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f28200a1150 msgr2=0x7f28200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.711+0000 7f283cf9d700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f28200a1150 0x7f28200a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x560af0c830b0 tx=0x7f2830006040 comp rx=0 tx=0).stop 2023-11-27T15:26:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.711+0000 7f283cf9d700 1 -- 172.21.15.8:0/4048913439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2838100060 msgr2=0x7f28381980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.711+0000 7f283cf9d700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2838100060 0x7f28381980a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f282800ec10 tx=0x7f282800c2d0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.712+0000 7f283cf9d700 1 -- 172.21.15.8:0/4048913439 shutdown_connections 2023-11-27T15:26:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.712+0000 7f283cf9d700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2838101ca0 0x7f283819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.712+0000 7f283cf9d700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2838100060 0x7f28381980a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.712+0000 7f283cf9d700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f28200a1150 0x7f28200a3600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.712+0000 7f283cf9d700 1 --2- 172.21.15.8:0/4048913439 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f28381012f0 0x7f28381985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.712+0000 7f283cf9d700 1 -- 172.21.15.8:0/4048913439 >> 172.21.15.8:0/4048913439 conn(0x7f28380fb640 msgr2=0x7f2838106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.712+0000 7f283cf9d700 1 -- 172.21.15.8:0/4048913439 shutdown_connections 2023-11-27T15:26:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:20.712+0000 7f283cf9d700 1 -- 172.21.15.8:0/4048913439 wait complete. 2023-11-27T15:26:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:21 smithi060 ceph-mon[131518]: pgmap v373: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-11-27T15:26:21.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:21 smithi008 ceph-mon[128788]: pgmap v373: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-11-27T15:26:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:21 smithi029 ceph-mon[131166]: pgmap v373: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-11-27T15:26:23.615 INFO:tasks.workunit.client.0.smithi008.stderr:Updating files: 68% (8233/12072) Updating files: 69% (8330/12072) Updating files: 70% (8451/12072) Updating files: 71% (8572/12072) Updating files: 72% (8692/12072) Updating files: 73% (8813/12072) Updating files: 74% (8934/12072) Updating files: 75% (9054/12072) Updating files: 76% (9175/12072) Updating files: 77% (9296/12072) Updating files: 78% (9417/12072) Updating files: 79% (9537/12072) Updating files: 80% (9658/12072) Updating files: 81% (9779/12072) Updating files: 82% (9900/12072) Updating files: 83% (10020/12072) Updating files: 84% (10141/12072) Updating files: 85% (10262/12072) Updating files: 86% (10382/12072) Updating files: 87% (10503/12072) Updating files: 88% (10624/12072) Updating files: 89% (10745/12072) Updating files: 90% (10865/12072) Updating files: 91% (10986/12072) Updating files: 92% (11107/12072) Updating files: 93% (11227/12072) Updating files: 94% (11348/12072) Updating files: 95% (11469/12072) Updating files: 96% (11590/12072) Updating files: 97% (11710/12072) Updating files: 98% (11831/12072) Updating files: 99% (11952/12072) Updating files: 100% (12072/12072) Updating files: 100% (12072/12072), done. 2023-11-27T15:26:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:23 smithi060 ceph-mon[131518]: pgmap v374: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:26:23.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:23 smithi008 ceph-mon[128788]: pgmap v374: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:26:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:23 smithi029 ceph-mon[131166]: pgmap v374: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:26:24.529 INFO:tasks.workunit.client.0.smithi008.stderr:Note: switching to '5e4a743ad7fd1422ebd6c4dc950a21a303279207'. 2023-11-27T15:26:24.529 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-11-27T15:26:24.529 INFO:tasks.workunit.client.0.smithi008.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-11-27T15:26:24.529 INFO:tasks.workunit.client.0.smithi008.stderr:changes and commit them, and you can discard any commits you make in this 2023-11-27T15:26:24.529 INFO:tasks.workunit.client.0.smithi008.stderr:state without impacting any branches by switching back to a branch. 2023-11-27T15:26:24.529 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-11-27T15:26:24.529 INFO:tasks.workunit.client.0.smithi008.stderr:If you want to create a new branch to retain commits you create, you may 2023-11-27T15:26:24.529 INFO:tasks.workunit.client.0.smithi008.stderr:do so (now or later) by using -c with the switch command. Example: 2023-11-27T15:26:24.530 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-11-27T15:26:24.530 INFO:tasks.workunit.client.0.smithi008.stderr: git switch -c 2023-11-27T15:26:24.530 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-11-27T15:26:24.530 INFO:tasks.workunit.client.0.smithi008.stderr:Or undo this operation with: 2023-11-27T15:26:24.530 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-11-27T15:26:24.530 INFO:tasks.workunit.client.0.smithi008.stderr: git switch - 2023-11-27T15:26:24.530 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-11-27T15:26:24.530 INFO:tasks.workunit.client.0.smithi008.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-11-27T15:26:24.530 INFO:tasks.workunit.client.0.smithi008.stderr: 2023-11-27T15:26:24.530 INFO:tasks.workunit.client.0.smithi008.stderr:HEAD is now at 5e4a743ad7f doc/cephfs: Update docs to match remove functionality and respective vxattrs 2023-11-27T15:26:24.539 DEBUG:teuthology.orchestra.run.smithi008:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-11-27T15:26:24.595 INFO:tasks.workunit.client.0.smithi008.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-11-27T15:26:24.600 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-27T15:26:24.600 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-11-27T15:26:25.164 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-11-27T15:26:25.284 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-11-27T15:26:25.349 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-27T15:26:25.353 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-27T15:26:25.353 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-11-27T15:26:25.418 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-27T15:26:25.426 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:26:25.426 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-11-27T15:26:25.470 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2023-11-27T15:26:25.471 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2023-11-27T15:26:25.471 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2023-11-27T15:26:25.567 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2023-11-27T15:26:25.567 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir -p testdir 2023-11-27T15:26:25.571 INFO:tasks.workunit.client.0.smithi008.stderr:+ cd testdir 2023-11-27T15:26:25.571 INFO:tasks.workunit.client.0.smithi008.stderr:+ set +e 2023-11-27T15:26:25.571 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfacl -d -m u:nobody:rw . 2023-11-27T15:26:25.583 INFO:tasks.workunit.client.0.smithi008.stderr:+ test 0 '!=' 0 2023-11-27T15:26:25.583 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2023-11-27T15:26:25.583 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=0 2023-11-27T15:26:25.583 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 0 -lt 100 ']' 2023-11-27T15:26:25.584 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 0 + 1 2023-11-27T15:26:25.585 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=1 2023-11-27T15:26:25.586 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:25.591 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:25.591 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:25.598 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:25.601 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:25.602 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:25.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:25 smithi060 ceph-mon[131518]: pgmap v375: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:26:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:25 smithi008 ceph-mon[128788]: pgmap v375: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:26:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:25 smithi029 ceph-mon[131166]: pgmap v375: 129 pgs: 129 active+clean; 665 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:26:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:27 smithi060 ceph-mon[131518]: pgmap v376: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-11-27T15:26:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:27 smithi008 ceph-mon[128788]: pgmap v376: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-11-27T15:26:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:27 smithi029 ceph-mon[131166]: pgmap v376: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-11-27T15:26:28.554 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:28.554 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:28.706 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:28.706 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:28.719 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:28.719 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:28.719 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 1 -lt 100 ']' 2023-11-27T15:26:28.720 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 1 + 1 2023-11-27T15:26:28.760 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=2 2023-11-27T15:26:28.760 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:28.796 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:28.797 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:28.799 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:28.800 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:28.800 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:29.253 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:29.253 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:29.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:29 smithi008 ceph-mon[128788]: pgmap v377: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:26:29.365 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:29.365 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:29.376 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:29.376 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:29.376 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -lt 100 ']' 2023-11-27T15:26:29.377 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 2 + 1 2023-11-27T15:26:29.409 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=3 2023-11-27T15:26:29.410 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:29.441 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:29.441 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:29.443 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:29.444 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:29.444 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:29.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:29 smithi060 ceph-mon[131518]: pgmap v377: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:26:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:29 smithi029 ceph-mon[131166]: pgmap v377: 129 pgs: 129 active+clean; 666 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:26:29.828 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:29.828 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:29.923 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:29.923 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:29.935 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:29.935 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:29.935 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 3 -lt 100 ']' 2023-11-27T15:26:29.936 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 3 + 1 2023-11-27T15:26:29.968 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=4 2023-11-27T15:26:29.968 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:29.995 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:29.996 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:29.998 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:29.999 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:29.999 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:30.420 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:30.420 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:30.531 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:30.531 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:30.568 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:30.569 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:30.569 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 4 -lt 100 ']' 2023-11-27T15:26:30.569 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 4 + 1 2023-11-27T15:26:30.606 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=5 2023-11-27T15:26:30.607 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:30.679 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:30.680 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:30.682 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:30.683 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:30.683 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:31.103 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:31.103 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:31.231 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:31.231 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:31.244 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:31.244 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:31.244 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 5 -lt 100 ']' 2023-11-27T15:26:31.244 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 5 + 1 2023-11-27T15:26:31.276 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=6 2023-11-27T15:26:31.277 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:31.322 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:31.323 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:31.325 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:31.326 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:31.326 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:31.683 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:31 smithi008 ceph-mon[128788]: pgmap v378: 129 pgs: 129 active+clean; 679 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2023-11-27T15:26:31.745 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:31.745 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:31 smithi060 ceph-mon[131518]: pgmap v378: 129 pgs: 129 active+clean; 679 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2023-11-27T15:26:31.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:31.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:31.885 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:31.885 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:31.885 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 6 -lt 100 ']' 2023-11-27T15:26:31.885 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 6 + 1 2023-11-27T15:26:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:31 smithi029 ceph-mon[131166]: pgmap v378: 129 pgs: 129 active+clean; 679 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2023-11-27T15:26:31.917 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=7 2023-11-27T15:26:31.918 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:31.971 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:31.972 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:31.974 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:31.974 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:31.975 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:32.403 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:32.403 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:32.531 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:32.531 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:32.543 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:32.544 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:32.544 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 7 -lt 100 ']' 2023-11-27T15:26:32.544 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 7 + 1 2023-11-27T15:26:32.576 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=8 2023-11-27T15:26:32.576 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:32.631 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:32.631 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:32.633 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:32.634 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:32.634 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:33.111 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:33.111 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:33.256 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:33.256 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:33.267 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:33.267 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:33.268 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 8 -lt 100 ']' 2023-11-27T15:26:33.268 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 8 + 1 2023-11-27T15:26:33.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=9 2023-11-27T15:26:33.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:33.346 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:33.347 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:33.349 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:33.350 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:33.350 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:33.768 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:33 smithi008 ceph-mon[128788]: pgmap v379: 129 pgs: 129 active+clean; 679 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2023-11-27T15:26:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:33 smithi060 ceph-mon[131518]: pgmap v379: 129 pgs: 129 active+clean; 679 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2023-11-27T15:26:33.869 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:33.869 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:33 smithi029 ceph-mon[131166]: pgmap v379: 129 pgs: 129 active+clean; 679 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2023-11-27T15:26:33.964 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:33.964 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:33.984 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:33.984 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:33.985 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 9 -lt 100 ']' 2023-11-27T15:26:33.985 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 9 + 1 2023-11-27T15:26:34.039 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=10 2023-11-27T15:26:34.040 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:34.088 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:34.089 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:34.091 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:34.091 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:34.092 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:34.747 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:34.748 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:34.880 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:34.880 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:34.892 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:34.892 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:34.892 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 10 -lt 100 ']' 2023-11-27T15:26:34.893 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 10 + 1 2023-11-27T15:26:34.939 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=11 2023-11-27T15:26:34.940 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:35.042 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:35.042 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:35.045 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:35.046 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:35.046 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:35.689 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:35 smithi008 ceph-mon[128788]: pgmap v380: 129 pgs: 129 active+clean; 679 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-11-27T15:26:35.777 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:35.778 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:35 smithi060 ceph-mon[131518]: pgmap v380: 129 pgs: 129 active+clean; 679 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-11-27T15:26:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:35 smithi029 ceph-mon[131166]: pgmap v380: 129 pgs: 129 active+clean; 679 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-11-27T15:26:36.107 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:36.107 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.121+0000 7f21297ff700 1 -- 172.21.15.8:0/3749367567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2124100230 msgr2=0x7f2124100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.121+0000 7f21297ff700 1 --2- 172.21.15.8:0/3749367567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2124100230 0x7f2124100640 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2114009990 tx=0x7f2114009ca0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.121+0000 7f21297ff700 1 -- 172.21.15.8:0/3749367567 shutdown_connections 2023-11-27T15:26:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.121+0000 7f21297ff700 1 --2- 172.21.15.8:0/3749367567 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2124101e70 0x7f2124108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.121+0000 7f21297ff700 1 --2- 172.21.15.8:0/3749367567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2124100230 0x7f2124100640 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.121+0000 7f21297ff700 1 --2- 172.21.15.8:0/3749367567 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f21241014c0 0x7f2124101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.121+0000 7f21297ff700 1 -- 172.21.15.8:0/3749367567 >> 172.21.15.8:0/3749367567 conn(0x7f21240fb810 msgr2=0x7f21240fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.121+0000 7f21297ff700 1 -- 172.21.15.8:0/3749367567 shutdown_connections 2023-11-27T15:26:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.121+0000 7f21297ff700 1 -- 172.21.15.8:0/3749367567 wait complete. 2023-11-27T15:26:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.122+0000 7f21297ff700 1 Processor -- start 2023-11-27T15:26:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.122+0000 7f21297ff700 1 -- start start 2023-11-27T15:26:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f21297ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2124100230 0x7f21241981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f21297ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21241014c0 0x7f2124198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f21297ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2124101e70 0x7f212419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f21297ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212419fd70 con 0x7f2124100230 2023-11-27T15:26:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f21297ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212419feb0 con 0x7f2124101e70 2023-11-27T15:26:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f21297ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21241a0040 con 0x7f21241014c0 2023-11-27T15:26:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f2122ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2124100230 0x7f21241981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f2122ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2124100230 0x7f21241981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54716/0 (socket says 172.21.15.8:54716) 2023-11-27T15:26:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f2122ffd700 1 -- 172.21.15.8:0/3056683051 learned_addr learned my addr 172.21.15.8:0/3056683051 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f211bfff700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21241014c0 0x7f2124198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f21237fe700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2124101e70 0x7f212419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f2122ffd700 1 -- 172.21.15.8:0/3056683051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21241014c0 msgr2=0x7f2124198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f2122ffd700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21241014c0 0x7f2124198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f2122ffd700 1 -- 172.21.15.8:0/3056683051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2124101e70 msgr2=0x7f212419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f2122ffd700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2124101e70 0x7f212419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.123+0000 7f2122ffd700 1 -- 172.21.15.8:0/3056683051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21140095f0 con 0x7f2124100230 2023-11-27T15:26:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.124+0000 7f2122ffd700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2124100230 0x7f21241981f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f211400b680 tx=0x7f2114005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.124+0000 7f2120ff9700 1 -- 172.21.15.8:0/3056683051 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211401d070 con 0x7f2124100230 2023-11-27T15:26:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.124+0000 7f21297ff700 1 -- 172.21.15.8:0/3056683051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212410bc50 con 0x7f2124100230 2023-11-27T15:26:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.124+0000 7f2120ff9700 1 -- 172.21.15.8:0/3056683051 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2114022470 con 0x7f2124100230 2023-11-27T15:26:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.124+0000 7f2120ff9700 1 -- 172.21.15.8:0/3056683051 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114017650 con 0x7f2124100230 2023-11-27T15:26:36.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.124+0000 7f21297ff700 1 -- 172.21.15.8:0/3056683051 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212410bac0 con 0x7f2124100230 2023-11-27T15:26:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.127+0000 7f2120ff9700 1 -- 172.21.15.8:0/3056683051 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f21140225e0 con 0x7f2124100230 2023-11-27T15:26:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.129+0000 7f2120ff9700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f21100a1150 0x7f21100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.129+0000 7f2120ff9700 1 -- 172.21.15.8:0/3056683051 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f21140ce150 con 0x7f2124100230 2023-11-27T15:26:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.129+0000 7f211bfff700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f21100a1150 0x7f21100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.130+0000 7f211bfff700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f21100a1150 0x7f21100a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f210c009c60 tx=0x7f210c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:36.135 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:36.136 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:36.136 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 11 -lt 100 ']' 2023-11-27T15:26:36.136 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 11 + 1 2023-11-27T15:26:36.206 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=12 2023-11-27T15:26:36.206 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.282+0000 7f213d10ab80 1 Processor -- start 2023-11-27T15:26:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.282+0000 7f213d10ab80 1 -- start start 2023-11-27T15:26:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 0x5600027d4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d4db0 0x5600027d55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d5b10 0x5600027d5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f213d10ab80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600026da000 con 0x5600027d5b10 2023-11-27T15:26:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f213d10ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000272c840 con 0x5600027d4db0 2023-11-27T15:26:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f213d10ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560002725cd0 con 0x5600026306b0 2023-11-27T15:26:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f21237fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d5b10 0x5600027d5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f21237fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d5b10 0x5600027d5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54720/0 (socket says 172.21.15.8:54720) 2023-11-27T15:26:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f21237fe700 1 -- 172.21.15.8:0/889898174 learned_addr learned my addr 172.21.15.8:0/889898174 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f211bfff700 1 --2- 172.21.15.8:0/889898174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d4db0 0x5600027d55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f2122ffd700 1 --2- 172.21.15.8:0/889898174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 0x5600027d4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f21237fe700 1 -- 172.21.15.8:0/889898174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 msgr2=0x5600027d4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f21237fe700 1 --2- 172.21.15.8:0/889898174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 0x5600027d4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f21237fe700 1 -- 172.21.15.8:0/889898174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d4db0 msgr2=0x5600027d55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f21237fe700 1 --2- 172.21.15.8:0/889898174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d4db0 0x5600027d55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.283+0000 7f21237fe700 1 -- 172.21.15.8:0/889898174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600025a75d0 con 0x5600027d5b10 2023-11-27T15:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.284+0000 7f21237fe700 1 --2- 172.21.15.8:0/889898174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d5b10 0x5600027d5f80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f211c011680 tx=0x7f211c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d925c4ad1895e41 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.284+0000 7f21197fa700 1 -- 172.21.15.8:0/889898174 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c00e040 con 0x5600027d5b10 2023-11-27T15:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.284+0000 7f21197fa700 1 -- 172.21.15.8:0/889898174 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f211c00f040 con 0x5600027d5b10 2023-11-27T15:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.284+0000 7f21197fa700 1 -- 172.21.15.8:0/889898174 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c004880 con 0x5600027d5b10 2023-11-27T15:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.285+0000 7f213d10ab80 1 -- 172.21.15.8:0/889898174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d5b10 msgr2=0x5600027d5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.285+0000 7f213d10ab80 1 --2- 172.21.15.8:0/889898174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d5b10 0x5600027d5f80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f211c011680 tx=0x7f211c009920 comp rx=0 tx=0).stop 2023-11-27T15:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.285+0000 7f213d10ab80 1 -- 172.21.15.8:0/889898174 shutdown_connections 2023-11-27T15:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.285+0000 7f213d10ab80 1 --2- 172.21.15.8:0/889898174 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 0x5600027d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.285+0000 7f213d10ab80 1 --2- 172.21.15.8:0/889898174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d5b10 0x5600027d5f80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.285+0000 7f213d10ab80 1 --2- 172.21.15.8:0/889898174 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d4db0 0x5600027d55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.285+0000 7f213d10ab80 1 -- 172.21.15.8:0/889898174 >> 172.21.15.8:0/889898174 conn(0x5600027c9f50 msgr2=0x5600027cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.285+0000 7f213d10ab80 1 -- 172.21.15.8:0/889898174 shutdown_connections 2023-11-27T15:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.285+0000 7f213d10ab80 1 -- 172.21.15.8:0/889898174 wait complete. 2023-11-27T15:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.285+0000 7f213d10ab80 1 -- start start 2023-11-27T15:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.286+0000 7f213d10ab80 1 Processor -- start 2023-11-27T15:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.286+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600026306b0 0x5600027ddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.286+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d4db0 0x5600027de180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.286+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600027d5b10 0x5600027e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.286+0000 7f213d10ab80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600026c7880 con 0x5600027d4db0 2023-11-27T15:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.286+0000 7f213d10ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600026c7520 con 0x5600026306b0 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.286+0000 7f213d10ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600025a98c0 con 0x5600027d5b10 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.286+0000 7f213d10ab80 10 client.0 _ll_register_callbacks cb 0x5600026e5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.286+0000 7f2122ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600026306b0 0x5600027ddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f211bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d4db0 0x5600027de180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f2122ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600026306b0 0x5600027ddc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:35586/0 (socket says 172.21.15.8:35586) 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f2122ffd700 1 -- 172.21.15.8:0/1472301565 learned_addr learned my addr 172.21.15.8:0/1472301565 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f21237fe700 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600027d5b10 0x5600027e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f2122ffd700 1 -- 172.21.15.8:0/1472301565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600027d5b10 msgr2=0x5600027e51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f2122ffd700 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600027d5b10 0x5600027e51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f2122ffd700 1 -- 172.21.15.8:0/1472301565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d4db0 msgr2=0x5600027de180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:36.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f2122ffd700 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d4db0 0x5600027de180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f2122ffd700 1 -- 172.21.15.8:0/1472301565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c011330 con 0x5600026306b0 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f2122ffd700 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600026306b0 0x5600027ddc40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2114022890 tx=0x7f2114005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f216995e0b0739a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f210a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f210a7fc700 1 -- 172.21.15.8:0/1472301565 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211409b260 con 0x5600026306b0 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f213d10ab80 1 -- 172.21.15.8:0/1472301565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600027c7710 con 0x5600026306b0 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.287+0000 7f210a7fc700 1 -- 172.21.15.8:0/1472301565 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f211409f290 con 0x5600026306b0 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f210a7fc700 1 -- 172.21.15.8:0/1472301565 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21140a3620 con 0x5600026306b0 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f213d10ab80 10 client.25433 fetch_fsmap learned FSMap version 24 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f213d10ab80 1 -- 172.21.15.8:0/1472301565 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5600026ec130 con 0x5600026306b0 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f210a7fc700 1 -- 172.21.15.8:0/1472301565 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21140b7080 con 0x5600026306b0 2023-11-27T15:26:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f210a7fc700 1 -- 172.21.15.8:0/1472301565 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f21140a3190 con 0x5600026306b0 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f213d10ab80 10 client.25433 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f213d10ab80 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f213d10ab80 20 client.25433 populate_metadata read hostname 'smithi008' 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f213d10ab80 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5600027e9680 0x5600027eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f213d10ab80 4 client.25433 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.288+0000 7f213d10ab80 1 -- 172.21.15.8:0/1472301565 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600026e8040 con 0x5600027e9680 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.289+0000 7f211bfff700 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5600027e9680 0x5600027eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.289+0000 7f211bfff700 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5600027e9680 0x5600027eba60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b253397d227c593d server_cookie=7189396b8a0e5847 in_seq=0 out_seq=0 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.289+0000 7f210a7fc700 10 client.25433 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f210a7fc700 1 -- 172.21.15.8:0/1472301565 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5600026e8040 con 0x5600027e9680 2023-11-27T15:26:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f210a7fc700 10 client.25433 handle_command_reply: tid=0 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 client.25433 shutdown 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 -- 172.21.15.8:0/1472301565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600026306b0 msgr2=0x5600027ddc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600026306b0 0x5600027ddc40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2114022890 tx=0x7f2114005fd0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 -- 172.21.15.8:0/1472301565 shutdown_connections 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5600027e9680 0x5600027eba60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600027d5b10 0x5600027e51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600026306b0 0x5600027ddc40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 --2- 172.21.15.8:0/1472301565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d4db0 0x5600027de180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f210a7fc700 0 client.25433 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 -- 172.21.15.8:0/1472301565 >> 172.21.15.8:0/1472301565 conn(0x5600027c9f50 msgr2=0x5600027ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 -- 172.21.15.8:0/1472301565 shutdown_connections 2023-11-27T15:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.290+0000 7f213d10ab80 1 -- 172.21.15.8:0/1472301565 wait complete. 2023-11-27T15:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.291+0000 7f213d10ab80 20 client.25433 trim_cache size 0 max 16384 2023-11-27T15:26:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.306+0000 7f213d10ab80 1 Processor -- start 2023-11-27T15:26:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.306+0000 7f213d10ab80 1 -- start start 2023-11-27T15:26:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.306+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600026306b0 0x5600027d5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.306+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600027d4db0 0x5600027e5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.306+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d5b10 0x5600027f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.306+0000 7f213d10ab80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600027dd810 con 0x5600026306b0 2023-11-27T15:26:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.306+0000 7f213d10ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600027cef10 con 0x5600027d5b10 2023-11-27T15:26:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.306+0000 7f213d10ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600027d5450 con 0x5600027d4db0 2023-11-27T15:26:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.306+0000 7f2122ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600026306b0 0x5600027d5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.306+0000 7f2122ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600026306b0 0x5600027d5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54724/0 (socket says 172.21.15.8:54724) 2023-11-27T15:26:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f2122ffd700 1 -- 172.21.15.8:0/1770303021 learned_addr learned my addr 172.21.15.8:0/1770303021 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f211bfff700 1 --2- 172.21.15.8:0/1770303021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600027d4db0 0x5600027e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f21237fe700 1 --2- 172.21.15.8:0/1770303021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d5b10 0x5600027f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f2122ffd700 1 -- 172.21.15.8:0/1770303021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600027d4db0 msgr2=0x5600027e5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f2122ffd700 1 --2- 172.21.15.8:0/1770303021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600027d4db0 0x5600027e5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f2122ffd700 1 -- 172.21.15.8:0/1770303021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d5b10 msgr2=0x5600027f03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f2122ffd700 1 --2- 172.21.15.8:0/1770303021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d5b10 0x5600027f03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f2122ffd700 1 -- 172.21.15.8:0/1770303021 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600026ebe80 con 0x5600026306b0 2023-11-27T15:26:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f2122ffd700 1 --2- 172.21.15.8:0/1770303021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600026306b0 0x5600027d5ee0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2114000c00 tx=0x7f2114009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9d791d988179e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f2119ffb700 1 -- 172.21.15.8:0/1770303021 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114097400 con 0x5600026306b0 2023-11-27T15:26:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f2119ffb700 1 -- 172.21.15.8:0/1770303021 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5600027c7710 con 0x5600026306b0 2023-11-27T15:26:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.307+0000 7f2119ffb700 1 -- 172.21.15.8:0/1770303021 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140a3620 con 0x5600026306b0 2023-11-27T15:26:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.308+0000 7f213d10ab80 1 -- 172.21.15.8:0/1770303021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600026306b0 msgr2=0x5600027d5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.308+0000 7f213d10ab80 1 --2- 172.21.15.8:0/1770303021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600026306b0 0x5600027d5ee0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2114000c00 tx=0x7f2114009ca0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.308+0000 7f213d10ab80 1 -- 172.21.15.8:0/1770303021 shutdown_connections 2023-11-27T15:26:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.308+0000 7f213d10ab80 1 --2- 172.21.15.8:0/1770303021 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d5b10 0x5600027f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.308+0000 7f213d10ab80 1 --2- 172.21.15.8:0/1770303021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600026306b0 0x5600027d5ee0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.308+0000 7f213d10ab80 1 --2- 172.21.15.8:0/1770303021 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600027d4db0 0x5600027e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.308+0000 7f213d10ab80 1 -- 172.21.15.8:0/1770303021 >> 172.21.15.8:0/1770303021 conn(0x5600027c9f50 msgr2=0x5600027cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.308+0000 7f213d10ab80 1 -- 172.21.15.8:0/1770303021 shutdown_connections 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.308+0000 7f213d10ab80 1 -- 172.21.15.8:0/1770303021 wait complete. 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.309+0000 7f213d10ab80 1 -- start start 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.309+0000 7f213d10ab80 1 Processor -- start 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.309+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 0x5600027e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.309+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d4db0 0x5600027e1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.309+0000 7f213d10ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d5b10 0x5600027e4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.309+0000 7f213d10ab80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600027e47d0 con 0x5600027d4db0 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.309+0000 7f213d10ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600027e4940 con 0x5600027d5b10 2023-11-27T15:26:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.309+0000 7f213d10ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600027e4b00 con 0x5600026306b0 2023-11-27T15:26:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.309+0000 7f213d10ab80 10 client.0 _ll_register_callbacks cb 0x7f211409ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f2122ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 0x5600027e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f21237fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d5b10 0x5600027e4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f2122ffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 0x5600027e0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:47104/0 (socket says 172.21.15.8:47104) 2023-11-27T15:26:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f2122ffd700 1 -- 172.21.15.8:0/68534241 learned_addr learned my addr 172.21.15.8:0/68534241 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f211bfff700 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d4db0 0x5600027e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f2122ffd700 1 -- 172.21.15.8:0/68534241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d5b10 msgr2=0x5600027e4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f2122ffd700 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d5b10 0x5600027e4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f2122ffd700 1 -- 172.21.15.8:0/68534241 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d4db0 msgr2=0x5600027e1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f2122ffd700 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d4db0 0x5600027e1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f2122ffd700 1 -- 172.21.15.8:0/68534241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600026ec130 con 0x5600026306b0 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f2122ffd700 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 0x5600027e0d30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21140995b0 tx=0x7f21140b9ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d984200afc9a53c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f210a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.310+0000 7f210a7fc700 1 -- 172.21.15.8:0/68534241 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211409a950 con 0x5600026306b0 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f210a7fc700 1 -- 172.21.15.8:0/68534241 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f211409f290 con 0x5600026306b0 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f213d10ab80 1 -- 172.21.15.8:0/68534241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f211c00f040 con 0x5600026306b0 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f210a7fc700 1 -- 172.21.15.8:0/68534241 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2114097400 con 0x5600026306b0 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f213d10ab80 10 client.25462 fetch_fsmap learned FSMap version 24 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f213d10ab80 1 -- 172.21.15.8:0/68534241 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56000253cc00 con 0x5600026306b0 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f210a7fc700 1 -- 172.21.15.8:0/68534241 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f211c00f040 con 0x5600026306b0 2023-11-27T15:26:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f210a7fc700 1 -- 172.21.15.8:0/68534241 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2114084ab0 con 0x5600026306b0 2023-11-27T15:26:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f213d10ab80 10 client.25462 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:26:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f213d10ab80 10 client.25462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:26:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f213d10ab80 20 client.25462 populate_metadata read hostname 'smithi008' 2023-11-27T15:26:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f213d10ab80 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56000283da30 0x56000283fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f213d10ab80 4 client.25462 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:26:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.311+0000 7f213d10ab80 1 -- 172.21.15.8:0/68534241 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5600026f4600 con 0x56000283da30 2023-11-27T15:26:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.312+0000 7f211bfff700 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56000283da30 0x56000283fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.312+0000 7f211bfff700 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56000283da30 0x56000283fe10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b208137b1850565c server_cookie=d3536ccb46ccfe95 in_seq=0 out_seq=0 2023-11-27T15:26:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:36.312+0000 7f210a7fc700 10 client.25462 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:36.321 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:36.321 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:36.326 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:36.326 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:36.327 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:36.802 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:36.802 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:36.955 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:36.955 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:36.968 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:36.968 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:36.968 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 12 -lt 100 ']' 2023-11-27T15:26:36.968 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 12 + 1 2023-11-27T15:26:37.000 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=13 2023-11-27T15:26:37.000 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:37.029 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:37.029 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:37.032 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:37.032 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:37.033 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:37.585 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:37.586 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:37.697 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:37.697 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:37.711 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:37.711 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:37.711 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 13 -lt 100 ']' 2023-11-27T15:26:37.711 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 13 + 1 2023-11-27T15:26:37.742 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=14 2023-11-27T15:26:37.742 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:37.780 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:37.780 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:37.782 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:37.783 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:37.783 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:37 smithi060 ceph-mon[131518]: pgmap v381: 129 pgs: 129 active+clean; 748 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2023-11-27T15:26:37.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:37 smithi008 ceph-mon[128788]: pgmap v381: 129 pgs: 129 active+clean; 748 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2023-11-27T15:26:37.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:37 smithi029 ceph-mon[131166]: pgmap v381: 129 pgs: 129 active+clean; 748 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 8 op/s 2023-11-27T15:26:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f210a7fc700 1 -- 172.21.15.8:0/68534241 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5600026f4600 con 0x56000283da30 2023-11-27T15:26:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f210a7fc700 10 client.25462 handle_command_reply: tid=0 2023-11-27T15:26:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 client.25462 shutdown 2023-11-27T15:26:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 -- 172.21.15.8:0/68534241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 msgr2=0x5600027e0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 0x5600027e0d30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21140995b0 tx=0x7f21140b9ba0 comp rx=0 tx=0).stop 2023-11-27T15:26:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 -- 172.21.15.8:0/68534241 shutdown_connections 2023-11-27T15:26:38.031 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56000283da30 0x56000283fe10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5600026306b0 0x5600027e0d30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5600027d5b10 0x5600027e4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 --2- 172.21.15.8:0/68534241 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5600027d4db0 0x5600027e1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 -- 172.21.15.8:0/68534241 >> 172.21.15.8:0/68534241 conn(0x5600027c9f50 msgr2=0x5600027cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f210a7fc700 0 client.25462 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 -- 172.21.15.8:0/68534241 shutdown_connections 2023-11-27T15:26:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 1 -- 172.21.15.8:0/68534241 wait complete. 2023-11-27T15:26:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.027+0000 7f213d10ab80 20 client.25462 trim_cache size 0 max 16384 2023-11-27T15:26:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:26:38.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:26:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:26:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:26:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 -- 172.21.15.8:0/3056683051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f21100a1150 msgr2=0x7f21100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f21100a1150 0x7f21100a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f210c009c60 tx=0x7f210c009260 comp rx=0 tx=0).stop 2023-11-27T15:26:38.033 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 -- 172.21.15.8:0/3056683051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2124100230 msgr2=0x7f21241981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2124100230 0x7f21241981f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f211400b680 tx=0x7f2114005740 comp rx=0 tx=0).stop 2023-11-27T15:26:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 -- 172.21.15.8:0/3056683051 shutdown_connections 2023-11-27T15:26:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f2124101e70 0x7f212419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2124100230 0x7f21241981f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f21100a1150 0x7f21100a3600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 --2- 172.21.15.8:0/3056683051 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f21241014c0 0x7f2124198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 -- 172.21.15.8:0/3056683051 >> 172.21.15.8:0/3056683051 conn(0x7f21240fb810 msgr2=0x7f21240fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 -- 172.21.15.8:0/3056683051 shutdown_connections 2023-11-27T15:26:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:38.028+0000 7f21297ff700 1 -- 172.21.15.8:0/3056683051 wait complete. 2023-11-27T15:26:38.211 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:38.211 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:38.338 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:38.338 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:38.350 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:38.350 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:38.350 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 14 -lt 100 ']' 2023-11-27T15:26:38.350 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 14 + 1 2023-11-27T15:26:38.383 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=15 2023-11-27T15:26:38.384 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:38.421 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:38.421 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:38.424 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:38.424 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:38.425 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:38.944 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:38.944 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:39.030 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:39.030 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:39.043 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:39.043 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:39.043 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 15 -lt 100 ']' 2023-11-27T15:26:39.043 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 15 + 1 2023-11-27T15:26:39.075 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=16 2023-11-27T15:26:39.075 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:39.103 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:39.104 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:39.106 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:39.107 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:39.107 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:39.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:39 smithi060 ceph-mon[131518]: pgmap v382: 129 pgs: 129 active+clean; 748 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.3 KiB/s wr, 5 op/s 2023-11-27T15:26:39.585 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:39.586 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:39 smithi008 ceph-mon[128788]: pgmap v382: 129 pgs: 129 active+clean; 748 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.3 KiB/s wr, 5 op/s 2023-11-27T15:26:39.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:39 smithi029 ceph-mon[131166]: pgmap v382: 129 pgs: 129 active+clean; 748 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.3 KiB/s wr, 5 op/s 2023-11-27T15:26:39.713 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:39.713 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:39.725 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:39.725 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:39.725 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 16 -lt 100 ']' 2023-11-27T15:26:39.726 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 16 + 1 2023-11-27T15:26:39.758 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=17 2023-11-27T15:26:39.758 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:39.795 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:39.796 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:39.798 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:39.799 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:39.799 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:40.219 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:40.219 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:40.338 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:40.338 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:40.351 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:40.351 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:40.351 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 17 -lt 100 ']' 2023-11-27T15:26:40.352 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 17 + 1 2023-11-27T15:26:40.381 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=18 2023-11-27T15:26:40.381 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:40.420 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:40.420 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:40.422 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:40.423 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:40.423 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:40.860 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:40.861 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:40.971 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:40.972 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:40.983 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:40.983 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:40.983 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 18 -lt 100 ']' 2023-11-27T15:26:40.984 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 18 + 1 2023-11-27T15:26:41.016 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=19 2023-11-27T15:26:41.017 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:41.045 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:41.045 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:41.048 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:41.048 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:41.049 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:41.493 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:41.494 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:41.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:41 smithi008 ceph-mon[128788]: pgmap v383: 129 pgs: 129 active+clean; 823 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-11-27T15:26:41.638 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:41.638 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:41.650 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:41.650 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:41.651 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 19 -lt 100 ']' 2023-11-27T15:26:41.651 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 19 + 1 2023-11-27T15:26:41.683 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=20 2023-11-27T15:26:41.683 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:41.729 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:41.729 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:41.733 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:41.733 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:41.733 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:41 smithi060 ceph-mon[131518]: pgmap v383: 129 pgs: 129 active+clean; 823 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-11-27T15:26:41.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:41 smithi029 ceph-mon[131166]: pgmap v383: 129 pgs: 129 active+clean; 823 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-11-27T15:26:42.235 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:42.235 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:42.363 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:42.363 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:42.374 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:42.374 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:42.374 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 20 -lt 100 ']' 2023-11-27T15:26:42.375 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 20 + 1 2023-11-27T15:26:42.408 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=21 2023-11-27T15:26:42.408 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:42.444 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:42.445 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:42.447 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:42.448 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:42.448 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:42.918 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.291 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:43 smithi029 ceph-mon[131166]: pgmap v384: 129 pgs: 129 active+clean; 823 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-11-27T15:26:46.291 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:45 smithi029 ceph-mon[131166]: pgmap v385: 129 pgs: 129 active+clean; 823 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-11-27T15:26:46.291 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:43 smithi060 ceph-mon[131518]: pgmap v384: 129 pgs: 129 active+clean; 823 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-11-27T15:26:46.292 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:45 smithi060 ceph-mon[131518]: pgmap v385: 129 pgs: 129 active+clean; 823 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-11-27T15:26:46.293 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.293 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:46.293 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:46.293 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.293 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 21 -lt 100 ']' 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 21 + 1 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=22 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:46.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.295 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.295 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 22 -lt 100 ']' 2023-11-27T15:26:46.295 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 22 + 1 2023-11-27T15:26:46.295 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:43 smithi008 ceph-mon[128788]: pgmap v384: 129 pgs: 129 active+clean; 823 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 10 op/s 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=23 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:46.299 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 23 -lt 100 ']' 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 23 + 1 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=24 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.300 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 24 -lt 100 ']' 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 24 + 1 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=25 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.301 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 25 -lt 100 ']' 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 25 + 1 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=26 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:46.302 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:46.303 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:45 smithi008 ceph-mon[128788]: pgmap v385: 129 pgs: 129 active+clean; 823 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-11-27T15:26:46.303 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.303 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.370 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:46.370 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:46.382 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.382 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:46.383 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 26 -lt 100 ']' 2023-11-27T15:26:46.383 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 26 + 1 2023-11-27T15:26:46.416 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=27 2023-11-27T15:26:46.416 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:46.453 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.454 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:46.456 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:46.457 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:46.457 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:46.926 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:46.926 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:47.129 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:47.129 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:47.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:47.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:47.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 27 -lt 100 ']' 2023-11-27T15:26:47.142 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 27 + 1 2023-11-27T15:26:47.174 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=28 2023-11-27T15:26:47.174 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:47.220 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:47.220 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:47.223 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:47.223 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:47.224 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:47.722 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:47 smithi008 ceph-mon[128788]: pgmap v386: 129 pgs: 129 active+clean; 895 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-11-27T15:26:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:47 smithi060 ceph-mon[131518]: pgmap v386: 129 pgs: 129 active+clean; 895 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-11-27T15:26:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:47 smithi029 ceph-mon[131166]: pgmap v386: 129 pgs: 129 active+clean; 895 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 11 op/s 2023-11-27T15:26:48.126 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:48.126 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:48.312 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:48.312 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:48.326 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:48.326 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:48.326 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 28 -lt 100 ']' 2023-11-27T15:26:48.327 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 28 + 1 2023-11-27T15:26:48.357 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=29 2023-11-27T15:26:48.357 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:48.403 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:48.403 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:48.406 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:48.406 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:48.407 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:49.001 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:49.001 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:49.137 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:49.137 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:49.148 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:49.149 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:49.149 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 29 -lt 100 ']' 2023-11-27T15:26:49.149 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 29 + 1 2023-11-27T15:26:49.182 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=30 2023-11-27T15:26:49.182 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:49.218 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:49.219 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:49.221 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:49.222 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:49.222 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:49.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:49 smithi060 ceph-mon[131518]: pgmap v387: 129 pgs: 129 active+clean; 895 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-11-27T15:26:49.596 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:49 smithi008 ceph-mon[128788]: pgmap v387: 129 pgs: 129 active+clean; 895 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-11-27T15:26:49.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:49 smithi029 ceph-mon[131166]: pgmap v387: 129 pgs: 129 active+clean; 895 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 9 op/s 2023-11-27T15:26:49.667 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:49.668 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:49.787 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:49.787 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:49.799 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:49.800 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:49.800 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 30 -lt 100 ']' 2023-11-27T15:26:49.800 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 30 + 1 2023-11-27T15:26:49.832 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=31 2023-11-27T15:26:49.832 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:49.868 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:49.869 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:49.871 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:49.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:49.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:50.367 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:50.368 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:50.495 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:50.495 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:50.507 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:50.507 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:50.507 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 31 -lt 100 ']' 2023-11-27T15:26:50.508 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 31 + 1 2023-11-27T15:26:50.540 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=32 2023-11-27T15:26:50.540 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:50.577 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:50.577 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:50.580 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:50.580 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:50.581 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:51.001 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:51.001 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:51.128 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:51.129 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:51.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:51.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:51.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 32 -lt 100 ']' 2023-11-27T15:26:51.142 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 32 + 1 2023-11-27T15:26:51.173 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=33 2023-11-27T15:26:51.174 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:51.228 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:51.228 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:51.230 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:51.231 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:51.231 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:51.714 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:51 smithi008 ceph-mon[128788]: pgmap v388: 129 pgs: 129 active+clean; 963 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-27T15:26:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:51 smithi060 ceph-mon[131518]: pgmap v388: 129 pgs: 129 active+clean; 963 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-27T15:26:51.842 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:51.843 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:51.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:51 smithi029 ceph-mon[131166]: pgmap v388: 129 pgs: 129 active+clean; 963 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-27T15:26:51.921 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:51.921 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:51.932 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:51.933 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:51.933 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 33 -lt 100 ']' 2023-11-27T15:26:51.933 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 33 + 1 2023-11-27T15:26:52.007 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=34 2023-11-27T15:26:52.007 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:52.029 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:52.030 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:52.033 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:52.033 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:52.033 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:52.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:52 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:26:52.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:52 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:26:52.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:52 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:26:53.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:53 smithi060 ceph-mon[131518]: pgmap v389: 129 pgs: 129 active+clean; 963 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-11-27T15:26:53.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:53 smithi008 ceph-mon[128788]: pgmap v389: 129 pgs: 129 active+clean; 963 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-11-27T15:26:53.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:53 smithi029 ceph-mon[131166]: pgmap v389: 129 pgs: 129 active+clean; 963 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 9 op/s 2023-11-27T15:26:54.894 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:54.894 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:55.086 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:55.086 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:55.116 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:55.116 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:55.116 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 34 -lt 100 ']' 2023-11-27T15:26:55.116 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 34 + 1 2023-11-27T15:26:55.145 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=35 2023-11-27T15:26:55.145 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:55.200 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:55.201 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:55.204 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:55.204 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:55.205 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:55.712 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:55 smithi008 ceph-mon[128788]: pgmap v390: 129 pgs: 129 active+clean; 964 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-11-27T15:26:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:55 smithi060 ceph-mon[131518]: pgmap v390: 129 pgs: 129 active+clean; 964 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-11-27T15:26:55.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:55 smithi029 ceph-mon[131166]: pgmap v390: 129 pgs: 129 active+clean; 964 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 7 op/s 2023-11-27T15:26:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:55.986+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/2043173940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80feb30 msgr2=0x7ff4f80fef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:55.986+0000 7ff4fe3ff700 1 --2- 172.21.15.8:0/2043173940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80feb30 0x7ff4f80fef40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009990 tx=0x7ff4e8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:55.986+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/2043173940 shutdown_connections 2023-11-27T15:26:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:55.986+0000 7ff4fe3ff700 1 --2- 172.21.15.8:0/2043173940 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f80ffe30 0x7ff4f81002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:55.986+0000 7ff4fe3ff700 1 --2- 172.21.15.8:0/2043173940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80feb30 0x7ff4f80fef40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:55.986+0000 7ff4fe3ff700 1 --2- 172.21.15.8:0/2043173940 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f80ff480 0x7ff4f80ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:55.986+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/2043173940 >> 172.21.15.8:0/2043173940 conn(0x7ff4f80fa720 msgr2=0x7ff4f80fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:55.986+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/2043173940 shutdown_connections 2023-11-27T15:26:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:55.986+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/2043173940 wait complete. 2023-11-27T15:26:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.012+0000 7ff4fe3ff700 1 Processor -- start 2023-11-27T15:26:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4fe3ff700 1 -- start start 2023-11-27T15:26:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4fe3ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f80feb30 0x7ff4f8198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4fe3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f80ff480 0x7ff4f8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4fe3ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80ffe30 0x7ff4f819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4fe3ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f819fda0 con 0x7ff4f80ffe30 2023-11-27T15:26:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4fe3ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f819fee0 con 0x7ff4f80feb30 2023-11-27T15:26:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4fe3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81a0070 con 0x7ff4f80ff480 2023-11-27T15:26:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4f77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f80ff480 0x7ff4f8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f80feb30 0x7ff4f8198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4f77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f80ff480 0x7ff4f8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:60158/0 (socket says 172.21.15.8:60158) 2023-11-27T15:26:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.013+0000 7ff4f77fe700 1 -- 172.21.15.8:0/3603783088 learned_addr learned my addr 172.21.15.8:0/3603783088 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.014+0000 7ff4fc99c700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80ffe30 0x7ff4f819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.014+0000 7ff4f77fe700 1 -- 172.21.15.8:0/3603783088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f80feb30 msgr2=0x7ff4f8198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.014+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f80feb30 0x7ff4f8198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.014+0000 7ff4f77fe700 1 -- 172.21.15.8:0/3603783088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80ffe30 msgr2=0x7ff4f819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.014+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80ffe30 0x7ff4f819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.014+0000 7ff4f77fe700 1 -- 172.21.15.8:0/3603783088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80095f0 con 0x7ff4f80ff480 2023-11-27T15:26:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.014+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f80ff480 0x7ff4f8198760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00e9a0 tx=0x7ff4ec00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.014+0000 7ff4f57fa700 1 -- 172.21.15.8:0/3603783088 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec009790 con 0x7ff4f80ff480 2023-11-27T15:26:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.015+0000 7ff4f57fa700 1 -- 172.21.15.8:0/3603783088 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ec010430 con 0x7ff4f80ff480 2023-11-27T15:26:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.015+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/3603783088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f810ddf0 con 0x7ff4f80ff480 2023-11-27T15:26:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.015+0000 7ff4f57fa700 1 -- 172.21.15.8:0/3603783088 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec018680 con 0x7ff4f80ff480 2023-11-27T15:26:56.021 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.015+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/3603783088 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f810dc60 con 0x7ff4f80ff480 2023-11-27T15:26:56.022 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.018+0000 7ff4f57fa700 1 -- 172.21.15.8:0/3603783088 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff4ec0105a0 con 0x7ff4f80ff480 2023-11-27T15:26:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.019+0000 7ff4f57fa700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff4e00a1150 0x7ff4e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.020+0000 7ff4f57fa700 1 -- 172.21.15.8:0/3603783088 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff4ec014070 con 0x7ff4f80ff480 2023-11-27T15:26:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.020+0000 7ff4f7fff700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff4e00a1150 0x7ff4e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.021+0000 7ff4f7fff700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff4e00a1150 0x7ff4e00a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800b2e0 tx=0x7ff4e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:56.075 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:56.075 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:56.253 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:56.253 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:56.284 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:56.284 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:56.284 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 35 -lt 100 ']' 2023-11-27T15:26:56.285 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 35 + 1 2023-11-27T15:26:56.387 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=36 2023-11-27T15:26:56.387 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:56.441 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:56.441 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:56.444 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:56.445 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:56.445 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.646+0000 7ff511e19b80 1 Processor -- start 2023-11-27T15:26:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff511e19b80 1 -- start start 2023-11-27T15:26:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d93870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d945d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.652 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950d94f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff511e19b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950c99000 con 0x55f950bef6b0 2023-11-27T15:26:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff511e19b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950ceb840 con 0x55f950d94b10 2023-11-27T15:26:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff511e19b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950ce4cd0 con 0x55f950d93db0 2023-11-27T15:26:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d93870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d93870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57784/0 (socket says 172.21.15.8:57784) 2023-11-27T15:26:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff4f7fff700 1 -- 172.21.15.8:0/2552154742 learned_addr learned my addr 172.21.15.8:0/2552154742 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/2552154742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d945d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.647+0000 7ff4fc99c700 1 --2- 172.21.15.8:0/2552154742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950d94f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.648+0000 7ff4f77fe700 1 -- 172.21.15.8:0/2552154742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 msgr2=0x55f950d94f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.648+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/2552154742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950d94f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.648+0000 7ff4f77fe700 1 -- 172.21.15.8:0/2552154742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 msgr2=0x55f950d93870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.648+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/2552154742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d93870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.648+0000 7ff4f77fe700 1 -- 172.21.15.8:0/2552154742 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f950b665d0 con 0x55f950d93db0 2023-11-27T15:26:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.648+0000 7ff4f7fff700 1 --2- 172.21.15.8:0/2552154742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d93870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.648+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/2552154742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d945d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec099c30 tx=0x7ff4ec099910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a4c1c376c71e15a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.648+0000 7ff4de7fc700 1 -- 172.21.15.8:0/2552154742 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec09df60 con 0x55f950d93db0 2023-11-27T15:26:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.648+0000 7ff4de7fc700 1 -- 172.21.15.8:0/2552154742 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ec0a2380 con 0x55f950d93db0 2023-11-27T15:26:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.648+0000 7ff4de7fc700 1 -- 172.21.15.8:0/2552154742 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0a1ef0 con 0x55f950d93db0 2023-11-27T15:26:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.649+0000 7ff511e19b80 1 -- 172.21.15.8:0/2552154742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 msgr2=0x55f950d945d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.649+0000 7ff511e19b80 1 --2- 172.21.15.8:0/2552154742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d945d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec099c30 tx=0x7ff4ec099910 comp rx=0 tx=0).stop 2023-11-27T15:26:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.649+0000 7ff511e19b80 1 -- 172.21.15.8:0/2552154742 shutdown_connections 2023-11-27T15:26:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.649+0000 7ff511e19b80 1 --2- 172.21.15.8:0/2552154742 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950d94f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.649+0000 7ff511e19b80 1 --2- 172.21.15.8:0/2552154742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d93870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.649+0000 7ff511e19b80 1 --2- 172.21.15.8:0/2552154742 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d945d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.649+0000 7ff511e19b80 1 -- 172.21.15.8:0/2552154742 >> 172.21.15.8:0/2552154742 conn(0x55f950d88f50 msgr2=0x55f950d8e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.649+0000 7ff511e19b80 1 -- 172.21.15.8:0/2552154742 shutdown_connections 2023-11-27T15:26:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.650+0000 7ff511e19b80 1 -- 172.21.15.8:0/2552154742 wait complete. 2023-11-27T15:26:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.650+0000 7ff511e19b80 1 -- start start 2023-11-27T15:26:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.650+0000 7ff511e19b80 1 Processor -- start 2023-11-27T15:26:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d9cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d9d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950da4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff511e19b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950c86880 con 0x55f950bef6b0 2023-11-27T15:26:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff511e19b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950c86520 con 0x55f950d94b10 2023-11-27T15:26:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff511e19b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950b688c0 con 0x55f950d93db0 2023-11-27T15:26:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff511e19b80 10 client.0 _ll_register_callbacks cb 0x55f950ca4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d9cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff4fc99c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950da4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff4f77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d9d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff4f77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d9d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:60172/0 (socket says 172.21.15.8:60172) 2023-11-27T15:26:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d9cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57790/0 (socket says 172.21.15.8:57790) 2023-11-27T15:26:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff4f7fff700 1 -- 172.21.15.8:0/114306333 learned_addr learned my addr 172.21.15.8:0/114306333 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.651+0000 7ff4fc99c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950da4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:43294/0 (socket says 172.21.15.8:43294) 2023-11-27T15:26:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff4f77fe700 1 -- 172.21.15.8:0/114306333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 msgr2=0x55f950da4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950da4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff4f77fe700 1 -- 172.21.15.8:0/114306333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 msgr2=0x55f950d9cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d9cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff4f77fe700 1 -- 172.21.15.8:0/114306333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f810dc60 con 0x55f950d93db0 2023-11-27T15:26:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d9d250 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec098520 tx=0x7ff4ec099da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67619b7854c70af9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff4d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:26:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff4d77fe700 1 -- 172.21.15.8:0/114306333 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec098130 con 0x55f950d93db0 2023-11-27T15:26:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff511e19b80 1 -- 172.21.15.8:0/114306333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f950d86710 con 0x55f950d93db0 2023-11-27T15:26:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff4d77fe700 1 -- 172.21.15.8:0/114306333 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff4ec0ca200 con 0x55f950d93db0 2023-11-27T15:26:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff4d77fe700 1 -- 172.21.15.8:0/114306333 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4ec0ca3c0 con 0x55f950d93db0 2023-11-27T15:26:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff511e19b80 10 client.25474 fetch_fsmap learned FSMap version 24 2023-11-27T15:26:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.652+0000 7ff511e19b80 1 -- 172.21.15.8:0/114306333 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f950cab130 con 0x55f950d93db0 2023-11-27T15:26:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.653+0000 7ff4d77fe700 1 -- 172.21.15.8:0/114306333 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f950d86710 con 0x55f950d93db0 2023-11-27T15:26:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.653+0000 7ff4d77fe700 1 -- 172.21.15.8:0/114306333 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4ec095d20 con 0x55f950d93db0 2023-11-27T15:26:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.653+0000 7ff511e19b80 10 client.25474 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:26:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.653+0000 7ff511e19b80 10 client.25474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:26:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.653+0000 7ff511e19b80 20 client.25474 populate_metadata read hostname 'smithi008' 2023-11-27T15:26:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.653+0000 7ff511e19b80 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f950da87b0 0x55f950daabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.653+0000 7ff511e19b80 4 client.25474 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.653+0000 7ff511e19b80 1 -- 172.21.15.8:0/114306333 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f950ca7040 con 0x55f950da87b0 2023-11-27T15:26:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.653+0000 7ff4f7fff700 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f950da87b0 0x55f950daabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.654+0000 7ff4f7fff700 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f950da87b0 0x55f950daabc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a64ae8de66dde3e server_cookie=c78ca2f7a60f1f38 in_seq=0 out_seq=0 2023-11-27T15:26:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.654+0000 7ff4d77fe700 10 client.25474 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff4d77fe700 1 -- 172.21.15.8:0/114306333 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f950ca7040 con 0x55f950da87b0 2023-11-27T15:26:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff4d77fe700 10 client.25474 handle_command_reply: tid=0 2023-11-27T15:26:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 client.25474 shutdown 2023-11-27T15:26:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 -- 172.21.15.8:0/114306333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 msgr2=0x55f950d9d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.666 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d9d250 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec098520 tx=0x7ff4ec099da0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 -- 172.21.15.8:0/114306333 shutdown_connections 2023-11-27T15:26:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950da4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d9cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f950da87b0 0x55f950daabc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 --2- 172.21.15.8:0/114306333 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950d9d250 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 -- 172.21.15.8:0/114306333 >> 172.21.15.8:0/114306333 conn(0x55f950d88f50 msgr2=0x55f950d8d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff4d77fe700 0 client.25474 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 -- 172.21.15.8:0/114306333 shutdown_connections 2023-11-27T15:26:56.667 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.655+0000 7ff511e19b80 1 -- 172.21.15.8:0/114306333 wait complete. 2023-11-27T15:26:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.656+0000 7ff511e19b80 20 client.25474 trim_cache size 0 max 16384 2023-11-27T15:26:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.671+0000 7ff511e19b80 1 Processor -- start 2023-11-27T15:26:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.671+0000 7ff511e19b80 1 -- start start 2023-11-27T15:26:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950daf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950da4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950d9ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff511e19b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950d94420 con 0x55f950bef6b0 2023-11-27T15:26:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff511e19b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950dab220 con 0x55f950d94b10 2023-11-27T15:26:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff511e19b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950d98590 con 0x55f950d93db0 2023-11-27T15:26:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950daf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4f77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950da4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4f77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950da4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:60174/0 (socket says 172.21.15.8:60174) 2023-11-27T15:26:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950daf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57800/0 (socket says 172.21.15.8:57800) 2023-11-27T15:26:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4f77fe700 1 -- 172.21.15.8:0/2546037287 learned_addr learned my addr 172.21.15.8:0/2546037287 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4fc99c700 1 --2- 172.21.15.8:0/2546037287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950d9ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4f77fe700 1 -- 172.21.15.8:0/2546037287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 msgr2=0x55f950d9ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/2546037287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950d9ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4f77fe700 1 -- 172.21.15.8:0/2546037287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 msgr2=0x55f950daf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/2546037287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950daf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.672+0000 7ff4f77fe700 1 -- 172.21.15.8:0/2546037287 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f950caae80 con 0x55f950d93db0 2023-11-27T15:26:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.673+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/2546037287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950da4250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec000c00 tx=0x7ff4ec099420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429abfd870eb5b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.673+0000 7ff4deffd700 1 -- 172.21.15.8:0/2546037287 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0caea0 con 0x55f950d93db0 2023-11-27T15:26:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.673+0000 7ff4deffd700 1 -- 172.21.15.8:0/2546037287 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ec09dfe0 con 0x55f950d93db0 2023-11-27T15:26:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.673+0000 7ff4deffd700 1 -- 172.21.15.8:0/2546037287 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0caae0 con 0x55f950d93db0 2023-11-27T15:26:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.674+0000 7ff511e19b80 1 -- 172.21.15.8:0/2546037287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 msgr2=0x55f950da4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.674+0000 7ff511e19b80 1 --2- 172.21.15.8:0/2546037287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950da4250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec000c00 tx=0x7ff4ec099420 comp rx=0 tx=0).stop 2023-11-27T15:26:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.674+0000 7ff511e19b80 1 -- 172.21.15.8:0/2546037287 shutdown_connections 2023-11-27T15:26:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.674+0000 7ff511e19b80 1 --2- 172.21.15.8:0/2546037287 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d94b10 0x55f950d9ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.674+0000 7ff511e19b80 1 --2- 172.21.15.8:0/2546037287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950daf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.674+0000 7ff511e19b80 1 --2- 172.21.15.8:0/2546037287 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d93db0 0x55f950da4250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.674+0000 7ff511e19b80 1 -- 172.21.15.8:0/2546037287 >> 172.21.15.8:0/2546037287 conn(0x55f950d88f50 msgr2=0x55f950d8e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.674+0000 7ff511e19b80 1 -- 172.21.15.8:0/2546037287 shutdown_connections 2023-11-27T15:26:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.674+0000 7ff511e19b80 1 -- 172.21.15.8:0/2546037287 wait complete. 2023-11-27T15:26:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.674+0000 7ff511e19b80 1 -- start start 2023-11-27T15:26:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.675+0000 7ff511e19b80 1 Processor -- start 2023-11-27T15:26:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.675+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d9fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.675+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d93db0 0x55f950d9ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.675+0000 7ff511e19b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d94b10 0x55f950da2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.675+0000 7ff511e19b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950da34e0 con 0x55f950bef6b0 2023-11-27T15:26:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.675+0000 7ff511e19b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950da3650 con 0x55f950d93db0 2023-11-27T15:26:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.675+0000 7ff511e19b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f950da3810 con 0x55f950d94b10 2023-11-27T15:26:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.675+0000 7ff511e19b80 10 client.0 _ll_register_callbacks cb 0x7ff4e8005610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d9fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4fc99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d94b10 0x55f950da2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4f77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d93db0 0x55f950d9ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4f7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d9fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57812/0 (socket says 172.21.15.8:57812) 2023-11-27T15:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4f7fff700 1 -- 172.21.15.8:0/65651487 learned_addr learned my addr 172.21.15.8:0/65651487 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4fc99c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d94b10 0x55f950da2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:60178/0 (socket says 172.21.15.8:60178) 2023-11-27T15:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4f77fe700 1 -- 172.21.15.8:0/65651487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d94b10 msgr2=0x55f950da2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d94b10 0x55f950da2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4f77fe700 1 -- 172.21.15.8:0/65651487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 msgr2=0x55f950d9fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d9fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4f77fe700 1 -- 172.21.15.8:0/65651487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f950cab130 con 0x55f950d93db0 2023-11-27T15:26:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4fc99c700 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d94b10 0x55f950da2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4f77fe700 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d93db0 0x55f950d9ff80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00e970 tx=0x7ff4ec0ca6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4de2011ea9e844cd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:26:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.676+0000 7ff4d77fe700 1 -- 172.21.15.8:0/65651487 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec0caea0 con 0x55f950d93db0 2023-11-27T15:26:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff511e19b80 1 -- 172.21.15.8:0/65651487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4ec0a2380 con 0x55f950d93db0 2023-11-27T15:26:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff4d77fe700 1 -- 172.21.15.8:0/65651487 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7ff4ec095d20 con 0x55f950d93db0 2023-11-27T15:26:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff4d77fe700 1 -- 172.21.15.8:0/65651487 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4ec0cacf0 con 0x55f950d93db0 2023-11-27T15:26:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff511e19b80 10 client.25457 fetch_fsmap learned FSMap version 24 2023-11-27T15:26:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff511e19b80 1 -- 172.21.15.8:0/65651487 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f950afbc00 con 0x55f950d93db0 2023-11-27T15:26:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff4d77fe700 1 -- 172.21.15.8:0/65651487 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4ec0a2380 con 0x55f950d93db0 2023-11-27T15:26:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff4d77fe700 1 -- 172.21.15.8:0/65651487 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff4ec098ee0 con 0x55f950d93db0 2023-11-27T15:26:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff511e19b80 10 client.25457 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:26:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff511e19b80 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:26:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff511e19b80 20 client.25457 populate_metadata read hostname 'smithi008' 2023-11-27T15:26:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff511e19b80 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f950dfca80 0x55f950dfee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff511e19b80 4 client.25457 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:26:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.677+0000 7ff511e19b80 1 -- 172.21.15.8:0/65651487 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f950cb3600 con 0x55f950dfca80 2023-11-27T15:26:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.678+0000 7ff4f7fff700 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f950dfca80 0x55f950dfee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.678+0000 7ff4f7fff700 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f950dfca80 0x55f950dfee60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67cd86652d303aea server_cookie=acff20ca0f899708 in_seq=0 out_seq=0 2023-11-27T15:26:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.678+0000 7ff4d77fe700 10 client.25457 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff4d77fe700 1 -- 172.21.15.8:0/65651487 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f950cb3600 con 0x55f950dfca80 2023-11-27T15:26:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff4d77fe700 10 client.25457 handle_command_reply: tid=0 2023-11-27T15:26:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff511e19b80 1 client.25457 shutdown 2023-11-27T15:26:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff511e19b80 1 -- 172.21.15.8:0/65651487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d93db0 msgr2=0x55f950d9ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff511e19b80 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d93db0 0x55f950d9ff80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00e970 tx=0x7ff4ec0ca6a0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff511e19b80 1 -- 172.21.15.8:0/65651487 shutdown_connections 2023-11-27T15:26:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff511e19b80 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f950dfca80 0x55f950dfee60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff511e19b80 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f950d94b10 0x55f950da2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff511e19b80 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f950bef6b0 0x55f950d9fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff511e19b80 1 --2- 172.21.15.8:0/65651487 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f950d93db0 0x55f950d9ff80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff4d77fe700 0 client.25457 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:26:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.698+0000 7ff511e19b80 1 -- 172.21.15.8:0/65651487 >> 172.21.15.8:0/65651487 conn(0x55f950d88f50 msgr2=0x55f950d8e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.699+0000 7ff511e19b80 1 -- 172.21.15.8:0/65651487 shutdown_connections 2023-11-27T15:26:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.699+0000 7ff511e19b80 1 -- 172.21.15.8:0/65651487 wait complete. 2023-11-27T15:26:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.699+0000 7ff511e19b80 20 client.25457 trim_cache size 0 max 16384 2023-11-27T15:26:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:26:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:26:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:26:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:26:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/3603783088 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff4e00a1150 msgr2=0x7ff4e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff4e00a1150 0x7ff4e00a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800b2e0 tx=0x7ff4e8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/3603783088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f80ff480 msgr2=0x7ff4f8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f80ff480 0x7ff4f8198760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00e9a0 tx=0x7ff4ec00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/3603783088 shutdown_connections 2023-11-27T15:26:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4f80ffe30 0x7ff4f819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7ff4f80feb30 0x7ff4f8198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7ff4e00a1150 0x7ff4e00a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 --2- 172.21.15.8:0/3603783088 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7ff4f80ff480 0x7ff4f8198760 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/3603783088 >> 172.21.15.8:0/3603783088 conn(0x7ff4f80fa720 msgr2=0x7ff4f80fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/3603783088 shutdown_connections 2023-11-27T15:26:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:26:56.700+0000 7ff4fe3ff700 1 -- 172.21.15.8:0/3603783088 wait complete. 2023-11-27T15:26:57.408 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:57.408 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:57.569 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:57.570 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:57.590 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:57.590 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:57.590 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 36 -lt 100 ']' 2023-11-27T15:26:57.590 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 36 + 1 2023-11-27T15:26:57.601 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=37 2023-11-27T15:26:57.602 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:57.735 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:57.735 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:57.737 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:57.738 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:57.739 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:57 smithi060 ceph-mon[131518]: pgmap v391: 129 pgs: 129 active+clean; 1.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-27T15:26:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:57 smithi008 ceph-mon[128788]: pgmap v391: 129 pgs: 129 active+clean; 1.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-27T15:26:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:57 smithi029 ceph-mon[131166]: pgmap v391: 129 pgs: 129 active+clean; 1.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 9 op/s 2023-11-27T15:26:58.466 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:58.466 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:58.636 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:58.636 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:58.673 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:58.673 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:58.673 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 37 -lt 100 ']' 2023-11-27T15:26:58.673 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 37 + 1 2023-11-27T15:26:58.703 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=38 2023-11-27T15:26:58.703 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:58.718 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:58.718 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:58.721 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:58.721 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:58.722 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:59.216 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:59.217 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:59.394 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:26:59.394 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:26:59.431 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:59.431 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:26:59.431 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 38 -lt 100 ']' 2023-11-27T15:26:59.431 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 38 + 1 2023-11-27T15:26:59.486 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=39 2023-11-27T15:26:59.486 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:26:59.551 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:26:59.551 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:26:59.554 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:26:59.554 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:26:59.555 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:26:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:59 smithi060 ceph-mon[131518]: pgmap v392: 129 pgs: 129 active+clean; 1.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2023-11-27T15:26:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:59 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:26:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:26:59 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:26:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:59 smithi008 ceph-mon[128788]: pgmap v392: 129 pgs: 129 active+clean; 1.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2023-11-27T15:26:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:59 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:26:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:26:59 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:26:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:59 smithi029 ceph-mon[131166]: pgmap v392: 129 pgs: 129 active+clean; 1.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2023-11-27T15:26:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:59 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:26:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:26:59 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:27:00.149 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:00.150 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:00.335 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:00.335 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:00.365 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:00.365 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:00.365 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 39 -lt 100 ']' 2023-11-27T15:27:00.366 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 39 + 1 2023-11-27T15:27:00.394 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=40 2023-11-27T15:27:00.394 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:00.451 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:00.452 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:00.454 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:00.454 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:00.455 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:01.583 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:01.583 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:01.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:01 smithi008 ceph-mon[128788]: pgmap v393: 129 pgs: 129 active+clean; 1.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-11-27T15:27:01.760 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:01.760 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:01.782 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:01.782 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:01.782 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 40 -lt 100 ']' 2023-11-27T15:27:01.783 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 40 + 1 2023-11-27T15:27:01.811 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=41 2023-11-27T15:27:01.811 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:01 smithi060 ceph-mon[131518]: pgmap v393: 129 pgs: 129 active+clean; 1.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-11-27T15:27:01.893 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:01.894 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:01.896 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:01.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:01 smithi029 ceph-mon[131166]: pgmap v393: 129 pgs: 129 active+clean; 1.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-11-27T15:27:01.897 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:01.897 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:02.582 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:02.583 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:02.697 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:02.698 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:02.722 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:02.722 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:02.722 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 41 -lt 100 ']' 2023-11-27T15:27:02.723 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 41 + 1 2023-11-27T15:27:02.763 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=42 2023-11-27T15:27:02.763 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:02.775 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:02.776 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:02.778 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:02.779 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:02.779 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:03.208 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:03.209 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:03.345 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:03.345 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:03.356 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:03.356 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:03.356 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 42 -lt 100 ']' 2023-11-27T15:27:03.356 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 42 + 1 2023-11-27T15:27:03.388 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=43 2023-11-27T15:27:03.389 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:03.400 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:03.400 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:03.403 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:03.403 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:03.404 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:03.667 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:03 smithi008 ceph-mon[128788]: pgmap v394: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-11-27T15:27:03.807 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:03.807 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:03 smithi060 ceph-mon[131518]: pgmap v394: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-11-27T15:27:03.844 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:03.844 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:03.854 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:03.854 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:03.854 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 43 -lt 100 ']' 2023-11-27T15:27:03.855 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 43 + 1 2023-11-27T15:27:03.888 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=44 2023-11-27T15:27:03.888 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:03.895 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:03.895 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:03.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:03 smithi029 ceph-mon[131166]: pgmap v394: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-11-27T15:27:03.898 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:03.898 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:03.899 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:04.299 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:04.299 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:04.426 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:04.426 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:04.437 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:04.437 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:04.437 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 44 -lt 100 ']' 2023-11-27T15:27:04.437 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 44 + 1 2023-11-27T15:27:04.480 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=45 2023-11-27T15:27:04.480 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:04.524 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:04.525 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:04.527 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:04.528 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:04.528 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:04.792 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:04 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:27:04.792 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:04 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:27:04.792 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:04 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:27:04.792 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:04 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:27:04.792 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:04 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:27:04.792 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:04 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:27:04.793 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:04 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:27:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:04 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:27:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:04 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:27:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:04 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:27:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:04 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:27:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:04 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:27:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:04 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:27:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:04 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:27:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:04 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:27:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:04 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:27:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:04 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:27:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:04 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:27:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:04 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:27:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:04 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:27:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:04 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:27:04.906 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:04.907 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:05.026 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:05.027 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:05.038 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:05.039 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:05.039 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 45 -lt 100 ']' 2023-11-27T15:27:05.039 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 45 + 1 2023-11-27T15:27:05.071 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=46 2023-11-27T15:27:05.071 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:05.099 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:05.099 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:05.101 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:05.102 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:05.102 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:05.498 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:05.499 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:05.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:05 smithi008 ceph-mon[128788]: pgmap v395: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-11-27T15:27:05.643 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:05.643 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:05.656 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:05.656 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:05.656 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 46 -lt 100 ']' 2023-11-27T15:27:05.656 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 46 + 1 2023-11-27T15:27:05.688 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=47 2023-11-27T15:27:05.688 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:05.716 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:05.716 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:05.719 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:05.719 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:05.720 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:05.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:05 smithi060 ceph-mon[131518]: pgmap v395: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-11-27T15:27:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:05 smithi029 ceph-mon[131166]: pgmap v395: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 14 op/s 2023-11-27T15:27:06.165 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:06.166 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:06.335 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:06.335 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:06.365 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:06.365 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:06.365 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 47 -lt 100 ']' 2023-11-27T15:27:06.365 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 47 + 1 2023-11-27T15:27:06.413 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=48 2023-11-27T15:27:06.413 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:06.451 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:06.451 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:06.454 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:06.454 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:06.455 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:06.787 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:06.787 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:06.810 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:06.810 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:06.818 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:06.818 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:06.818 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 48 -lt 100 ']' 2023-11-27T15:27:06.819 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 48 + 1 2023-11-27T15:27:06.825 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=49 2023-11-27T15:27:06.825 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:06.835 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:06.835 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:06.838 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:06.838 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:06.839 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:06.964 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:06.964 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:06.987 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:06.988 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:06.994 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:06.994 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:06.994 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 49 -lt 100 ']' 2023-11-27T15:27:06.994 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 49 + 1 2023-11-27T15:27:07.021 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=50 2023-11-27T15:27:07.021 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:07.031 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:07.032 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:07.034 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:07.034 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:07.035 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:07.203 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:07.204 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:07.244 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:07.244 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:07.251 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:07.251 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:07.251 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 50 -lt 100 ']' 2023-11-27T15:27:07.251 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 50 + 1 2023-11-27T15:27:07.257 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=51 2023-11-27T15:27:07.258 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:07.266 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:07.267 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:07.269 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:07.270 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:07.270 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:07.448 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:07.449 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:07.551 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:07.552 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:07.563 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:07.564 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:07.564 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 51 -lt 100 ']' 2023-11-27T15:27:07.564 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 51 + 1 2023-11-27T15:27:07.597 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=52 2023-11-27T15:27:07.597 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:07.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:07 smithi008 ceph-mon[128788]: pgmap v396: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-11-27T15:27:07.626 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:07.626 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:07.628 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:07.629 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:07.629 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:07 smithi060 ceph-mon[131518]: pgmap v396: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-11-27T15:27:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:07 smithi029 ceph-mon[131166]: pgmap v396: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2023-11-27T15:27:07.998 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:07.998 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:08.134 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:08.134 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:08.148 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:08.149 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:08.149 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 52 -lt 100 ']' 2023-11-27T15:27:08.149 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 52 + 1 2023-11-27T15:27:08.179 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=53 2023-11-27T15:27:08.180 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:08.227 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:08.227 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:08.229 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:08.230 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:08.230 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:08.732 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:08.732 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:08.876 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:08.876 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:08.889 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:08.889 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:08.889 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 53 -lt 100 ']' 2023-11-27T15:27:08.890 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 53 + 1 2023-11-27T15:27:08.921 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=54 2023-11-27T15:27:08.922 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:08.967 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:08.968 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:08.970 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:08.971 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:08.971 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:09.357 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:09.357 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:09.467 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:09.468 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:09.486 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:09.486 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:09.486 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 54 -lt 100 ']' 2023-11-27T15:27:09.486 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 54 + 1 2023-11-27T15:27:09.513 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=55 2023-11-27T15:27:09.513 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:09.559 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:09.560 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:09.562 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:09.563 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:09.563 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:09.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:09 smithi060 ceph-mon[131518]: pgmap v397: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-11-27T15:27:09.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:09 smithi008 ceph-mon[128788]: pgmap v397: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-11-27T15:27:09.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:09 smithi029 ceph-mon[131166]: pgmap v397: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 13 op/s 2023-11-27T15:27:10.065 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:10.065 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:10.167 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:10.168 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:10.197 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:10.197 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:10.197 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 55 -lt 100 ']' 2023-11-27T15:27:10.198 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 55 + 1 2023-11-27T15:27:10.227 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=56 2023-11-27T15:27:10.227 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:10.274 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:10.275 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:10.277 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:10.277 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:10.278 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:10.848 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:10.848 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:11.017 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:11.017 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:11.031 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:11.031 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:11.031 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 56 -lt 100 ']' 2023-11-27T15:27:11.031 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 56 + 1 2023-11-27T15:27:11.085 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=57 2023-11-27T15:27:11.085 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:11.176 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:11.176 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:11.178 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:11.179 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:11.179 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:11.756 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:11.756 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:11.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:11 smithi060 ceph-mon[131518]: pgmap v398: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-11-27T15:27:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:11 smithi008 ceph-mon[128788]: pgmap v398: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-11-27T15:27:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:11 smithi029 ceph-mon[131166]: pgmap v398: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-11-27T15:27:11.944 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:11.944 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:12.012 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:12.012 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:12.013 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 57 -lt 100 ']' 2023-11-27T15:27:12.013 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 57 + 1 2023-11-27T15:27:12.118 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=58 2023-11-27T15:27:12.118 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.167+0000 7f4120bff700 1 -- 172.21.15.8:0/3670583673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f411c0ffe30 msgr2=0x7f411c1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.167+0000 7f4120bff700 1 --2- 172.21.15.8:0/3670583673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f411c0ffe30 0x7f411c1002a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f411400b0a0 tx=0x7f411400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.167+0000 7f4120bff700 1 -- 172.21.15.8:0/3670583673 shutdown_connections 2023-11-27T15:27:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.167+0000 7f4120bff700 1 --2- 172.21.15.8:0/3670583673 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f411c0ffe30 0x7f411c1002a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.167+0000 7f4120bff700 1 --2- 172.21.15.8:0/3670583673 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f411c0feb30 0x7f411c0fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.167+0000 7f4120bff700 1 --2- 172.21.15.8:0/3670583673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f411c0ff480 0x7f411c0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.167+0000 7f4120bff700 1 -- 172.21.15.8:0/3670583673 >> 172.21.15.8:0/3670583673 conn(0x7f411c0fa720 msgr2=0x7f411c0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.167+0000 7f4120bff700 1 -- 172.21.15.8:0/3670583673 shutdown_connections 2023-11-27T15:27:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.167+0000 7f4120bff700 1 -- 172.21.15.8:0/3670583673 wait complete. 2023-11-27T15:27:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.169+0000 7f4120bff700 1 Processor -- start 2023-11-27T15:27:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.169+0000 7f4120bff700 1 -- start start 2023-11-27T15:27:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.169+0000 7f4120bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f411c0feb30 0x7f411c198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.169+0000 7f4120bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f411c0ff480 0x7f411c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.169+0000 7f4120bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f411c0ffe30 0x7f411c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.169+0000 7f4120bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c19fcc0 con 0x7f411c0ff480 2023-11-27T15:27:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.169+0000 7f4120bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c19fe00 con 0x7f411c0ffe30 2023-11-27T15:27:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.169+0000 7f4120bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c19ff90 con 0x7f411c0feb30 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f4113fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f411c0ff480 0x7f411c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f4113fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f411c0ff480 0x7f411c198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47026/0 (socket says 172.21.15.8:47026) 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f4113fff700 1 -- 172.21.15.8:0/3418270488 learned_addr learned my addr 172.21.15.8:0/3418270488 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f411a59c700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f411c0feb30 0x7f411c198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f411ad9d700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f411c0ffe30 0x7f411c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f4113fff700 1 -- 172.21.15.8:0/3418270488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f411c0feb30 msgr2=0x7f411c198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f4113fff700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f411c0feb30 0x7f411c198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f4113fff700 1 -- 172.21.15.8:0/3418270488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f411c0ffe30 msgr2=0x7f411c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f4113fff700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f411c0ffe30 0x7f411c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f4113fff700 1 -- 172.21.15.8:0/3418270488 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114009d00 con 0x7f411c0ff480 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.170+0000 7f4113fff700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f411c0ff480 0x7f411c198680 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f410400e9a0 tx=0x7f410400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.171+0000 7f41137fe700 1 -- 172.21.15.8:0/3418270488 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104009790 con 0x7f411c0ff480 2023-11-27T15:27:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.171+0000 7f4120bff700 1 -- 172.21.15.8:0/3418270488 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c10dd30 con 0x7f411c0ff480 2023-11-27T15:27:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.171+0000 7f41137fe700 1 -- 172.21.15.8:0/3418270488 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4104010430 con 0x7f411c0ff480 2023-11-27T15:27:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.171+0000 7f4120bff700 1 -- 172.21.15.8:0/3418270488 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c10dba0 con 0x7f411c0ff480 2023-11-27T15:27:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.171+0000 7f41137fe700 1 -- 172.21.15.8:0/3418270488 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104018620 con 0x7f411c0ff480 2023-11-27T15:27:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.174+0000 7f41137fe700 1 -- 172.21.15.8:0/3418270488 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f41040105a0 con 0x7f411c0ff480 2023-11-27T15:27:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.175+0000 7f41137fe700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f41080a1150 0x7f41080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.176+0000 7f41137fe700 1 -- 172.21.15.8:0/3418270488 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4104014070 con 0x7f411c0ff480 2023-11-27T15:27:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.176+0000 7f411a59c700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f41080a1150 0x7f41080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.179 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.176+0000 7f411a59c700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f41080a1150 0x7f41080a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f410c009c60 tx=0x7f410c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:12.197 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:12.198 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:12.200 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:12.201 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:12.201 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.311+0000 7f4134486b80 1 Processor -- start 2023-11-27T15:27:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.311+0000 7f4134486b80 1 -- start start 2023-11-27T15:27:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.311+0000 7f4134486b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da539156b0 0x55da53ab9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.311+0000 7f4134486b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53ab9db0 0x55da53aba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.311+0000 7f4134486b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53abab10 0x55da53abaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.311+0000 7f4134486b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da539bf000 con 0x55da53abab10 2023-11-27T15:27:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.311+0000 7f4134486b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da53a11840 con 0x55da53ab9db0 2023-11-27T15:27:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.311+0000 7f4134486b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da53a0acd0 con 0x55da539156b0 2023-11-27T15:27:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f411ad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53abab10 0x55da53abaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f411a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da539156b0 0x55da53ab9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f4113fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53ab9db0 0x55da53aba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f411ad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53abab10 0x55da53abaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47028/0 (socket says 172.21.15.8:47028) 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f411ad9d700 1 -- 172.21.15.8:0/4069717322 learned_addr learned my addr 172.21.15.8:0/4069717322 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f411a59c700 1 -- 172.21.15.8:0/4069717322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53ab9db0 msgr2=0x55da53aba5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f411a59c700 1 --2- 172.21.15.8:0/4069717322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53ab9db0 0x55da53aba5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f411a59c700 1 -- 172.21.15.8:0/4069717322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53abab10 msgr2=0x55da53abaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f411a59c700 1 --2- 172.21.15.8:0/4069717322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53abab10 0x55da53abaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f411a59c700 1 -- 172.21.15.8:0/4069717322 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da5388c5d0 con 0x55da539156b0 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.312+0000 7f411ad9d700 1 --2- 172.21.15.8:0/4069717322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53abab10 0x55da53abaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.313+0000 7f411a59c700 1 --2- 172.21.15.8:0/4069717322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da539156b0 0x55da53ab9870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f410c00e3b0 tx=0x7f410c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9afa70d5f4248a5f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.313+0000 7f4110ff9700 1 -- 172.21.15.8:0/4069717322 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c00ed50 con 0x55da539156b0 2023-11-27T15:27:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.313+0000 7f4110ff9700 1 -- 172.21.15.8:0/4069717322 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f410c021070 con 0x55da539156b0 2023-11-27T15:27:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.313+0000 7f4110ff9700 1 -- 172.21.15.8:0/4069717322 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c026470 con 0x55da539156b0 2023-11-27T15:27:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.314+0000 7f4134486b80 1 -- 172.21.15.8:0/4069717322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da539156b0 msgr2=0x55da53ab9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.314+0000 7f4134486b80 1 --2- 172.21.15.8:0/4069717322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da539156b0 0x55da53ab9870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f410c00e3b0 tx=0x7f410c014000 comp rx=0 tx=0).stop 2023-11-27T15:27:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.314+0000 7f4134486b80 1 -- 172.21.15.8:0/4069717322 shutdown_connections 2023-11-27T15:27:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.314+0000 7f4134486b80 1 --2- 172.21.15.8:0/4069717322 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da539156b0 0x55da53ab9870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.314+0000 7f4134486b80 1 --2- 172.21.15.8:0/4069717322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53abab10 0x55da53abaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.314+0000 7f4134486b80 1 --2- 172.21.15.8:0/4069717322 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53ab9db0 0x55da53aba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.314+0000 7f4134486b80 1 -- 172.21.15.8:0/4069717322 >> 172.21.15.8:0/4069717322 conn(0x55da53aaef50 msgr2=0x55da53ab4100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.314+0000 7f4134486b80 1 -- 172.21.15.8:0/4069717322 shutdown_connections 2023-11-27T15:27:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.314+0000 7f4134486b80 1 -- 172.21.15.8:0/4069717322 wait complete. 2023-11-27T15:27:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.315+0000 7f4134486b80 1 -- start start 2023-11-27T15:27:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.315+0000 7f4134486b80 1 Processor -- start 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f4134486b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ac2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f4134486b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 0x55da53ac3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f4134486b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53abab10 0x55da53aca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f4134486b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da539ac880 con 0x55da53ab9db0 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f4134486b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da539ac520 con 0x55da539156b0 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f4134486b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da5388e8c0 con 0x55da53abab10 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f4134486b80 10 client.0 _ll_register_callbacks cb 0x55da539ca720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f4113fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 0x55da53ac3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f411a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ac2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f4113fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 0x55da53ac3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47040/0 (socket says 172.21.15.8:47040) 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f4113fff700 1 -- 172.21.15.8:0/2288577235 learned_addr learned my addr 172.21.15.8:0/2288577235 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f411ad9d700 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53abab10 0x55da53aca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f411a59c700 1 -- 172.21.15.8:0/2288577235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53abab10 msgr2=0x55da53aca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f411a59c700 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53abab10 0x55da53aca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f411a59c700 1 -- 172.21.15.8:0/2288577235 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 msgr2=0x55da53ac3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f411a59c700 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 0x55da53ac3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.316+0000 7f411a59c700 1 -- 172.21.15.8:0/2288577235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c017070 con 0x55da539156b0 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.317+0000 7f411a59c700 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ac2cd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f410c0097c0 tx=0x7f410c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c9763661bdd02c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.317+0000 7f4101ffb700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.317+0000 7f4101ffb700 1 -- 172.21.15.8:0/2288577235 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c02a040 con 0x55da539156b0 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.317+0000 7f4101ffb700 1 -- 172.21.15.8:0/2288577235 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f410c0054d0 con 0x55da539156b0 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.317+0000 7f4134486b80 1 -- 172.21.15.8:0/2288577235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da53aac710 con 0x55da539156b0 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.317+0000 7f4101ffb700 1 -- 172.21.15.8:0/2288577235 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f410c00ee70 con 0x55da539156b0 2023-11-27T15:27:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.318+0000 7f4134486b80 10 client.25469 fetch_fsmap learned FSMap version 24 2023-11-27T15:27:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.318+0000 7f4134486b80 1 -- 172.21.15.8:0/2288577235 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55da539d1130 con 0x55da539156b0 2023-11-27T15:27:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.318+0000 7f4101ffb700 1 -- 172.21.15.8:0/2288577235 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55da53aac710 con 0x55da539156b0 2023-11-27T15:27:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.318+0000 7f4101ffb700 1 -- 172.21.15.8:0/2288577235 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f410c01d730 con 0x55da539156b0 2023-11-27T15:27:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.318+0000 7f4134486b80 10 client.25469 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:27:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.318+0000 7f4134486b80 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:27:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.318+0000 7f4134486b80 20 client.25469 populate_metadata read hostname 'smithi008' 2023-11-27T15:27:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.318+0000 7f4134486b80 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55da53acea00 0x55da53ad0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.318+0000 7f4134486b80 4 client.25469 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.318+0000 7f4134486b80 1 -- 172.21.15.8:0/2288577235 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da539cd040 con 0x55da53acea00 2023-11-27T15:27:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.319+0000 7f4113fff700 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55da53acea00 0x55da53ad0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.319+0000 7f4113fff700 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55da53acea00 0x55da53ad0e10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f44e51f1b29652b0 server_cookie=446cd95f7ab7566 in_seq=0 out_seq=0 2023-11-27T15:27:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.319+0000 7f4101ffb700 10 client.25469 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4101ffb700 1 -- 172.21.15.8:0/2288577235 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55da539cd040 con 0x55da53acea00 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4101ffb700 10 client.25469 handle_command_reply: tid=0 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 client.25469 shutdown 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 -- 172.21.15.8:0/2288577235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 msgr2=0x55da53ac2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ac2cd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f410c0097c0 tx=0x7f410c00ec80 comp rx=0 tx=0).stop 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 -- 172.21.15.8:0/2288577235 shutdown_connections 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55da53acea00 0x55da53ad0e10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53abab10 0x55da53aca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ac2cd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 --2- 172.21.15.8:0/2288577235 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 0x55da53ac3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 -- 172.21.15.8:0/2288577235 >> 172.21.15.8:0/2288577235 conn(0x55da53aaef50 msgr2=0x55da53ab37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4101ffb700 0 client.25469 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 -- 172.21.15.8:0/2288577235 shutdown_connections 2023-11-27T15:27:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 1 -- 172.21.15.8:0/2288577235 wait complete. 2023-11-27T15:27:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.320+0000 7f4134486b80 20 client.25469 trim_cache size 0 max 16384 2023-11-27T15:27:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.337+0000 7f4134486b80 1 Processor -- start 2023-11-27T15:27:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.337+0000 7f4134486b80 1 -- start start 2023-11-27T15:27:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.337+0000 7f4134486b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ad5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.337+0000 7f4134486b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 0x55da53aca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.337+0000 7f4134486b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53abab10 0x55da53abda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.337+0000 7f4134486b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da53ac2180 con 0x55da53ab9db0 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.337+0000 7f4134486b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da53aba420 con 0x55da539156b0 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.337+0000 7f4134486b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da53ab5930 con 0x55da53abab10 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f411a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ad5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f411a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ad5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:58066/0 (socket says 172.21.15.8:58066) 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f411a59c700 1 -- 172.21.15.8:0/1195398880 learned_addr learned my addr 172.21.15.8:0/1195398880 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f4113fff700 1 --2- 172.21.15.8:0/1195398880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 0x55da53aca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f411ad9d700 1 --2- 172.21.15.8:0/1195398880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53abab10 0x55da53abda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f411a59c700 1 -- 172.21.15.8:0/1195398880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53abab10 msgr2=0x55da53abda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f411a59c700 1 --2- 172.21.15.8:0/1195398880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53abab10 0x55da53abda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f411a59c700 1 -- 172.21.15.8:0/1195398880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 msgr2=0x55da53aca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f411a59c700 1 --2- 172.21.15.8:0/1195398880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 0x55da53aca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f411a59c700 1 -- 172.21.15.8:0/1195398880 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da539d0e80 con 0x55da539156b0 2023-11-27T15:27:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.338+0000 7f411a59c700 1 --2- 172.21.15.8:0/1195398880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ad5730 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f410c016040 tx=0x7f410c00ead0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=789423c3a9216696 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f41117fa700 1 -- 172.21.15.8:0/1195398880 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c02a070 con 0x55da539156b0 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f41117fa700 1 -- 172.21.15.8:0/1195398880 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f410c003a10 con 0x55da539156b0 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f41117fa700 1 -- 172.21.15.8:0/1195398880 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c01d8d0 con 0x55da539156b0 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f4134486b80 1 -- 172.21.15.8:0/1195398880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 msgr2=0x55da53ad5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f4134486b80 1 --2- 172.21.15.8:0/1195398880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ad5730 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f410c016040 tx=0x7f410c00ead0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f4134486b80 1 -- 172.21.15.8:0/1195398880 shutdown_connections 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f4134486b80 1 --2- 172.21.15.8:0/1195398880 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53abab10 0x55da53abda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f4134486b80 1 --2- 172.21.15.8:0/1195398880 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da539156b0 0x55da53ad5730 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f4134486b80 1 --2- 172.21.15.8:0/1195398880 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da53ab9db0 0x55da53aca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f4134486b80 1 -- 172.21.15.8:0/1195398880 >> 172.21.15.8:0/1195398880 conn(0x55da53aaef50 msgr2=0x55da53ac1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f4134486b80 1 -- 172.21.15.8:0/1195398880 shutdown_connections 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.339+0000 7f4134486b80 1 -- 172.21.15.8:0/1195398880 wait complete. 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.340+0000 7f4134486b80 1 -- start start 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.340+0000 7f4134486b80 1 Processor -- start 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f4134486b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da539156b0 0x55da53ac8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f4134486b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53ab9db0 0x55da53ac8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f4134486b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53abab10 0x55da53ac8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f4134486b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da53ac96b0 con 0x55da539156b0 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f4134486b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da53ac9820 con 0x55da53abab10 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f4134486b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da53ac99e0 con 0x55da53ab9db0 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f4134486b80 10 client.0 _ll_register_callbacks cb 0x7f4104005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f411a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da539156b0 0x55da53ac8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f411ad9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53abab10 0x55da53ac8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f411a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da539156b0 0x55da53ac8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47066/0 (socket says 172.21.15.8:47066) 2023-11-27T15:27:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f411a59c700 1 -- 172.21.15.8:0/3355063790 learned_addr learned my addr 172.21.15.8:0/3355063790 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.341+0000 7f4113fff700 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53ab9db0 0x55da53ac8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.342+0000 7f411ad9d700 1 -- 172.21.15.8:0/3355063790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53ab9db0 msgr2=0x55da53ac8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.342+0000 7f411ad9d700 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53ab9db0 0x55da53ac8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.342+0000 7f411ad9d700 1 -- 172.21.15.8:0/3355063790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da539156b0 msgr2=0x55da53ac8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.342+0000 7f411ad9d700 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da539156b0 0x55da53ac8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.342+0000 7f411ad9d700 1 -- 172.21.15.8:0/3355063790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da539d1130 con 0x55da53abab10 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.342+0000 7f411ad9d700 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53abab10 0x55da53ac8fa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f411400b070 tx=0x7f4114007870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51048cdd3f676a3b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.342+0000 7f4101ffb700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.342+0000 7f4101ffb700 1 -- 172.21.15.8:0/3355063790 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411400e040 con 0x55da53abab10 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.342+0000 7f4134486b80 1 -- 172.21.15.8:0/3355063790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f410c021070 con 0x55da53abab10 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.342+0000 7f4101ffb700 1 -- 172.21.15.8:0/3355063790 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4114028680 con 0x55da53abab10 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4101ffb700 1 -- 172.21.15.8:0/3355063790 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4114028840 con 0x55da53abab10 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4134486b80 10 client.25475 fetch_fsmap learned FSMap version 24 2023-11-27T15:27:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4134486b80 1 -- 172.21.15.8:0/3355063790 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55da53821c00 con 0x55da53abab10 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4101ffb700 1 -- 172.21.15.8:0/3355063790 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f410c021070 con 0x55da53abab10 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4101ffb700 1 -- 172.21.15.8:0/3355063790 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4114003d20 con 0x55da53abab10 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4134486b80 10 client.25475 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4134486b80 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4134486b80 20 client.25475 populate_metadata read hostname 'smithi008' 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4134486b80 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55da53b23500 0x55da53b25910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4134486b80 4 client.25475 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f4134486b80 1 -- 172.21.15.8:0/3355063790 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55da539d9600 con 0x55da53b23500 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.343+0000 7f411a59c700 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55da53b23500 0x55da53b25910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.344+0000 7f411a59c700 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55da53b23500 0x55da53b25910 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4159d1b15a4579cb server_cookie=5bf61fc1076655b7 in_seq=0 out_seq=0 2023-11-27T15:27:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.344+0000 7f4101ffb700 10 client.25475 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4101ffb700 1 -- 172.21.15.8:0/3355063790 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55da539d9600 con 0x55da53b23500 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4101ffb700 10 client.25475 handle_command_reply: tid=0 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 client.25475 shutdown 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 -- 172.21.15.8:0/3355063790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53abab10 msgr2=0x55da53ac8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53abab10 0x55da53ac8fa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f411400b070 tx=0x7f4114007870 comp rx=0 tx=0).stop 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 -- 172.21.15.8:0/3355063790 shutdown_connections 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55da53abab10 0x55da53ac8fa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55da539156b0 0x55da53ac8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55da53b23500 0x55da53b25910 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 --2- 172.21.15.8:0/3355063790 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55da53ab9db0 0x55da53ac8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4101ffb700 0 client.25475 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 -- 172.21.15.8:0/3355063790 >> 172.21.15.8:0/3355063790 conn(0x55da53aaef50 msgr2=0x55da53ad5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 -- 172.21.15.8:0/3355063790 shutdown_connections 2023-11-27T15:27:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 1 -- 172.21.15.8:0/3355063790 wait complete. 2023-11-27T15:27:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.371+0000 7f4134486b80 20 client.25475 trim_cache size 0 max 16384 2023-11-27T15:27:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:27:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:27:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:27:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:27:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 -- 172.21.15.8:0/3418270488 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f41080a1150 msgr2=0x7f41080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f41080a1150 0x7f41080a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f410c009c60 tx=0x7f410c009260 comp rx=0 tx=0).stop 2023-11-27T15:27:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 -- 172.21.15.8:0/3418270488 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f411c0ff480 msgr2=0x7f411c198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f411c0ff480 0x7f411c198680 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f410400e9a0 tx=0x7f410400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 -- 172.21.15.8:0/3418270488 shutdown_connections 2023-11-27T15:27:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f41080a1150 0x7f41080a3600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f411c0feb30 0x7f411c198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f411c0ffe30 0x7f411c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 --2- 172.21.15.8:0/3418270488 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f411c0ff480 0x7f411c198680 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 -- 172.21.15.8:0/3418270488 >> 172.21.15.8:0/3418270488 conn(0x7f411c0fa720 msgr2=0x7f411c0fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 -- 172.21.15.8:0/3418270488 shutdown_connections 2023-11-27T15:27:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:12.372+0000 7f4120bff700 1 -- 172.21.15.8:0/3418270488 wait complete. 2023-11-27T15:27:12.872 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:12.873 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:13.025 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:13.026 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:13.037 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:13.037 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:13.037 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 58 -lt 100 ']' 2023-11-27T15:27:13.038 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 58 + 1 2023-11-27T15:27:13.070 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=59 2023-11-27T15:27:13.070 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:13.116 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:13.117 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:13.119 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:13.119 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:13.120 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:13.548 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:13.548 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:13.683 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:13.684 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:13.696 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:13.696 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:13.696 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 59 -lt 100 ']' 2023-11-27T15:27:13.696 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 59 + 1 2023-11-27T15:27:13.729 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=60 2023-11-27T15:27:13.729 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:13.775 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:13.775 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:13.778 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:13.778 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:13.778 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:13 smithi060 ceph-mon[131518]: pgmap v399: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-11-27T15:27:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:13 smithi008 ceph-mon[128788]: pgmap v399: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-11-27T15:27:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:13 smithi029 ceph-mon[131166]: pgmap v399: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-11-27T15:27:14.181 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:14.181 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:14.317 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:14.317 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:14.329 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:14.329 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:14.329 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 60 -lt 100 ']' 2023-11-27T15:27:14.330 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 60 + 1 2023-11-27T15:27:14.362 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=61 2023-11-27T15:27:14.362 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:14.399 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:14.399 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:14.402 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:14.402 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:14.402 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:14.747 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:14.748 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:14.917 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:14.917 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:14.971 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:14.972 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:14.972 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 61 -lt 100 ']' 2023-11-27T15:27:14.972 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 61 + 1 2023-11-27T15:27:15.004 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=62 2023-11-27T15:27:15.004 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:15.041 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:15.041 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:15.043 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:15.044 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:15.044 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:15.447 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:15.448 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:15.583 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:15.584 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:15.595 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:15.595 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:15.595 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 62 -lt 100 ']' 2023-11-27T15:27:15.596 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 62 + 1 2023-11-27T15:27:15.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:15 smithi008 ceph-mon[128788]: pgmap v400: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 8 op/s 2023-11-27T15:27:15.628 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=63 2023-11-27T15:27:15.629 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:15.724 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:15.724 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:15.727 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:15.727 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:15.728 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:15 smithi060 ceph-mon[131518]: pgmap v400: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 8 op/s 2023-11-27T15:27:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:15 smithi029 ceph-mon[131166]: pgmap v400: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 KiB/s wr, 8 op/s 2023-11-27T15:27:16.130 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:16.131 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:16.267 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:16.267 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:16.279 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:16.279 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:16.279 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 63 -lt 100 ']' 2023-11-27T15:27:16.280 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 63 + 1 2023-11-27T15:27:16.312 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=64 2023-11-27T15:27:16.312 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:16.358 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:16.358 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:16.361 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:16.361 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:16.362 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:16.772 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:16.772 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:16.917 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:16.917 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:16.928 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:16.928 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:16.928 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 64 -lt 100 ']' 2023-11-27T15:27:16.929 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 64 + 1 2023-11-27T15:27:16.961 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=65 2023-11-27T15:27:16.962 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:17.032 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:17.033 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:17.035 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:17.036 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:17.036 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:17.447 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:17.447 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:17.566 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:17.566 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:17.579 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:17.579 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:17.579 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 65 -lt 100 ']' 2023-11-27T15:27:17.580 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 65 + 1 2023-11-27T15:27:17.612 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=66 2023-11-27T15:27:17.612 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:17.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:17 smithi008 ceph-mon[128788]: pgmap v401: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-11-27T15:27:17.657 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:17.657 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:17.660 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:17.660 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:17.661 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:17 smithi060 ceph-mon[131518]: pgmap v401: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-11-27T15:27:17.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:17 smithi029 ceph-mon[131166]: pgmap v401: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-11-27T15:27:18.047 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:18.047 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:18.183 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:18.183 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:18.195 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:18.195 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:18.195 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 66 -lt 100 ']' 2023-11-27T15:27:18.196 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 66 + 1 2023-11-27T15:27:18.228 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=67 2023-11-27T15:27:18.228 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:18.274 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:18.274 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:18.277 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:18.277 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:18.278 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:18.697 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:18.697 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:18.825 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:18.825 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:18.837 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:18.837 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:18.837 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 67 -lt 100 ']' 2023-11-27T15:27:18.837 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 67 + 1 2023-11-27T15:27:18.871 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=68 2023-11-27T15:27:18.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:18.915 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:18.916 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:18.918 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:18.919 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:18.919 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:19.422 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:19.422 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:19.566 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:19.566 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:19 smithi060 ceph-mon[131518]: pgmap v402: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-11-27T15:27:19.578 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:19.578 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:19.579 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 68 -lt 100 ']' 2023-11-27T15:27:19.579 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 68 + 1 2023-11-27T15:27:19.611 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=69 2023-11-27T15:27:19.611 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:19.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:19 smithi008 ceph-mon[128788]: pgmap v402: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-11-27T15:27:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:19 smithi029 ceph-mon[131166]: pgmap v402: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-11-27T15:27:19.658 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:19.658 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:19.661 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:19.661 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:19.662 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:20.088 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:20.089 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:20.249 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:20.250 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:20.262 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:20.262 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:20.262 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 69 -lt 100 ']' 2023-11-27T15:27:20.263 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 69 + 1 2023-11-27T15:27:20.292 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=70 2023-11-27T15:27:20.292 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:20.332 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:20.332 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:20.334 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:20.335 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:20.335 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:20.813 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:20.814 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:20.941 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:20.941 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:20.951 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:20.951 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:20.951 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 70 -lt 100 ']' 2023-11-27T15:27:20.951 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 70 + 1 2023-11-27T15:27:20.986 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=71 2023-11-27T15:27:20.986 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:21.023 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:21.023 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:21.025 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:21.026 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:21.026 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:21.513 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:21.514 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:21.724 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:21.724 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:21.737 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:21.737 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:21.737 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 71 -lt 100 ']' 2023-11-27T15:27:21.738 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 71 + 1 2023-11-27T15:27:21.769 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=72 2023-11-27T15:27:21.769 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:21.797 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:21.797 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:21.800 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:21.801 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:21.801 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:21 smithi060 ceph-mon[131518]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-11-27T15:27:21.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:21 smithi008 ceph-mon[128788]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-11-27T15:27:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:21 smithi029 ceph-mon[131166]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-11-27T15:27:22.230 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:22.230 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:22.357 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:22.358 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:22.369 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:22.370 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:22.370 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 72 -lt 100 ']' 2023-11-27T15:27:22.370 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 72 + 1 2023-11-27T15:27:22.403 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=73 2023-11-27T15:27:22.403 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:22.456 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:22.457 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:22.459 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:22.460 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:22.460 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:22.863 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:22.863 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:23.016 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:23.016 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:23.037 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:23.037 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:23.037 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 73 -lt 100 ']' 2023-11-27T15:27:23.038 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 73 + 1 2023-11-27T15:27:23.070 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=74 2023-11-27T15:27:23.070 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:23.098 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:23.098 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:23.100 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:23.101 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:23.102 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:23.505 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:23.505 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:23.624 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:23.624 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:23.636 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:23.636 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:23.636 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 74 -lt 100 ']' 2023-11-27T15:27:23.636 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 74 + 1 2023-11-27T15:27:23.669 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=75 2023-11-27T15:27:23.669 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:23.706 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:23.707 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:23.709 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:23.710 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:23.710 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:23 smithi060 ceph-mon[131518]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-11-27T15:27:23.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:23 smithi008 ceph-mon[128788]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-11-27T15:27:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:23 smithi029 ceph-mon[131166]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-11-27T15:27:24.113 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:24.113 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:24.324 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:24.324 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:24.338 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:24.338 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:24.338 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 75 -lt 100 ']' 2023-11-27T15:27:24.339 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 75 + 1 2023-11-27T15:27:24.369 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=76 2023-11-27T15:27:24.369 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:24.407 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:24.407 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:24.410 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:24.410 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:24.411 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:24.896 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:24.896 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:25.057 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:25.057 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:25.087 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:25.087 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:25.087 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 76 -lt 100 ']' 2023-11-27T15:27:25.087 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 76 + 1 2023-11-27T15:27:25.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=77 2023-11-27T15:27:25.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:25.199 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:25.199 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:25.201 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:25.202 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:25.202 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:25.679 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:25.679 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:25.823 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:25.824 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:25.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:25 smithi060 ceph-mon[131518]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-11-27T15:27:25.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:25 smithi008 ceph-mon[128788]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-11-27T15:27:25.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:25.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:25.872 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 77 -lt 100 ']' 2023-11-27T15:27:25.872 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 77 + 1 2023-11-27T15:27:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:25 smithi029 ceph-mon[131166]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-11-27T15:27:25.916 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=78 2023-11-27T15:27:25.916 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:25.951 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:25.951 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:25.954 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:25.954 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:25.955 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:26.538 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:26.538 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:26.815 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:26.815 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:26.827 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:26.828 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:26.828 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 78 -lt 100 ']' 2023-11-27T15:27:26.828 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 78 + 1 2023-11-27T15:27:26.866 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=79 2023-11-27T15:27:26.866 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:26.883 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:26.883 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:26.885 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:26.886 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:26.886 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.640+0000 7f47ccdff700 1 -- 172.21.15.8:0/2821728356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47c81014c0 msgr2=0x7f47c8101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.640+0000 7f47ccdff700 1 --2- 172.21.15.8:0/2821728356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47c81014c0 0x7f47c8101930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0098f0 tx=0x7f47bc009c00 comp rx=0 tx=0).stop 2023-11-27T15:27:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.640+0000 7f47ccdff700 1 -- 172.21.15.8:0/2821728356 shutdown_connections 2023-11-27T15:27:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.640+0000 7f47ccdff700 1 --2- 172.21.15.8:0/2821728356 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47c8101e70 0x7f47c8108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.640+0000 7f47ccdff700 1 --2- 172.21.15.8:0/2821728356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47c8100230 0x7f47c8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.640+0000 7f47ccdff700 1 --2- 172.21.15.8:0/2821728356 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47c81014c0 0x7f47c8101930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.640+0000 7f47ccdff700 1 -- 172.21.15.8:0/2821728356 >> 172.21.15.8:0/2821728356 conn(0x7f47c80fb810 msgr2=0x7f47c80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.640+0000 7f47ccdff700 1 -- 172.21.15.8:0/2821728356 shutdown_connections 2023-11-27T15:27:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.640+0000 7f47ccdff700 1 -- 172.21.15.8:0/2821728356 wait complete. 2023-11-27T15:27:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.642+0000 7f47ccdff700 1 Processor -- start 2023-11-27T15:27:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.642+0000 7f47ccdff700 1 -- start start 2023-11-27T15:27:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.642+0000 7f47ccdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47c8100230 0x7f47c80748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.642+0000 7f47ccdff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47c81014c0 0x7f47c8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.642+0000 7f47ccdff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47c8101e70 0x7f47c8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.642+0000 7f47ccdff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8073bc0 con 0x7f47c81014c0 2023-11-27T15:27:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.642+0000 7f47ccdff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8073d30 con 0x7f47c8101e70 2023-11-27T15:27:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.642+0000 7f47ccdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81a00b0 con 0x7f47c8100230 2023-11-27T15:27:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.643+0000 7f47c5d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47c81014c0 0x7f47c8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.643+0000 7f47c5d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47c81014c0 0x7f47c8072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46046/0 (socket says 172.21.15.8:46046) 2023-11-27T15:27:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.643+0000 7f47c659c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47c8100230 0x7f47c80748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.643+0000 7f47c5d9b700 1 -- 172.21.15.8:0/532922959 learned_addr learned my addr 172.21.15.8:0/532922959 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:27.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.643+0000 7f47c6d9d700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47c8101e70 0x7f47c8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.643+0000 7f47c5d9b700 1 -- 172.21.15.8:0/532922959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47c8100230 msgr2=0x7f47c80748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.643+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47c8100230 0x7f47c80748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.643+0000 7f47c5d9b700 1 -- 172.21.15.8:0/532922959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47c8101e70 msgr2=0x7f47c8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.643+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47c8101e70 0x7f47c8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.643+0000 7f47c5d9b700 1 -- 172.21.15.8:0/532922959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b8009520 con 0x7f47c81014c0 2023-11-27T15:27:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.644+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47c81014c0 0x7f47c8072ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009fd0 tx=0x7f47bc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.644+0000 7f47b77fe700 1 -- 172.21.15.8:0/532922959 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc01c070 con 0x7f47c81014c0 2023-11-27T15:27:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.644+0000 7f47ccdff700 1 -- 172.21.15.8:0/532922959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c0009d00 con 0x7f47c81014c0 2023-11-27T15:27:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.644+0000 7f47b77fe700 1 -- 172.21.15.8:0/532922959 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47bc00f7d0 con 0x7f47c81014c0 2023-11-27T15:27:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.644+0000 7f47b77fe700 1 -- 172.21.15.8:0/532922959 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc004ad0 con 0x7f47c81014c0 2023-11-27T15:27:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.644+0000 7f47ccdff700 1 -- 172.21.15.8:0/532922959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47bc009520 con 0x7f47c81014c0 2023-11-27T15:27:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.647+0000 7f47b77fe700 1 -- 172.21.15.8:0/532922959 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f47bc020070 con 0x7f47c81014c0 2023-11-27T15:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.649+0000 7f47b77fe700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47b00a1150 0x7f47b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.649+0000 7f47b77fe700 1 -- 172.21.15.8:0/532922959 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f47bc0cd8c0 con 0x7f47c81014c0 2023-11-27T15:27:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.649+0000 7f47c659c700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47b00a1150 0x7f47b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.650+0000 7f47c659c700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47b00a1150 0x7f47b00a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009df0 tx=0x7f47b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:27.740 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:27 smithi008 ceph-mon[128788]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-11-27T15:27:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:27 smithi060 ceph-mon[131518]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.829+0000 7f47e068ab80 1 Processor -- start 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47e068ab80 1 -- start start 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b29626b0 0x55e1b2b06870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b2b07b10 0x55e1b2b07f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47e068ab80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b2a0c000 con 0x55e1b2b07b10 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47e068ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b2a5e840 con 0x55e1b29626b0 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47e068ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b2a57cd0 con 0x55e1b2b06db0 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47c659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b29626b0 0x55e1b2b06870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47c6d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b2b07b10 0x55e1b2b07f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47c659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b29626b0 0x55e1b2b06870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:59552/0 (socket says 172.21.15.8:59552) 2023-11-27T15:27:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47c659c700 1 -- 172.21.15.8:0/1559804044 learned_addr learned my addr 172.21.15.8:0/1559804044 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:27.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.830+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/1559804044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.831+0000 7f47c6d9d700 1 -- 172.21.15.8:0/1559804044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 msgr2=0x55e1b2b075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.831+0000 7f47c6d9d700 1 --2- 172.21.15.8:0/1559804044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b075d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.831+0000 7f47c6d9d700 1 -- 172.21.15.8:0/1559804044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b29626b0 msgr2=0x55e1b2b06870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.831+0000 7f47c6d9d700 1 --2- 172.21.15.8:0/1559804044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b29626b0 0x55e1b2b06870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.831+0000 7f47c6d9d700 1 -- 172.21.15.8:0/1559804044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b28d95d0 con 0x55e1b2b07b10 2023-11-27T15:27:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.831+0000 7f47c6d9d700 1 --2- 172.21.15.8:0/1559804044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b2b07b10 0x55e1b2b07f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009fd0 tx=0x7f47c00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c022d9bb621694 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.831+0000 7f47b4ff9700 1 -- 172.21.15.8:0/1559804044 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000e030 con 0x55e1b2b07b10 2023-11-27T15:27:27.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.831+0000 7f47b4ff9700 1 -- 172.21.15.8:0/1559804044 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c000f040 con 0x55e1b2b07b10 2023-11-27T15:27:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.831+0000 7f47b4ff9700 1 -- 172.21.15.8:0/1559804044 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0005280 con 0x55e1b2b07b10 2023-11-27T15:27:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.832+0000 7f47e068ab80 1 -- 172.21.15.8:0/1559804044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b2b07b10 msgr2=0x55e1b2b07f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.832+0000 7f47e068ab80 1 --2- 172.21.15.8:0/1559804044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b2b07b10 0x55e1b2b07f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009fd0 tx=0x7f47c00099e0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.832+0000 7f47e068ab80 1 -- 172.21.15.8:0/1559804044 shutdown_connections 2023-11-27T15:27:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.832+0000 7f47e068ab80 1 --2- 172.21.15.8:0/1559804044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b2b07b10 0x55e1b2b07f80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.832+0000 7f47e068ab80 1 --2- 172.21.15.8:0/1559804044 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b29626b0 0x55e1b2b06870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.832+0000 7f47e068ab80 1 --2- 172.21.15.8:0/1559804044 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.832+0000 7f47e068ab80 1 -- 172.21.15.8:0/1559804044 >> 172.21.15.8:0/1559804044 conn(0x55e1b2afbf50 msgr2=0x55e1b2b01100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.832+0000 7f47e068ab80 1 -- 172.21.15.8:0/1559804044 shutdown_connections 2023-11-27T15:27:27.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.832+0000 7f47e068ab80 1 -- 172.21.15.8:0/1559804044 wait complete. 2023-11-27T15:27:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.845+0000 7f47e068ab80 1 -- start start 2023-11-27T15:27:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.845+0000 7f47e068ab80 1 Processor -- start 2023-11-27T15:27:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b0fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 0x55e1b2b17230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47e068ab80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b29f9880 con 0x55e1b29626b0 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47e068ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b29f9520 con 0x55e1b2b07b10 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47e068ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b28db8c0 con 0x55e1b2b06db0 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47e068ab80 10 client.0 _ll_register_callbacks cb 0x55e1b2a17720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47c659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b0fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47c659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b0fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46068/0 (socket says 172.21.15.8:46068) 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47c659c700 1 -- 172.21.15.8:0/3643933974 learned_addr learned my addr 172.21.15.8:0/3643933974 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47c5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.846+0000 7f47c6d9d700 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 0x55e1b2b17230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47c5d9b700 1 -- 172.21.15.8:0/3643933974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 msgr2=0x55e1b2b17230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:27.851 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 0x55e1b2b17230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47c5d9b700 1 -- 172.21.15.8:0/3643933974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 msgr2=0x55e1b2b0fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b0fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47c5d9b700 1 -- 172.21.15.8:0/3643933974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c0009160 con 0x55e1b2b06db0 2023-11-27T15:27:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47c659c700 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b0fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47c6d9d700 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 0x55e1b2b17230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b101f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0cb030 tx=0x7f47bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7763884e71b1e69 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47adffb700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:27:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47adffb700 1 -- 172.21.15.8:0/3643933974 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc09ea00 con 0x55e1b2b06db0 2023-11-27T15:27:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47e068ab80 1 -- 172.21.15.8:0/3643933974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1b2af9710 con 0x55e1b2b06db0 2023-11-27T15:27:27.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.847+0000 7f47adffb700 1 -- 172.21.15.8:0/3643933974 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f47bc0a2d90 con 0x55e1b2b06db0 2023-11-27T15:27:27.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.858+0000 7f47adffb700 1 -- 172.21.15.8:0/3643933974 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47bc09ac30 con 0x55e1b2b06db0 2023-11-27T15:27:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.858+0000 7f47e068ab80 10 client.25510 fetch_fsmap learned FSMap version 24 2023-11-27T15:27:27.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.858+0000 7f47e068ab80 1 -- 172.21.15.8:0/3643933974 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e1b2a1e130 con 0x55e1b2b06db0 2023-11-27T15:27:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.859+0000 7f47adffb700 1 -- 172.21.15.8:0/3643933974 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1b2af9710 con 0x55e1b2b06db0 2023-11-27T15:27:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.859+0000 7f47adffb700 1 -- 172.21.15.8:0/3643933974 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47bc0a2900 con 0x55e1b2b06db0 2023-11-27T15:27:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.859+0000 7f47e068ab80 10 client.25510 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:27:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.859+0000 7f47e068ab80 10 client.25510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:27:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.859+0000 7f47e068ab80 20 client.25510 populate_metadata read hostname 'smithi008' 2023-11-27T15:27:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.859+0000 7f47e068ab80 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e1b2b1b6c0 0x55e1b2b1daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.859+0000 7f47e068ab80 4 client.25510 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.859+0000 7f47e068ab80 1 -- 172.21.15.8:0/3643933974 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1b2a1a040 con 0x55e1b2b1b6c0 2023-11-27T15:27:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.859+0000 7f47c659c700 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e1b2b1b6c0 0x55e1b2b1daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.860+0000 7f47c659c700 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e1b2b1b6c0 0x55e1b2b1daa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b62f2d16cdf07e server_cookie=aad97e28ae6feda5 in_seq=0 out_seq=0 2023-11-27T15:27:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.860+0000 7f47adffb700 10 client.25510 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:27.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.860+0000 7f47adffb700 1 -- 172.21.15.8:0/3643933974 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e1b2a1a040 con 0x55e1b2b1b6c0 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.860+0000 7f47adffb700 10 client.25510 handle_command_reply: tid=0 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.860+0000 7f47e068ab80 1 client.25510 shutdown 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 1 -- 172.21.15.8:0/3643933974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 msgr2=0x55e1b2b101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b101f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0cb030 tx=0x7f47bc009260 comp rx=0 tx=0).stop 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 1 -- 172.21.15.8:0/3643933974 shutdown_connections 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 0x55e1b2b17230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b0fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e1b2b1b6c0 0x55e1b2b1daa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 1 --2- 172.21.15.8:0/3643933974 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b101f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47adffb700 0 client.25510 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 1 -- 172.21.15.8:0/3643933974 >> 172.21.15.8:0/3643933974 conn(0x55e1b2afbf50 msgr2=0x55e1b2b007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 1 -- 172.21.15.8:0/3643933974 shutdown_connections 2023-11-27T15:27:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 1 -- 172.21.15.8:0/3643933974 wait complete. 2023-11-27T15:27:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.861+0000 7f47e068ab80 20 client.25510 trim_cache size 0 max 16384 2023-11-27T15:27:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47e068ab80 1 Processor -- start 2023-11-27T15:27:27.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47e068ab80 1 -- start start 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b07ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b06db0 0x55e1b2b171f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b07b10 0x55e1b2b0e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47e068ab80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b2b0e7a0 con 0x55e1b29626b0 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47e068ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b2b07420 con 0x55e1b2b06db0 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47e068ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b2b02bf0 con 0x55e1b2b07b10 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47c659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b07ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47c659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b07ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46084/0 (socket says 172.21.15.8:46084) 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47c659c700 1 -- 172.21.15.8:0/4183439669 learned_addr learned my addr 172.21.15.8:0/4183439669 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.877+0000 7f47c6d9d700 1 --2- 172.21.15.8:0/4183439669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b07b10 0x55e1b2b0e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/4183439669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b06db0 0x55e1b2b171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47c659c700 1 -- 172.21.15.8:0/4183439669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b07b10 msgr2=0x55e1b2b0e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47c659c700 1 --2- 172.21.15.8:0/4183439669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b07b10 0x55e1b2b0e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47c659c700 1 -- 172.21.15.8:0/4183439669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b06db0 msgr2=0x55e1b2b171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47c659c700 1 --2- 172.21.15.8:0/4183439669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b06db0 0x55e1b2b171f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47c659c700 1 -- 172.21.15.8:0/4183439669 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b2a1de80 con 0x55e1b29626b0 2023-11-27T15:27:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/4183439669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b06db0 0x55e1b2b171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47c659c700 1 --2- 172.21.15.8:0/4183439669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b07ee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009bd0 tx=0x7f47b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2796501fdc73f382 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47b57fa700 1 -- 172.21.15.8:0/4183439669 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8022030 con 0x55e1b29626b0 2023-11-27T15:27:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47b57fa700 1 -- 172.21.15.8:0/4183439669 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b8023040 con 0x55e1b29626b0 2023-11-27T15:27:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.878+0000 7f47b57fa700 1 -- 172.21.15.8:0/4183439669 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8003a00 con 0x55e1b29626b0 2023-11-27T15:27:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.879+0000 7f47e068ab80 1 -- 172.21.15.8:0/4183439669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 msgr2=0x55e1b2b07ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.879+0000 7f47e068ab80 1 --2- 172.21.15.8:0/4183439669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b07ee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009bd0 tx=0x7f47b8014000 comp rx=0 tx=0).stop 2023-11-27T15:27:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.879+0000 7f47e068ab80 1 -- 172.21.15.8:0/4183439669 shutdown_connections 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.879+0000 7f47e068ab80 1 --2- 172.21.15.8:0/4183439669 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b07b10 0x55e1b2b0e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.879+0000 7f47e068ab80 1 --2- 172.21.15.8:0/4183439669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b07ee0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.879+0000 7f47e068ab80 1 --2- 172.21.15.8:0/4183439669 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b06db0 0x55e1b2b171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.879+0000 7f47e068ab80 1 -- 172.21.15.8:0/4183439669 >> 172.21.15.8:0/4183439669 conn(0x55e1b2afbf50 msgr2=0x55e1b2b22400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.879+0000 7f47e068ab80 1 -- 172.21.15.8:0/4183439669 shutdown_connections 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.879+0000 7f47e068ab80 1 -- 172.21.15.8:0/4183439669 wait complete. 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.879+0000 7f47e068ab80 1 -- start start 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47e068ab80 1 Processor -- start 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b15140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b6b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47e068ab80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 0x55e1b2b6db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47e068ab80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b2b6e280 con 0x55e1b29626b0 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47e068ab80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b2b6e3f0 con 0x55e1b2b07b10 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47e068ab80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b2b6e5b0 con 0x55e1b2b06db0 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47e068ab80 10 client.0 _ll_register_callbacks cb 0x7f47bc09a890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47c659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b15140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47c659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b15140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46090/0 (socket says 172.21.15.8:46090) 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.880+0000 7f47c659c700 1 -- 172.21.15.8:0/1431447350 learned_addr learned my addr 172.21.15.8:0/1431447350 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b6b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47c6d9d700 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 0x55e1b2b6db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47c659c700 1 -- 172.21.15.8:0/1431447350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 msgr2=0x55e1b2b6b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47c659c700 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b6b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47c659c700 1 -- 172.21.15.8:0/1431447350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 msgr2=0x55e1b2b6db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47c659c700 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 0x55e1b2b6db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47c659c700 1 -- 172.21.15.8:0/1431447350 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b8017070 con 0x55e1b29626b0 2023-11-27T15:27:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b6b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47c659c700 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b15140 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f47b8005d90 tx=0x7f47b8029df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a518469ba528b1fc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47adffb700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.881+0000 7f47adffb700 1 -- 172.21.15.8:0/1431447350 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8036070 con 0x55e1b29626b0 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47adffb700 1 -- 172.21.15.8:0/1431447350 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f47b8004250 con 0x55e1b29626b0 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47e068ab80 1 -- 172.21.15.8:0/1431447350 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47c000f040 con 0x55e1b29626b0 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47adffb700 1 -- 172.21.15.8:0/1431447350 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b8022040 con 0x55e1b29626b0 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47e068ab80 10 client.15468 fetch_fsmap learned FSMap version 24 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47e068ab80 1 -- 172.21.15.8:0/1431447350 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e1b286ec00 con 0x55e1b29626b0 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47adffb700 1 -- 172.21.15.8:0/1431447350 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c000f040 con 0x55e1b29626b0 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47adffb700 1 -- 172.21.15.8:0/1431447350 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47b8039020 con 0x55e1b29626b0 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47e068ab80 10 client.15468 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47e068ab80 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:27:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47e068ab80 20 client.15468 populate_metadata read hostname 'smithi008' 2023-11-27T15:27:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47e068ab80 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e1b2b6fe90 0x55e1b2b722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47e068ab80 4 client.15468 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:27:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.882+0000 7f47e068ab80 1 -- 172.21.15.8:0/1431447350 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e1b2a26600 con 0x55e1b2b6fe90 2023-11-27T15:27:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.883+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e1b2b6fe90 0x55e1b2b722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.883+0000 7f47adffb700 10 client.15468 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.883+0000 7f47c5d9b700 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e1b2b6fe90 0x55e1b2b722a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b461a9269dcc5ed8 server_cookie=4f2e495192e73de0 in_seq=0 out_seq=0 2023-11-27T15:27:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:27 smithi029 ceph-mon[131166]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47adffb700 1 -- 172.21.15.8:0/1431447350 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e1b2a26600 con 0x55e1b2b6fe90 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47adffb700 10 client.15468 handle_command_reply: tid=0 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 client.15468 shutdown 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 -- 172.21.15.8:0/1431447350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 msgr2=0x55e1b2b15140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b15140 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f47b8005d90 tx=0x7f47b8029df0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 -- 172.21.15.8:0/1431447350 shutdown_connections 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55e1b2b07b10 0x55e1b2b6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55e1b29626b0 0x55e1b2b15140 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55e1b2b6fe90 0x55e1b2b722a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 --2- 172.21.15.8:0/1431447350 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55e1b2b06db0 0x55e1b2b6b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47adffb700 0 client.15468 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 -- 172.21.15.8:0/1431447350 >> 172.21.15.8:0/1431447350 conn(0x55e1b2afbf50 msgr2=0x55e1b2b0e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 -- 172.21.15.8:0/1431447350 shutdown_connections 2023-11-27T15:27:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 1 -- 172.21.15.8:0/1431447350 wait complete. 2023-11-27T15:27:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.908+0000 7f47e068ab80 20 client.15468 trim_cache size 0 max 16384 2023-11-27T15:27:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:27:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:27:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:27:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:27:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 -- 172.21.15.8:0/532922959 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47b00a1150 msgr2=0x7f47b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47b00a1150 0x7f47b00a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009df0 tx=0x7f47b8009260 comp rx=0 tx=0).stop 2023-11-27T15:27:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 -- 172.21.15.8:0/532922959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47c81014c0 msgr2=0x7f47c8072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47c81014c0 0x7f47c8072ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f47bc009fd0 tx=0x7f47bc00b790 comp rx=0 tx=0).stop 2023-11-27T15:27:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 -- 172.21.15.8:0/532922959 shutdown_connections 2023-11-27T15:27:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47b00a1150 0x7f47b00a3600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f47c8100230 0x7f47c80748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f47c8101e70 0x7f47c8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 --2- 172.21.15.8:0/532922959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47c81014c0 0x7f47c8072ee0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 -- 172.21.15.8:0/532922959 >> 172.21.15.8:0/532922959 conn(0x7f47c80fb810 msgr2=0x7f47c81049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 -- 172.21.15.8:0/532922959 shutdown_connections 2023-11-27T15:27:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:27.909+0000 7f47ccdff700 1 -- 172.21.15.8:0/532922959 wait complete. 2023-11-27T15:27:27.920 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:27.921 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:28.091 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:28.092 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:28.100 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:28.101 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:28.101 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 79 -lt 100 ']' 2023-11-27T15:27:28.101 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 79 + 1 2023-11-27T15:27:28.135 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=80 2023-11-27T15:27:28.135 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:28.163 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:28.163 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:28.165 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:28.166 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:28.166 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:28.595 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:28.596 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:28.646 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:28.646 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:28.651 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:28.651 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:28.652 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 80 -lt 100 ']' 2023-11-27T15:27:28.652 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 80 + 1 2023-11-27T15:27:28.658 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=81 2023-11-27T15:27:28.658 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:28.669 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:28.669 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:28.672 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:28.672 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:28.672 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:28.811 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:28.812 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:28.873 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:28.873 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:28.885 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:28.885 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:28.885 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 81 -lt 100 ']' 2023-11-27T15:27:28.885 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 81 + 1 2023-11-27T15:27:28.918 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=82 2023-11-27T15:27:28.918 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:28.947 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:28.947 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:28.949 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:28.950 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:28.950 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:29.295 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:29.296 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:29.424 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:29.424 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:29.438 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:29.438 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:29.438 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 82 -lt 100 ']' 2023-11-27T15:27:29.438 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 82 + 1 2023-11-27T15:27:29.468 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=83 2023-11-27T15:27:29.469 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:29.479 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:29.480 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:29.482 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:29.483 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:29.483 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:29.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:29 smithi060 ceph-mon[131518]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-27T15:27:29.590 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:29 smithi008 ceph-mon[128788]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-27T15:27:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:29 smithi029 ceph-mon[131166]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 8 op/s 2023-11-27T15:27:29.652 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:29.653 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:29.773 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:29.773 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:29.812 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:29.812 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:29.812 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 83 -lt 100 ']' 2023-11-27T15:27:29.813 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 83 + 1 2023-11-27T15:27:29.869 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=84 2023-11-27T15:27:29.869 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:29.908 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:29.908 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:29.910 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:29.911 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:29.912 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:30.212 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:30.213 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:30.365 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:30.365 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:30.377 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:30.378 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:30.378 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 84 -lt 100 ']' 2023-11-27T15:27:30.378 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 84 + 1 2023-11-27T15:27:30.407 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=85 2023-11-27T15:27:30.407 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:30.463 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:30.463 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:30.465 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:30.466 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:30.466 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:30.837 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:30.837 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:30.990 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:30.990 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:31.002 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:31.002 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:31.002 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 85 -lt 100 ']' 2023-11-27T15:27:31.003 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 85 + 1 2023-11-27T15:27:31.043 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=86 2023-11-27T15:27:31.043 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:31.097 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:31.098 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:31.100 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:31.101 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:31.101 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:31.478 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:31.479 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:31.598 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:31.598 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:31.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:31 smithi008 ceph-mon[128788]: pgmap v408: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-27T15:27:31.613 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:31.613 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:31.613 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 86 -lt 100 ']' 2023-11-27T15:27:31.613 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 86 + 1 2023-11-27T15:27:31.643 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=87 2023-11-27T15:27:31.643 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:31.671 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:31.671 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:31.674 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:31.674 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:31.675 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:31 smithi060 ceph-mon[131518]: pgmap v408: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-27T15:27:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:31 smithi029 ceph-mon[131166]: pgmap v408: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 12 op/s 2023-11-27T15:27:32.145 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:32.146 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:32.281 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:32.281 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:32.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:32.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:32.294 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 87 -lt 100 ']' 2023-11-27T15:27:32.295 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 87 + 1 2023-11-27T15:27:32.326 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=88 2023-11-27T15:27:32.327 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:32.374 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:32.374 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:32.377 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:32.377 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:32.378 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:32.745 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:32.745 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:32.889 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:32.890 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:32.902 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:32.902 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:32.902 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 88 -lt 100 ']' 2023-11-27T15:27:32.903 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 88 + 1 2023-11-27T15:27:32.951 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=89 2023-11-27T15:27:32.951 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:32.989 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:32.990 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:32.992 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:32.993 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:32.993 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:33.562 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:33.562 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:33.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:33 smithi008 ceph-mon[128788]: pgmap v409: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-11-27T15:27:33.698 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:33.698 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:33.709 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:33.709 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:33.710 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 89 -lt 100 ']' 2023-11-27T15:27:33.710 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 89 + 1 2023-11-27T15:27:33.751 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=90 2023-11-27T15:27:33.751 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:33.788 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:33.788 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:33.791 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:33.791 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:33.792 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:33 smithi060 ceph-mon[131518]: pgmap v409: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-11-27T15:27:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:33 smithi029 ceph-mon[131166]: pgmap v409: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-11-27T15:27:34.328 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:34.328 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:34.439 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:34.439 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:34.452 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:34.452 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:34.452 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 90 -lt 100 ']' 2023-11-27T15:27:34.452 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 90 + 1 2023-11-27T15:27:34.484 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=91 2023-11-27T15:27:34.484 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:34.512 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:34.513 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:34.515 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:34.516 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:34.516 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:34.928 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:34.928 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:35.064 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:35.064 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:35.076 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:35.076 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:35.076 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 91 -lt 100 ']' 2023-11-27T15:27:35.077 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 91 + 1 2023-11-27T15:27:35.109 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=92 2023-11-27T15:27:35.109 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:35.154 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:35.155 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:35.157 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:35.158 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:35.158 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:35.661 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:35.662 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:35.797 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:35.797 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:35.810 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:35.810 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:35.810 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 92 -lt 100 ']' 2023-11-27T15:27:35.810 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 92 + 1 2023-11-27T15:27:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:35 smithi060 ceph-mon[131518]: pgmap v410: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-11-27T15:27:35.851 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=93 2023-11-27T15:27:35.851 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:35.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:35 smithi008 ceph-mon[128788]: pgmap v410: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-11-27T15:27:35.889 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:35.889 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:35.892 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:35.892 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:35.893 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:35 smithi029 ceph-mon[131166]: pgmap v410: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2023-11-27T15:27:36.353 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:36.353 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:36.489 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:36.489 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:36.501 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:36.501 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:36.501 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 93 -lt 100 ']' 2023-11-27T15:27:36.501 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 93 + 1 2023-11-27T15:27:36.543 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=94 2023-11-27T15:27:36.543 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:36.579 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:36.579 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:36.582 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:36.582 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:36.583 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:36.919 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:36.920 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:37.064 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:37.064 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:37.085 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:37.085 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:37.085 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 94 -lt 100 ']' 2023-11-27T15:27:37.085 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 94 + 1 2023-11-27T15:27:37.117 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=95 2023-11-27T15:27:37.118 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:37.162 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:37.162 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:37.165 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:37.165 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:37.166 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:37.561 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:37.562 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:37.714 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:37.714 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:37.726 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:37.726 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:37.726 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 95 -lt 100 ']' 2023-11-27T15:27:37.727 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 95 + 1 2023-11-27T15:27:37.767 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=96 2023-11-27T15:27:37.767 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:37.796 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:37.796 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:37.799 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:37.799 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:37.800 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:37 smithi060 ceph-mon[131518]: pgmap v411: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-11-27T15:27:37.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:37 smithi008 ceph-mon[128788]: pgmap v411: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-11-27T15:27:37.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:37 smithi029 ceph-mon[131166]: pgmap v411: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2023-11-27T15:27:38.261 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:38.261 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:38.489 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:38.489 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:38.501 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:38.501 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:38.502 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 96 -lt 100 ']' 2023-11-27T15:27:38.502 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 96 + 1 2023-11-27T15:27:38.534 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=97 2023-11-27T15:27:38.534 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:38.580 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:38.580 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:38.583 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:38.583 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:38.584 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:38.978 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:38.978 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:39.122 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:39.122 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:39.143 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:39.143 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:39.143 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 97 -lt 100 ']' 2023-11-27T15:27:39.144 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 97 + 1 2023-11-27T15:27:39.175 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=98 2023-11-27T15:27:39.175 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:39.215 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:39.215 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:39.217 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:39.218 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:39.218 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:39.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:39 smithi060 ceph-mon[131518]: pgmap v412: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-11-27T15:27:39.602 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:39.603 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:39.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:39 smithi008 ceph-mon[128788]: pgmap v412: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-11-27T15:27:39.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:39 smithi029 ceph-mon[131166]: pgmap v412: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-11-27T15:27:39.738 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:39.739 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:39.751 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:39.751 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:39.751 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 98 -lt 100 ']' 2023-11-27T15:27:39.751 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 98 + 1 2023-11-27T15:27:39.792 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=99 2023-11-27T15:27:39.792 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:39.854 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:39.854 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:39.857 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:39.857 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:39.858 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:40.236 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:40.236 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:40.405 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:40.431 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:40.467 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:40.467 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:40.467 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 99 -lt 100 ']' 2023-11-27T15:27:40.468 INFO:tasks.workunit.client.0.smithi008.stderr:++ expr 99 + 1 2023-11-27T15:27:40.500 INFO:tasks.workunit.client.0.smithi008.stderr:+ c=100 2023-11-27T15:27:40.501 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:40.555 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:40.555 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:40.557 INFO:tasks.workunit.client.0.smithi008.stderr:+ c1=2 2023-11-27T15:27:40.558 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo 3 2023-11-27T15:27:40.558 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:27:41.028 INFO:tasks.workunit.client.0.smithi008.stderr:++ getfacl d1 2023-11-27T15:27:41.028 INFO:tasks.workunit.client.0.smithi008.stderr:++ grep -c nobody:rw 2023-11-27T15:27:41.172 INFO:tasks.workunit.client.0.smithi008.stderr:+ c2=2 2023-11-27T15:27:41.172 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:41.194 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:41.194 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:27:41.194 INFO:tasks.workunit.client.0.smithi008.stderr:+ '[' 100 -lt 100 ']' 2023-11-27T15:27:41.204 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir d1 2023-11-27T15:27:41.253 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2023-11-27T15:27:41.266 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2023-11-27T15:27:41.270 INFO:tasks.workunit.client.0.smithi008.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2023-11-27T15:27:41.270 INFO:tasks.workunit.client.0.smithi008.stderr:+ getfattr -n system.posix_acl_access d1 2023-11-27T15:27:41.290 INFO:tasks.workunit.client.0.smithi008.stderr:d1: system.posix_acl_access: No such attribute 2023-11-27T15:27:41.290 INFO:tasks.workunit.client.0.smithi008.stderr:+ return 0 2023-11-27T15:27:41.291 INFO:tasks.workunit.client.0.smithi008.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2023-11-27T15:27:41.291 INFO:tasks.workunit.client.0.smithi008.stderr:+ getfattr -n system.posix_acl_default . 2023-11-27T15:27:41.292 INFO:tasks.workunit.client.0.smithi008.stderr:.: system.posix_acl_default: No such attribute 2023-11-27T15:27:41.292 INFO:tasks.workunit.client.0.smithi008.stderr:+ return 0 2023-11-27T15:27:41.293 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir d1 2023-11-27T15:27:41.296 INFO:tasks.workunit.client.0.smithi008.stderr:+ cd .. 2023-11-27T15:27:41.296 INFO:tasks.workunit.client.0.smithi008.stderr:+ rmdir testdir 2023-11-27T15:27:41.310 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo OK 2023-11-27T15:27:41.311 INFO:tasks.workunit.client.0.smithi008.stdout:OK 2023-11-27T15:27:41.312 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:27:41.312 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:27:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:41 smithi060 ceph-mon[131518]: pgmap v413: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-11-27T15:27:41.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:41 smithi008 ceph-mon[128788]: pgmap v413: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-11-27T15:27:41.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:41 smithi029 ceph-mon[131166]: pgmap v413: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2023-11-27T15:27:41.970 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2023-11-27T15:27:41.970 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2023-11-27T15:27:42.148 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2023-11-27T15:27:42.148 INFO:tasks.workunit.client.0.smithi008.stderr:+ file=test_chmod.181398 2023-11-27T15:27:42.149 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo foo 2023-11-27T15:27:42.155 INFO:tasks.workunit.client.0.smithi008.stderr:+ test 0 '!=' 0 2023-11-27T15:27:42.155 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 400 test_chmod.181398 2023-11-27T15:27:42.173 INFO:tasks.workunit.client.0.smithi008.stderr:+ test 0 '!=' 0 2023-11-27T15:27:42.173 INFO:tasks.workunit.client.0.smithi008.stderr:+ check_perms test_chmod.181398 -r-------- 2023-11-27T15:27:42.173 INFO:tasks.workunit.client.0.smithi008.stderr:+ file=test_chmod.181398 2023-11-27T15:27:42.174 INFO:tasks.workunit.client.0.smithi008.stderr:++ ls -la test_chmod.181398 2023-11-27T15:27:42.184 INFO:tasks.workunit.client.0.smithi008.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Nov 27 15:27 test_chmod.181398' 2023-11-27T15:27:42.185 INFO:tasks.workunit.client.0.smithi008.stderr:+ test 0 '!=' 0 2023-11-27T15:27:42.185 INFO:tasks.workunit.client.0.smithi008.stderr:+ perms=-r-------- 2023-11-27T15:27:42.186 INFO:tasks.workunit.client.0.smithi008.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 27 15:27 test_chmod.181398 2023-11-27T15:27:42.186 INFO:tasks.workunit.client.0.smithi008.stderr:++ awk '{print $1}' 2023-11-27T15:27:42.307 INFO:tasks.workunit.client.0.smithi008.stderr:+ test -r-------- '!=' -r--------. 2023-11-27T15:27:42.308 INFO:tasks.workunit.client.0.smithi008.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 27 15:27 test_chmod.181398 2023-11-27T15:27:42.309 INFO:tasks.workunit.client.0.smithi008.stderr:++ awk '{print $1}' 2023-11-27T15:27:42.313 INFO:tasks.workunit.client.0.smithi008.stderr:+ test -r--------. '!=' -r--------. 2023-11-27T15:27:42.313 INFO:tasks.workunit.client.0.smithi008.stderr:+ set +e 2023-11-27T15:27:42.313 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo bar 2023-11-27T15:27:42.313 INFO:tasks.workunit.client.0.smithi008.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.181398: Permission denied 2023-11-27T15:27:42.313 INFO:tasks.workunit.client.0.smithi008.stderr:+ test 1 = 0 2023-11-27T15:27:42.313 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2023-11-27T15:27:42.313 INFO:tasks.workunit.client.0.smithi008.stderr:+ chmod 600 test_chmod.181398 2023-11-27T15:27:42.315 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo bar 2023-11-27T15:27:42.315 INFO:tasks.workunit.client.0.smithi008.stderr:+ test 0 '!=' 0 2023-11-27T15:27:42.315 INFO:tasks.workunit.client.0.smithi008.stderr:+ check_perms test_chmod.181398 -rw------- 2023-11-27T15:27:42.316 INFO:tasks.workunit.client.0.smithi008.stderr:+ file=test_chmod.181398 2023-11-27T15:27:42.316 INFO:tasks.workunit.client.0.smithi008.stderr:++ ls -la test_chmod.181398 2023-11-27T15:27:42.319 INFO:tasks.workunit.client.0.smithi008.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Nov 27 15:27 test_chmod.181398' 2023-11-27T15:27:42.319 INFO:tasks.workunit.client.0.smithi008.stderr:+ test 0 '!=' 0 2023-11-27T15:27:42.319 INFO:tasks.workunit.client.0.smithi008.stderr:+ perms=-rw------- 2023-11-27T15:27:42.320 INFO:tasks.workunit.client.0.smithi008.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 27 15:27 test_chmod.181398 2023-11-27T15:27:42.320 INFO:tasks.workunit.client.0.smithi008.stderr:++ awk '{print $1}' 2023-11-27T15:27:42.324 INFO:tasks.workunit.client.0.smithi008.stderr:+ test -rw------- '!=' -rw-------. 2023-11-27T15:27:42.325 INFO:tasks.workunit.client.0.smithi008.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 27 15:27 test_chmod.181398 2023-11-27T15:27:42.325 INFO:tasks.workunit.client.0.smithi008.stderr:++ awk '{print $1}' 2023-11-27T15:27:42.329 INFO:tasks.workunit.client.0.smithi008.stderr:+ test -rw-------. '!=' -rw-------. 2023-11-27T15:27:42.330 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo foo 2023-11-27T15:27:42.330 INFO:tasks.workunit.client.0.smithi008.stderr:+ test 0 '!=' 0 2023-11-27T15:27:42.330 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:27:42.330 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:27:42.421 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2023-11-27T15:27:42.421 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2023-11-27T15:27:42.478 INFO:tasks.workunit.client.0.smithi008.stderr:+ set -e 2023-11-27T15:27:42.478 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir -p testdir 2023-11-27T15:27:42.481 INFO:tasks.workunit.client.0.smithi008.stderr:+ file=test_chmod.181458 2023-11-27T15:27:42.482 INFO:tasks.workunit.client.0.smithi008.stderr:+ echo foo 2023-11-27T15:27:42.482 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo chmod 600 testdir 2023-11-27T15:27:42.518 INFO:tasks.workunit.client.0.smithi008.stderr:+ expect_failure cat testdir/test_chmod.181458 2023-11-27T15:27:42.518 INFO:tasks.workunit.client.0.smithi008.stderr:+ cat testdir/test_chmod.181458 2023-11-27T15:27:42.529 INFO:tasks.workunit.client.0.smithi008.stderr:cat: testdir/test_chmod.181458: Permission denied 2023-11-27T15:27:42.529 INFO:tasks.workunit.client.0.smithi008.stderr:+ return 0 2023-11-27T15:27:42.529 INFO:tasks.workunit.client.0.smithi008.stderr:+ sudo cat testdir/test_chmod.181458 2023-11-27T15:27:42.555 INFO:tasks.workunit.client.0.smithi008.stdout:foo 2023-11-27T15:27:42.559 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:27:42.559 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:27:42.649 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2023-11-27T15:27:42.649 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2023-11-27T15:27:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.680+0000 7fa2209ff700 1 -- 172.21.15.8:0/73869122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa21c074a90 msgr2=0x7fa21c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.680+0000 7fa2209ff700 1 --2- 172.21.15.8:0/73869122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa21c074a90 0x7fa21c072f80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009990 tx=0x7fa20c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.680+0000 7fa2209ff700 1 -- 172.21.15.8:0/73869122 shutdown_connections 2023-11-27T15:27:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.680+0000 7fa2209ff700 1 --2- 172.21.15.8:0/73869122 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa21c103ca0 0x7fa21c10e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.680+0000 7fa2209ff700 1 --2- 172.21.15.8:0/73869122 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa21c074a90 0x7fa21c072f80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.680+0000 7fa2209ff700 1 --2- 172.21.15.8:0/73869122 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c0734c0 0x7fa21c073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.680+0000 7fa2209ff700 1 -- 172.21.15.8:0/73869122 >> 172.21.15.8:0/73869122 conn(0x7fa21c0fb810 msgr2=0x7fa21c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.680+0000 7fa2209ff700 1 -- 172.21.15.8:0/73869122 shutdown_connections 2023-11-27T15:27:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.680+0000 7fa2209ff700 1 -- 172.21.15.8:0/73869122 wait complete. 2023-11-27T15:27:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.681+0000 7fa2209ff700 1 Processor -- start 2023-11-27T15:27:42.685 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.682+0000 7fa2209ff700 1 -- start start 2023-11-27T15:27:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.682+0000 7fa2209ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa21c0734c0 0x7fa21c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.682+0000 7fa2209ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa21c074a90 0x7fa21c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.682+0000 7fa2209ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c103ca0 0x7fa21c1a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.682+0000 7fa2209ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c1a4130 con 0x7fa21c074a90 2023-11-27T15:27:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.682+0000 7fa2209ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c1a4270 con 0x7fa21c0734c0 2023-11-27T15:27:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.682+0000 7fa2209ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c1a4400 con 0x7fa21c103ca0 2023-11-27T15:27:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa21c074a90 0x7fa21c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa21c0734c0 0x7fa21c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa21c0734c0 0x7fa21c19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:38132/0 (socket says 172.21.15.8:38132) 2023-11-27T15:27:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa21c074a90 0x7fa21c19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38986/0 (socket says 172.21.15.8:38986) 2023-11-27T15:27:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa21a59c700 1 -- 172.21.15.8:0/2131308848 learned_addr learned my addr 172.21.15.8:0/2131308848 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:42.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa21ad9d700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c103ca0 0x7fa21c1a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa219d9b700 1 -- 172.21.15.8:0/2131308848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c103ca0 msgr2=0x7fa21c1a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa219d9b700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c103ca0 0x7fa21c1a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa219d9b700 1 -- 172.21.15.8:0/2131308848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa21c0734c0 msgr2=0x7fa21c19c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa219d9b700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa21c0734c0 0x7fa21c19c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.683+0000 7fa219d9b700 1 -- 172.21.15.8:0/2131308848 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c0095f0 con 0x7fa21c074a90 2023-11-27T15:27:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.684+0000 7fa219d9b700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa21c074a90 0x7fa21c19caf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e9f0 tx=0x7fa21000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.684+0000 7fa20b7fe700 1 -- 172.21.15.8:0/2131308848 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21000c9c0 con 0x7fa21c074a90 2023-11-27T15:27:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.684+0000 7fa2209ff700 1 -- 172.21.15.8:0/2131308848 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c1121a0 con 0x7fa21c074a90 2023-11-27T15:27:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.684+0000 7fa20b7fe700 1 -- 172.21.15.8:0/2131308848 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa210010430 con 0x7fa21c074a90 2023-11-27T15:27:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.684+0000 7fa20b7fe700 1 -- 172.21.15.8:0/2131308848 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa210018680 con 0x7fa21c074a90 2023-11-27T15:27:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.684+0000 7fa2209ff700 1 -- 172.21.15.8:0/2131308848 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c112010 con 0x7fa21c074a90 2023-11-27T15:27:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.687+0000 7fa20b7fe700 1 -- 172.21.15.8:0/2131308848 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa210010860 con 0x7fa21c074a90 2023-11-27T15:27:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.688+0000 7fa20b7fe700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2040a1150 0x7fa2040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.688+0000 7fa20b7fe700 1 -- 172.21.15.8:0/2131308848 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa210014070 con 0x7fa21c074a90 2023-11-27T15:27:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.689+0000 7fa21a59c700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2040a1150 0x7fa2040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.689+0000 7fa21a59c700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2040a1150 0x7fa2040a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00b2e0 tx=0x7fa20c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.824+0000 7fa234355b80 1 Processor -- start 2023-11-27T15:27:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.824+0000 7fa234355b80 1 -- start start 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.824+0000 7fa234355b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba079a6b0 0x55eba093e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.824+0000 7fa234355b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093edb0 0x55eba093f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.824+0000 7fa234355b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 0x55eba093ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.824+0000 7fa234355b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba0844000 con 0x55eba079a6b0 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.824+0000 7fa234355b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba0896840 con 0x55eba093edb0 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.824+0000 7fa234355b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba088fcd0 con 0x55eba093fb10 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba079a6b0 0x55eba093e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093edb0 0x55eba093f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093edb0 0x55eba093f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:38142/0 (socket says 172.21.15.8:38142) 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba079a6b0 0x55eba093e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39000/0 (socket says 172.21.15.8:39000) 2023-11-27T15:27:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa219d9b700 1 -- 172.21.15.8:0/643193294 learned_addr learned my addr 172.21.15.8:0/643193294 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa21ad9d700 1 --2- 172.21.15.8:0/643193294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 0x55eba093ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa219d9b700 1 -- 172.21.15.8:0/643193294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 msgr2=0x55eba093ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa219d9b700 1 --2- 172.21.15.8:0/643193294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 0x55eba093ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa219d9b700 1 -- 172.21.15.8:0/643193294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba079a6b0 msgr2=0x55eba093e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa219d9b700 1 --2- 172.21.15.8:0/643193294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba079a6b0 0x55eba093e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa219d9b700 1 -- 172.21.15.8:0/643193294 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eba07115d0 con 0x55eba093edb0 2023-11-27T15:27:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa219d9b700 1 --2- 172.21.15.8:0/643193294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093edb0 0x55eba093f5d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa210099dd0 tx=0x7fa2100997a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33d469c7ad8862a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.825+0000 7fa208ff9700 1 -- 172.21.15.8:0/643193294 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21009de70 con 0x55eba093edb0 2023-11-27T15:27:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.826+0000 7fa208ff9700 1 -- 172.21.15.8:0/643193294 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2100a2290 con 0x55eba093edb0 2023-11-27T15:27:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.826+0000 7fa208ff9700 1 -- 172.21.15.8:0/643193294 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2100a1e00 con 0x55eba093edb0 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.826+0000 7fa234355b80 1 -- 172.21.15.8:0/643193294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093edb0 msgr2=0x55eba093f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.826+0000 7fa234355b80 1 --2- 172.21.15.8:0/643193294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093edb0 0x55eba093f5d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa210099dd0 tx=0x7fa2100997a0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.826+0000 7fa234355b80 1 -- 172.21.15.8:0/643193294 shutdown_connections 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.826+0000 7fa234355b80 1 --2- 172.21.15.8:0/643193294 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 0x55eba093ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.826+0000 7fa234355b80 1 --2- 172.21.15.8:0/643193294 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba079a6b0 0x55eba093e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.826+0000 7fa234355b80 1 --2- 172.21.15.8:0/643193294 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093edb0 0x55eba093f5d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.826+0000 7fa234355b80 1 -- 172.21.15.8:0/643193294 >> 172.21.15.8:0/643193294 conn(0x55eba0933f50 msgr2=0x55eba0939100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.826+0000 7fa234355b80 1 -- 172.21.15.8:0/643193294 shutdown_connections 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.827+0000 7fa234355b80 1 -- 172.21.15.8:0/643193294 wait complete. 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.827+0000 7fa234355b80 1 -- start start 2023-11-27T15:27:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.827+0000 7fa234355b80 1 Processor -- start 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa234355b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 0x55eba0947cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa234355b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093edb0 0x55eba0948210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa234355b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093fb10 0x55eba094f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa234355b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba0831880 con 0x55eba093fb10 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa234355b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba0831520 con 0x55eba079a6b0 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa234355b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba07138c0 con 0x55eba093edb0 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa234355b80 10 client.0 _ll_register_callbacks cb 0x55eba084f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa21ad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093fb10 0x55eba094f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093edb0 0x55eba0948210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa21ad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093fb10 0x55eba094f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39012/0 (socket says 172.21.15.8:39012) 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa21ad9d700 1 -- 172.21.15.8:0/20050014 learned_addr learned my addr 172.21.15.8:0/20050014 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:42.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa21a59c700 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 0x55eba0947cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa21ad9d700 1 -- 172.21.15.8:0/20050014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093edb0 msgr2=0x55eba0948210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa21ad9d700 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093edb0 0x55eba0948210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa21ad9d700 1 -- 172.21.15.8:0/20050014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 msgr2=0x55eba0947cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa21ad9d700 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 0x55eba0947cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa21ad9d700 1 -- 172.21.15.8:0/20050014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c112010 con 0x55eba093fb10 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.828+0000 7fa21a59c700 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 0x55eba0947cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.829+0000 7fa21ad9d700 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093fb10 0x55eba094f250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa214011f60 tx=0x7fa214014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3739fa09a1b5df31 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.829+0000 7fa201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.829+0000 7fa201ffb700 1 -- 172.21.15.8:0/20050014 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21400e040 con 0x55eba093fb10 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.829+0000 7fa234355b80 1 -- 172.21.15.8:0/20050014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eba0931710 con 0x55eba093fb10 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.829+0000 7fa201ffb700 1 -- 172.21.15.8:0/20050014 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa2140090d0 con 0x55eba093fb10 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.829+0000 7fa201ffb700 1 -- 172.21.15.8:0/20050014 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa214009290 con 0x55eba093fb10 2023-11-27T15:27:42.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.829+0000 7fa234355b80 10 client.15480 fetch_fsmap learned FSMap version 24 2023-11-27T15:27:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.829+0000 7fa234355b80 1 -- 172.21.15.8:0/20050014 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55eba0856130 con 0x55eba093fb10 2023-11-27T15:27:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.830+0000 7fa201ffb700 1 -- 172.21.15.8:0/20050014 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eba0931710 con 0x55eba093fb10 2023-11-27T15:27:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.830+0000 7fa201ffb700 1 -- 172.21.15.8:0/20050014 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa21401c020 con 0x55eba093fb10 2023-11-27T15:27:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.830+0000 7fa234355b80 10 client.15480 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:27:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.830+0000 7fa234355b80 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:27:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.830+0000 7fa234355b80 20 client.15480 populate_metadata read hostname 'smithi008' 2023-11-27T15:27:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.830+0000 7fa234355b80 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55eba0953ae0 0x55eba0955ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.830+0000 7fa234355b80 4 client.15480 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.830+0000 7fa234355b80 1 -- 172.21.15.8:0/20050014 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eba0852040 con 0x55eba0953ae0 2023-11-27T15:27:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.830+0000 7fa21a59c700 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55eba0953ae0 0x55eba0955ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.838 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.831+0000 7fa201ffb700 10 client.15480 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.831+0000 7fa21a59c700 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55eba0953ae0 0x55eba0955ef0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a2c7501b0fb8a server_cookie=6e0517dc68b07456 in_seq=0 out_seq=0 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.831+0000 7fa201ffb700 1 -- 172.21.15.8:0/20050014 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eba0852040 con 0x55eba0953ae0 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.831+0000 7fa201ffb700 10 client.15480 handle_command_reply: tid=0 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.831+0000 7fa234355b80 1 client.15480 shutdown 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.831+0000 7fa234355b80 1 -- 172.21.15.8:0/20050014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093fb10 msgr2=0x55eba094f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.831+0000 7fa234355b80 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093fb10 0x55eba094f250 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa214011f60 tx=0x7fa214014600 comp rx=0 tx=0).stop 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.832+0000 7fa234355b80 1 -- 172.21.15.8:0/20050014 shutdown_connections 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.832+0000 7fa234355b80 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093fb10 0x55eba094f250 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.832+0000 7fa234355b80 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 0x55eba0947cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.832+0000 7fa234355b80 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55eba0953ae0 0x55eba0955ef0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.832+0000 7fa234355b80 1 --2- 172.21.15.8:0/20050014 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093edb0 0x55eba0948210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.832+0000 7fa234355b80 1 -- 172.21.15.8:0/20050014 >> 172.21.15.8:0/20050014 conn(0x55eba0933f50 msgr2=0x55eba09387e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:42.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.832+0000 7fa201ffb700 0 client.15480 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.832+0000 7fa234355b80 1 -- 172.21.15.8:0/20050014 shutdown_connections 2023-11-27T15:27:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.832+0000 7fa234355b80 1 -- 172.21.15.8:0/20050014 wait complete. 2023-11-27T15:27:42.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.832+0000 7fa234355b80 20 client.15480 trim_cache size 0 max 16384 2023-11-27T15:27:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.847+0000 7fa234355b80 1 Processor -- start 2023-11-27T15:27:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa234355b80 1 -- start start 2023-11-27T15:27:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa234355b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba079a6b0 0x55eba0942a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa234355b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba0942fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa234355b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093fb10 0x55eba094f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa234355b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba093f420 con 0x55eba093edb0 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa234355b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba094f6b0 con 0x55eba093fb10 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa234355b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba093abf0 con 0x55eba079a6b0 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba0942fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa21a59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba079a6b0 0x55eba0942a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba0942fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39024/0 (socket says 172.21.15.8:39024) 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa219d9b700 1 -- 172.21.15.8:0/2224982217 learned_addr learned my addr 172.21.15.8:0/2224982217 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa21ad9d700 1 --2- 172.21.15.8:0/2224982217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093fb10 0x55eba094f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.848+0000 7fa21a59c700 1 -- 172.21.15.8:0/2224982217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093fb10 msgr2=0x55eba094f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.849+0000 7fa21a59c700 1 --2- 172.21.15.8:0/2224982217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093fb10 0x55eba094f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.849+0000 7fa21a59c700 1 -- 172.21.15.8:0/2224982217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 msgr2=0x55eba0942fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.849+0000 7fa21a59c700 1 --2- 172.21.15.8:0/2224982217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba0942fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.853 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.849+0000 7fa21a59c700 1 -- 172.21.15.8:0/2224982217 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eba0855e80 con 0x55eba079a6b0 2023-11-27T15:27:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.849+0000 7fa219d9b700 1 --2- 172.21.15.8:0/2224982217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba0942fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.849+0000 7fa21ad9d700 1 --2- 172.21.15.8:0/2224982217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093fb10 0x55eba094f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.849+0000 7fa21a59c700 1 --2- 172.21.15.8:0/2224982217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba079a6b0 0x55eba0942a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55eba09567a0 tx=0x7fa20c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eca12c5ce052078 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.849+0000 7fa2097fa700 1 -- 172.21.15.8:0/2224982217 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c02d040 con 0x55eba079a6b0 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.849+0000 7fa2097fa700 1 -- 172.21.15.8:0/2224982217 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa20c01e040 con 0x55eba079a6b0 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.849+0000 7fa2097fa700 1 -- 172.21.15.8:0/2224982217 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20c004e10 con 0x55eba079a6b0 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.850+0000 7fa234355b80 1 -- 172.21.15.8:0/2224982217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba079a6b0 msgr2=0x55eba0942a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.850+0000 7fa234355b80 1 --2- 172.21.15.8:0/2224982217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba079a6b0 0x55eba0942a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55eba09567a0 tx=0x7fa20c020000 comp rx=0 tx=0).stop 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.850+0000 7fa234355b80 1 -- 172.21.15.8:0/2224982217 shutdown_connections 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.850+0000 7fa234355b80 1 --2- 172.21.15.8:0/2224982217 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba079a6b0 0x55eba0942a60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.850+0000 7fa234355b80 1 --2- 172.21.15.8:0/2224982217 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba093fb10 0x55eba094f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.850+0000 7fa234355b80 1 --2- 172.21.15.8:0/2224982217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba0942fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.850+0000 7fa234355b80 1 -- 172.21.15.8:0/2224982217 >> 172.21.15.8:0/2224982217 conn(0x55eba0933f50 msgr2=0x55eba09469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.850+0000 7fa234355b80 1 -- 172.21.15.8:0/2224982217 shutdown_connections 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.850+0000 7fa234355b80 1 -- 172.21.15.8:0/2224982217 wait complete. 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.850+0000 7fa234355b80 1 -- start start 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa234355b80 1 Processor -- start 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa234355b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 0x55eba094e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa234355b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba094eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa234355b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 0x55eba094b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa234355b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba094b9a0 con 0x55eba093edb0 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa234355b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba094bb10 con 0x55eba079a6b0 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa234355b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba094bcd0 con 0x55eba093fb10 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa234355b80 10 client.0 _ll_register_callbacks cb 0x7fa2140118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba094eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa219d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba094eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39032/0 (socket says 172.21.15.8:39032) 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.851+0000 7fa219d9b700 1 -- 172.21.15.8:0/3003150902 learned_addr learned my addr 172.21.15.8:0/3003150902 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa21ad9d700 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 0x55eba094b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa21a59c700 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 0x55eba094e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa219d9b700 1 -- 172.21.15.8:0/3003150902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 msgr2=0x55eba094b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa219d9b700 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 0x55eba094b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa219d9b700 1 -- 172.21.15.8:0/3003150902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 msgr2=0x55eba094e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa219d9b700 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 0x55eba094e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa219d9b700 1 -- 172.21.15.8:0/3003150902 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eba07115d0 con 0x55eba093edb0 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa21ad9d700 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 0x55eba094b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa219d9b700 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba094eeb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa210099b40 tx=0x7fa210099a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75bf24c2d944af3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa201ffb700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa201ffb700 1 -- 172.21.15.8:0/3003150902 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2100a1e00 con 0x55eba093edb0 2023-11-27T15:27:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa201ffb700 1 -- 172.21.15.8:0/3003150902 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa2100a1f60 con 0x55eba093edb0 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.852+0000 7fa234355b80 1 -- 172.21.15.8:0/3003150902 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2100a2290 con 0x55eba093edb0 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa201ffb700 1 -- 172.21.15.8:0/3003150902 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2100adb40 con 0x55eba093edb0 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa234355b80 10 client.15486 fetch_fsmap learned FSMap version 24 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa234355b80 1 -- 172.21.15.8:0/3003150902 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fa21c10f590 con 0x55eba093edb0 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa201ffb700 1 -- 172.21.15.8:0/3003150902 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa210063080 con 0x55eba093edb0 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa201ffb700 1 -- 172.21.15.8:0/3003150902 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa2100b8020 con 0x55eba093edb0 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa234355b80 10 client.15486 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa234355b80 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa234355b80 20 client.15486 populate_metadata read hostname 'smithi008' 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa234355b80 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55eba09a8b60 0x55eba09aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa234355b80 4 client.15486 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa234355b80 1 -- 172.21.15.8:0/3003150902 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eba085e600 con 0x55eba09a8b60 2023-11-27T15:27:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.853+0000 7fa21a59c700 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55eba09a8b60 0x55eba09aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.854+0000 7fa201ffb700 10 client.15486 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.854+0000 7fa21a59c700 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55eba09a8b60 0x55eba09aaf70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769ef7fa22da5e2f server_cookie=f1e5c1c0fe938f0d in_seq=0 out_seq=0 2023-11-27T15:27:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa201ffb700 1 -- 172.21.15.8:0/3003150902 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eba085e600 con 0x55eba09a8b60 2023-11-27T15:27:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa201ffb700 10 client.15486 handle_command_reply: tid=0 2023-11-27T15:27:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa234355b80 1 client.15486 shutdown 2023-11-27T15:27:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa234355b80 1 -- 172.21.15.8:0/3003150902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 msgr2=0x55eba094eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:42.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa234355b80 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba094eeb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa210099b40 tx=0x7fa210099a30 comp rx=0 tx=0).stop 2023-11-27T15:27:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa234355b80 1 -- 172.21.15.8:0/3003150902 shutdown_connections 2023-11-27T15:27:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa234355b80 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55eba09a8b60 0x55eba09aaf70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa234355b80 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55eba093fb10 0x55eba094b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa234355b80 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55eba079a6b0 0x55eba094e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa234355b80 1 --2- 172.21.15.8:0/3003150902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55eba093edb0 0x55eba094eeb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.874+0000 7fa234355b80 1 -- 172.21.15.8:0/3003150902 >> 172.21.15.8:0/3003150902 conn(0x55eba0933f50 msgr2=0x55eba0942a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.875+0000 7fa201ffb700 0 client.15486 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.875+0000 7fa234355b80 1 -- 172.21.15.8:0/3003150902 shutdown_connections 2023-11-27T15:27:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.875+0000 7fa234355b80 1 -- 172.21.15.8:0/3003150902 wait complete. 2023-11-27T15:27:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.875+0000 7fa234355b80 20 client.15486 trim_cache size 0 max 16384 2023-11-27T15:27:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:27:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:27:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:27:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:27:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.875+0000 7fa2209ff700 1 -- 172.21.15.8:0/2131308848 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2040a1150 msgr2=0x7fa2040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.875+0000 7fa2209ff700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2040a1150 0x7fa2040a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00b2e0 tx=0x7fa20c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.875+0000 7fa2209ff700 1 -- 172.21.15.8:0/2131308848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa21c074a90 msgr2=0x7fa21c19caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.875+0000 7fa2209ff700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa21c074a90 0x7fa21c19caf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa21000e9f0 tx=0x7fa21000ed00 comp rx=0 tx=0).stop 2023-11-27T15:27:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.876+0000 7fa2209ff700 1 -- 172.21.15.8:0/2131308848 shutdown_connections 2023-11-27T15:27:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.876+0000 7fa2209ff700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa2040a1150 0x7fa2040a3600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.876+0000 7fa2209ff700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa21c103ca0 0x7fa21c1a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.876+0000 7fa2209ff700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa21c0734c0 0x7fa21c19c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.876+0000 7fa2209ff700 1 --2- 172.21.15.8:0/2131308848 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa21c074a90 0x7fa21c19caf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.876+0000 7fa2209ff700 1 -- 172.21.15.8:0/2131308848 >> 172.21.15.8:0/2131308848 conn(0x7fa21c0fb810 msgr2=0x7fa21c0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.876+0000 7fa2209ff700 1 -- 172.21.15.8:0/2131308848 shutdown_connections 2023-11-27T15:27:42.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:42.876+0000 7fa2209ff700 1 -- 172.21.15.8:0/2131308848 wait complete. 2023-11-27T15:27:43.060 INFO:tasks.workunit.client.0.smithi008.stdout:ok 2023-11-27T15:27:43.066 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:27:43.066 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:27:43.188 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2023-11-27T15:27:43.189 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2023-11-27T15:27:43.276 INFO:tasks.workunit.client.0.smithi008.stdout:creating folder hierarchy 2023-11-27T15:27:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:43 smithi060 ceph-mon[131518]: pgmap v414: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-11-27T15:27:43.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:43 smithi008 ceph-mon[128788]: pgmap v414: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-11-27T15:27:43.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:43 smithi029 ceph-mon[131166]: pgmap v414: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 11 op/s 2023-11-27T15:27:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:45 smithi060 ceph-mon[131518]: pgmap v415: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-11-27T15:27:45.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:45 smithi008 ceph-mon[128788]: pgmap v415: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-11-27T15:27:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:45 smithi029 ceph-mon[131166]: pgmap v415: 129 pgs: 129 active+clean; 1.7 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 9 op/s 2023-11-27T15:27:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:47 smithi060 ceph-mon[131518]: pgmap v416: 129 pgs: 129 active+clean; 1.8 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 14 op/s 2023-11-27T15:27:47.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:47 smithi008 ceph-mon[128788]: pgmap v416: 129 pgs: 129 active+clean; 1.8 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 14 op/s 2023-11-27T15:27:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:47 smithi029 ceph-mon[131166]: pgmap v416: 129 pgs: 129 active+clean; 1.8 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 14 op/s 2023-11-27T15:27:49.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:49 smithi060 ceph-mon[131518]: pgmap v417: 129 pgs: 129 active+clean; 1.8 MiB data, 93 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-11-27T15:27:49.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:49 smithi008 ceph-mon[128788]: pgmap v417: 129 pgs: 129 active+clean; 1.8 MiB data, 93 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-11-27T15:27:49.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:49 smithi029 ceph-mon[131166]: pgmap v417: 129 pgs: 129 active+clean; 1.8 MiB data, 93 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-11-27T15:27:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:51 smithi060 ceph-mon[131518]: pgmap v418: 129 pgs: 129 active+clean; 4.6 MiB data, 96 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 271 KiB/s wr, 19 op/s 2023-11-27T15:27:51.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:51 smithi008 ceph-mon[128788]: pgmap v418: 129 pgs: 129 active+clean; 4.6 MiB data, 96 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 271 KiB/s wr, 19 op/s 2023-11-27T15:27:51.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:51 smithi029 ceph-mon[131166]: pgmap v418: 129 pgs: 129 active+clean; 4.6 MiB data, 96 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 271 KiB/s wr, 19 op/s 2023-11-27T15:27:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:53 smithi060 ceph-mon[131518]: pgmap v419: 129 pgs: 129 active+clean; 8.6 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 637 KiB/s wr, 25 op/s 2023-11-27T15:27:53.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:53 smithi008 ceph-mon[128788]: pgmap v419: 129 pgs: 129 active+clean; 8.6 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 637 KiB/s wr, 25 op/s 2023-11-27T15:27:53.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:53 smithi029 ceph-mon[131166]: pgmap v419: 129 pgs: 129 active+clean; 8.6 MiB data, 115 MiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 637 KiB/s wr, 25 op/s 2023-11-27T15:27:54.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:54 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:27:54.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:54 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:27:54.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:54 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:27:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:55 smithi060 ceph-mon[131518]: pgmap v420: 129 pgs: 129 active+clean; 13 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 982 KiB/s wr, 27 op/s 2023-11-27T15:27:55.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:55 smithi008 ceph-mon[128788]: pgmap v420: 129 pgs: 129 active+clean; 13 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 982 KiB/s wr, 27 op/s 2023-11-27T15:27:55.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:55 smithi029 ceph-mon[131166]: pgmap v420: 129 pgs: 129 active+clean; 13 MiB data, 122 MiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 982 KiB/s wr, 27 op/s 2023-11-27T15:27:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.086+0000 7f3716bff700 1 -- 172.21.15.8:0/1140963166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3710101440 msgr2=0x7f3710101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.086+0000 7f3716bff700 1 --2- 172.21.15.8:0/1140963166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3710101440 0x7f3710101890 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009990 tx=0x7f36fc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.086+0000 7f3716bff700 1 -- 172.21.15.8:0/1140963166 shutdown_connections 2023-11-27T15:27:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.086+0000 7f3716bff700 1 --2- 172.21.15.8:0/1140963166 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3710100250 0x7f3710100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.086+0000 7f3716bff700 1 --2- 172.21.15.8:0/1140963166 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3710101dd0 0x7f3710108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.086+0000 7f3716bff700 1 --2- 172.21.15.8:0/1140963166 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3710101440 0x7f3710101890 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.086+0000 7f3716bff700 1 -- 172.21.15.8:0/1140963166 >> 172.21.15.8:0/1140963166 conn(0x7f37100fb850 msgr2=0x7f37100fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.086+0000 7f3716bff700 1 -- 172.21.15.8:0/1140963166 shutdown_connections 2023-11-27T15:27:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.086+0000 7f3716bff700 1 -- 172.21.15.8:0/1140963166 wait complete. 2023-11-27T15:27:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.087+0000 7f3716bff700 1 Processor -- start 2023-11-27T15:27:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.087+0000 7f3716bff700 1 -- start start 2023-11-27T15:27:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.088+0000 7f3716bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3710100250 0x7f3710198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.088+0000 7f3716bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3710101440 0x7f3710198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.088+0000 7f3716bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3710101dd0 0x7f371019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.088+0000 7f3716bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371019fdc0 con 0x7f3710101440 2023-11-27T15:27:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.088+0000 7f3716bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371019ff00 con 0x7f3710101dd0 2023-11-27T15:27:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.088+0000 7f3716bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37101a0090 con 0x7f3710100250 2023-11-27T15:27:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.088+0000 7f371499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3710100250 0x7f3710198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.088+0000 7f370ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3710101440 0x7f3710198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.088+0000 7f371499b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3710100250 0x7f3710198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:52216/0 (socket says 172.21.15.8:52216) 2023-11-27T15:27:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.088+0000 7f371499b700 1 -- 172.21.15.8:0/2891450665 learned_addr learned my addr 172.21.15.8:0/2891450665 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.089+0000 7f371519c700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3710101dd0 0x7f371019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.089+0000 7f371499b700 1 -- 172.21.15.8:0/2891450665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3710101dd0 msgr2=0x7f371019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.089+0000 7f371499b700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3710101dd0 0x7f371019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.089+0000 7f371499b700 1 -- 172.21.15.8:0/2891450665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3710101440 msgr2=0x7f3710198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.089+0000 7f371499b700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3710101440 0x7f3710198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.089+0000 7f371499b700 1 -- 172.21.15.8:0/2891450665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3708009c40 con 0x7f3710100250 2023-11-27T15:27:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.089+0000 7f371499b700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3710100250 0x7f3710198240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f370400e970 tx=0x7f370400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.089+0000 7f370dffb700 1 -- 172.21.15.8:0/2891450665 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704009790 con 0x7f3710100250 2023-11-27T15:27:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.090+0000 7f370dffb700 1 -- 172.21.15.8:0/2891450665 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f370400c7c0 con 0x7f3710100250 2023-11-27T15:27:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.090+0000 7f370dffb700 1 -- 172.21.15.8:0/2891450665 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704003c20 con 0x7f3710100250 2023-11-27T15:27:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.090+0000 7f3716bff700 1 -- 172.21.15.8:0/2891450665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc0095f0 con 0x7f3710100250 2023-11-27T15:27:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.090+0000 7f3716bff700 1 -- 172.21.15.8:0/2891450665 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371010bc10 con 0x7f3710100250 2023-11-27T15:27:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.095+0000 7f370dffb700 1 -- 172.21.15.8:0/2891450665 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f370400c930 con 0x7f3710100250 2023-11-27T15:27:56.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.096+0000 7f372a4e0b80 1 Processor -- start 2023-11-27T15:27:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.096+0000 7f372a4e0b80 1 -- start start 2023-11-27T15:27:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.096+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.096+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.096+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 0x55d4fd8e2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd7de000 con 0x55d4fd8e15c0 2023-11-27T15:27:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd830840 con 0x55d4fd8e2320 2023-11-27T15:27:56.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd829cd0 con 0x55d4fd7346b0 2023-11-27T15:27:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f371519c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f371519c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:52230/0 (socket says 172.21.15.8:52230) 2023-11-27T15:27:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f371519c700 1 -- 172.21.15.8:0/3521250493 learned_addr learned my addr 172.21.15.8:0/3521250493 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f371499b700 1 --2- 172.21.15.8:0/3521250493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f371519c700 1 -- 172.21.15.8:0/3521250493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 msgr2=0x55d4fd8e2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f370ffff700 1 --2- 172.21.15.8:0/3521250493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 0x55d4fd8e2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f371519c700 1 --2- 172.21.15.8:0/3521250493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 0x55d4fd8e2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f371519c700 1 -- 172.21.15.8:0/3521250493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd8e15c0 msgr2=0x55d4fd8e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f371519c700 1 --2- 172.21.15.8:0/3521250493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8e1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.097+0000 7f371519c700 1 -- 172.21.15.8:0/3521250493 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4fd6ab5d0 con 0x55d4fd7346b0 2023-11-27T15:27:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.098+0000 7f370ffff700 1 --2- 172.21.15.8:0/3521250493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 0x55d4fd8e2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.098+0000 7f371519c700 1 --2- 172.21.15.8:0/3521250493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e1080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f370800e730 tx=0x7f3708009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7af95c67268a87c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.098+0000 7f370dffb700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f37000a11d0 0x7f37000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.098+0000 7f36faffd700 1 -- 172.21.15.8:0/3521250493 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370800e040 con 0x55d4fd7346b0 2023-11-27T15:27:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.099+0000 7f36faffd700 1 -- 172.21.15.8:0/3521250493 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f370800f040 con 0x55d4fd7346b0 2023-11-27T15:27:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.099+0000 7f36faffd700 1 -- 172.21.15.8:0/3521250493 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708007760 con 0x55d4fd7346b0 2023-11-27T15:27:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.099+0000 7f370dffb700 1 -- 172.21.15.8:0/2891450665 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3704014070 con 0x7f3710100250 2023-11-27T15:27:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.099+0000 7f371499b700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f37000a11d0 0x7f37000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.100+0000 7f371499b700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f37000a11d0 0x7f37000a3680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f370400c5b0 tx=0x7f3704000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.100+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3521250493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd7346b0 msgr2=0x55d4fd8e1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.100+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/3521250493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e1080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f370800e730 tx=0x7f3708009920 comp rx=0 tx=0).stop 2023-11-27T15:27:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.100+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3521250493 shutdown_connections 2023-11-27T15:27:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.100+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/3521250493 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e1080 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.100+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/3521250493 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 0x55d4fd8e2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.100+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/3521250493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.100+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3521250493 >> 172.21.15.8:0/3521250493 conn(0x55d4fd8cdf50 msgr2=0x55d4fd8d3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.100+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3521250493 shutdown_connections 2023-11-27T15:27:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.100+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3521250493 wait complete. 2023-11-27T15:27:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.101+0000 7f372a4e0b80 1 -- start start 2023-11-27T15:27:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.101+0000 7f372a4e0b80 1 Processor -- start 2023-11-27T15:27:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd7346b0 0x55d4fd8ea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8ea9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd7cb880 con 0x55d4fd8e15c0 2023-11-27T15:27:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd7cb520 con 0x55d4fd7346b0 2023-11-27T15:27:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd6ad8c0 con 0x55d4fd8e2320 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f372a4e0b80 10 client.0 _ll_register_callbacks cb 0x55d4fd7e9720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f371519c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd7346b0 0x55d4fd8ea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f371519c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd7346b0 0x55d4fd8ea480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:41670/0 (socket says 172.21.15.8:41670) 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f371519c700 1 -- 172.21.15.8:0/2154886764 learned_addr learned my addr 172.21.15.8:0/2154886764 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f371519c700 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f371519c700 1 -- 172.21.15.8:0/2154886764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e2320 msgr2=0x55d4fd8f1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f371519c700 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f371519c700 1 -- 172.21.15.8:0/2154886764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd8e15c0 msgr2=0x55d4fd8ea9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f371519c700 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8ea9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.102+0000 7f371519c700 1 -- 172.21.15.8:0/2154886764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370800e410 con 0x55d4fd7346b0 2023-11-27T15:27:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.103+0000 7f371519c700 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd7346b0 0x55d4fd8ea480 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3708009b50 tx=0x7f37080043d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b2b624ccdffcd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.103+0000 7f36effff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:27:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.103+0000 7f36effff700 1 -- 172.21.15.8:0/2154886764 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708007910 con 0x55d4fd7346b0 2023-11-27T15:27:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.103+0000 7f372a4e0b80 1 -- 172.21.15.8:0/2154886764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4fd8cb710 con 0x55d4fd7346b0 2023-11-27T15:27:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.103+0000 7f36effff700 1 -- 172.21.15.8:0/2154886764 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3708007a70 con 0x55d4fd7346b0 2023-11-27T15:27:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.103+0000 7f36effff700 1 -- 172.21.15.8:0/2154886764 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3708003e90 con 0x55d4fd7346b0 2023-11-27T15:27:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.103+0000 7f372a4e0b80 10 client.25538 fetch_fsmap learned FSMap version 25 2023-11-27T15:27:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.103+0000 7f372a4e0b80 1 -- 172.21.15.8:0/2154886764 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d4fd7f0130 con 0x55d4fd7346b0 2023-11-27T15:27:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.104+0000 7f36effff700 1 -- 172.21.15.8:0/2154886764 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3708030080 con 0x55d4fd7346b0 2023-11-27T15:27:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.104+0000 7f36effff700 1 -- 172.21.15.8:0/2154886764 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f370801f430 con 0x55d4fd7346b0 2023-11-27T15:27:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.104+0000 7f372a4e0b80 10 client.25538 fetch_fsmap finished waiting for FSMap version 25 2023-11-27T15:27:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.104+0000 7f372a4e0b80 10 client.25538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:27:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.104+0000 7f372a4e0b80 20 client.25538 populate_metadata read hostname 'smithi008' 2023-11-27T15:27:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.104+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d4fd8f6150 0x55d4fd8f8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.104+0000 7f372a4e0b80 4 client.25538 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.104+0000 7f372a4e0b80 1 -- 172.21.15.8:0/2154886764 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4fd7ec040 con 0x55d4fd8f6150 2023-11-27T15:27:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.104+0000 7f370ffff700 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d4fd8f6150 0x55d4fd8f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.105+0000 7f370ffff700 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d4fd8f6150 0x55d4fd8f8530 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8438ed4d59e5526d server_cookie=159f8db9e5a57d99 in_seq=0 out_seq=0 2023-11-27T15:27:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.105+0000 7f36effff700 10 client.25538 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.106+0000 7f36effff700 1 -- 172.21.15.8:0/2154886764 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d4fd7ec040 con 0x55d4fd8f6150 2023-11-27T15:27:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.106+0000 7f36effff700 10 client.25538 handle_command_reply: tid=0 2023-11-27T15:27:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.107+0000 7f372a4e0b80 1 client.25538 shutdown 2023-11-27T15:27:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.107+0000 7f372a4e0b80 1 -- 172.21.15.8:0/2154886764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd7346b0 msgr2=0x55d4fd8ea480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.107+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd7346b0 0x55d4fd8ea480 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3708009b50 tx=0x7f37080043d0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.107+0000 7f372a4e0b80 1 -- 172.21.15.8:0/2154886764 shutdown_connections 2023-11-27T15:27:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.107+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d4fd8f6150 0x55d4fd8f8530 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.107+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.107+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd7346b0 0x55d4fd8ea480 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.107+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/2154886764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8ea9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.107+0000 7f372a4e0b80 1 -- 172.21.15.8:0/2154886764 >> 172.21.15.8:0/2154886764 conn(0x55d4fd8cdf50 msgr2=0x55d4fd8d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.107+0000 7f36effff700 0 client.25538 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.108+0000 7f372a4e0b80 1 -- 172.21.15.8:0/2154886764 shutdown_connections 2023-11-27T15:27:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.108+0000 7f372a4e0b80 1 -- 172.21.15.8:0/2154886764 wait complete. 2023-11-27T15:27:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.108+0000 7f372a4e0b80 20 client.25538 trim_cache size 0 max 16384 2023-11-27T15:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.124+0000 7f372a4e0b80 1 Processor -- start 2023-11-27T15:27:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.124+0000 7f372a4e0b80 1 -- start start 2023-11-27T15:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.124+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.124+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd8e1990 con 0x55d4fd7346b0 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd8eb600 con 0x55d4fd8e2320 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd8e8470 con 0x55d4fd8e15c0 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f371519c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f370ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f371519c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:52870/0 (socket says 172.21.15.8:52870) 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f371519c700 1 -- 172.21.15.8:0/584013978 learned_addr learned my addr 172.21.15.8:0/584013978 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f371519c700 1 --2- 172.21.15.8:0/584013978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f370ffff700 1 -- 172.21.15.8:0/584013978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 msgr2=0x55d4fd8f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f370ffff700 1 --2- 172.21.15.8:0/584013978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f370ffff700 1 -- 172.21.15.8:0/584013978 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 msgr2=0x55d4fd8e86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f370ffff700 1 --2- 172.21.15.8:0/584013978 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f370ffff700 1 -- 172.21.15.8:0/584013978 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4fd7efe80 con 0x55d4fd8e15c0 2023-11-27T15:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.125+0000 7f371519c700 1 --2- 172.21.15.8:0/584013978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.126+0000 7f370ffff700 1 --2- 172.21.15.8:0/584013978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8e8c00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000c00 tx=0x7f36fc009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1667c725951a5d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.126+0000 7f36fb7fe700 1 -- 172.21.15.8:0/584013978 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc025430 con 0x55d4fd8e15c0 2023-11-27T15:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.126+0000 7f36fb7fe700 1 -- 172.21.15.8:0/584013978 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36fc01e030 con 0x55d4fd8e15c0 2023-11-27T15:27:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.126+0000 7f36fb7fe700 1 -- 172.21.15.8:0/584013978 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc00ba50 con 0x55d4fd8e15c0 2023-11-27T15:27:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.127+0000 7f372a4e0b80 1 -- 172.21.15.8:0/584013978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e15c0 msgr2=0x55d4fd8e8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.127+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/584013978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8e8c00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000c00 tx=0x7f36fc009290 comp rx=0 tx=0).stop 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.127+0000 7f372a4e0b80 1 -- 172.21.15.8:0/584013978 shutdown_connections 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.127+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/584013978 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.127+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/584013978 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 0x55d4fd8e86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.127+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/584013978 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8e8c00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.127+0000 7f372a4e0b80 1 -- 172.21.15.8:0/584013978 >> 172.21.15.8:0/584013978 conn(0x55d4fd8cdf50 msgr2=0x55d4fd8fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.127+0000 7f372a4e0b80 1 -- 172.21.15.8:0/584013978 shutdown_connections 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.127+0000 7f372a4e0b80 1 -- 172.21.15.8:0/584013978 wait complete. 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.127+0000 7f372a4e0b80 1 -- start start 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f372a4e0b80 1 Processor -- start 2023-11-27T15:27:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 0x55d4fd8edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f372a4e0b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd8f1660 con 0x55d4fd7346b0 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd8f17d0 con 0x55d4fd8e15c0 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f372a4e0b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4fd8f1990 con 0x55d4fd8e2320 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f372a4e0b80 10 client.0 _ll_register_callbacks cb 0x7f370800e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f370ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f371519c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 0x55d4fd8edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f370ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8ee100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:41692/0 (socket says 172.21.15.8:41692) 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.128+0000 7f370ffff700 1 -- 172.21.15.8:0/3743444723 learned_addr learned my addr 172.21.15.8:0/3743444723 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.129+0000 7f370ffff700 1 -- 172.21.15.8:0/3743444723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e2320 msgr2=0x55d4fd8f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.129+0000 7f370ffff700 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.129+0000 7f370ffff700 1 -- 172.21.15.8:0/3743444723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 msgr2=0x55d4fd8edbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.129+0000 7f370ffff700 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 0x55d4fd8edbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.129+0000 7f370ffff700 1 -- 172.21.15.8:0/3743444723 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3710108fc0 con 0x55d4fd8e15c0 2023-11-27T15:27:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.129+0000 7f370ffff700 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8ee100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36fc005ff0 tx=0x7f36fc005670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7731598d35190de9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.129+0000 7f36effff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:27:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.130+0000 7f36effff700 1 -- 172.21.15.8:0/3743444723 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0259e0 con 0x55d4fd8e15c0 2023-11-27T15:27:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.130+0000 7f36effff700 1 -- 172.21.15.8:0/3743444723 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f36fc025b40 con 0x55d4fd8e15c0 2023-11-27T15:27:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.130+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3743444723 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f370800f040 con 0x55d4fd8e15c0 2023-11-27T15:27:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.130+0000 7f36effff700 1 -- 172.21.15.8:0/3743444723 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36fc025d00 con 0x55d4fd8e15c0 2023-11-27T15:27:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.130+0000 7f372a4e0b80 10 client.25544 fetch_fsmap learned FSMap version 25 2023-11-27T15:27:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.130+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3743444723 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d4fd640c00 con 0x55d4fd8e15c0 2023-11-27T15:27:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.130+0000 7f36effff700 1 -- 172.21.15.8:0/3743444723 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f370800f040 con 0x55d4fd8e15c0 2023-11-27T15:27:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.131+0000 7f36effff700 1 -- 172.21.15.8:0/3743444723 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f36fc004800 con 0x55d4fd8e15c0 2023-11-27T15:27:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.131+0000 7f372a4e0b80 10 client.25544 fetch_fsmap finished waiting for FSMap version 25 2023-11-27T15:27:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.131+0000 7f372a4e0b80 10 client.25544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:27:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.131+0000 7f372a4e0b80 20 client.25544 populate_metadata read hostname 'smithi008' 2023-11-27T15:27:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.131+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d4fd94afc0 0x55d4fd94d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.131+0000 7f372a4e0b80 4 client.25544 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:27:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.131+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3743444723 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d4fd7f8600 con 0x55d4fd94afc0 2023-11-27T15:27:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.131+0000 7f371519c700 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d4fd94afc0 0x55d4fd94d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.132+0000 7f36effff700 10 client.25544 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:56.132+0000 7f371519c700 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d4fd94afc0 0x55d4fd94d3d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ee931b603ea72f server_cookie=9820a2ad2cb0363e in_seq=0 out_seq=0 2023-11-27T15:27:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:57 smithi060 ceph-mon[131518]: pgmap v421: 129 pgs: 129 active+clean; 18 MiB data, 129 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 62 op/s 2023-11-27T15:27:57.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:57 smithi008 ceph-mon[128788]: pgmap v421: 129 pgs: 129 active+clean; 18 MiB data, 129 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 62 op/s 2023-11-27T15:27:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:57 smithi029 ceph-mon[131166]: pgmap v421: 129 pgs: 129 active+clean; 18 MiB data, 129 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 62 op/s 2023-11-27T15:27:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f36effff700 1 -- 172.21.15.8:0/3743444723 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d4fd7f8600 con 0x55d4fd94afc0 2023-11-27T15:27:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f36effff700 10 client.25544 handle_command_reply: tid=0 2023-11-27T15:27:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f372a4e0b80 1 client.25544 shutdown 2023-11-27T15:27:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3743444723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e15c0 msgr2=0x55d4fd8ee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8ee100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36fc005ff0 tx=0x7f36fc005670 comp rx=0 tx=0).stop 2023-11-27T15:27:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3743444723 shutdown_connections 2023-11-27T15:27:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d4fd94afc0 0x55d4fd94d3d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d4fd8e2320 0x55d4fd8f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d4fd7346b0 0x55d4fd8edbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f372a4e0b80 1 --2- 172.21.15.8:0/3743444723 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d4fd8e15c0 0x55d4fd8ee100 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f36effff700 0 client.25544 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:27:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.693+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3743444723 >> 172.21.15.8:0/3743444723 conn(0x55d4fd8cdf50 msgr2=0x55d4fd8e86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.694+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3743444723 shutdown_connections 2023-11-27T15:27:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.694+0000 7f372a4e0b80 1 -- 172.21.15.8:0/3743444723 wait complete. 2023-11-27T15:27:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.694+0000 7f372a4e0b80 20 client.25544 trim_cache size 0 max 16384 2023-11-27T15:27:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:27:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:27:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:27:58.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:27:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.695+0000 7f3716bff700 1 -- 172.21.15.8:0/2891450665 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f37000a11d0 msgr2=0x7f37000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.695+0000 7f3716bff700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f37000a11d0 0x7f37000a3680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f370400c5b0 tx=0x7f3704000bc0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.695+0000 7f3716bff700 1 -- 172.21.15.8:0/2891450665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3710100250 msgr2=0x7f3710198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.695+0000 7f3716bff700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3710100250 0x7f3710198240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f370400e970 tx=0x7f370400ec80 comp rx=0 tx=0).stop 2023-11-27T15:27:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.695+0000 7f3716bff700 1 -- 172.21.15.8:0/2891450665 shutdown_connections 2023-11-27T15:27:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.695+0000 7f3716bff700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f37000a11d0 0x7f37000a3680 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.696+0000 7f3716bff700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3710100250 0x7f3710198240 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.696+0000 7f3716bff700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3710101dd0 0x7f371019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.696+0000 7f3716bff700 1 --2- 172.21.15.8:0/2891450665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3710101440 0x7f3710198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.696+0000 7f3716bff700 1 -- 172.21.15.8:0/2891450665 >> 172.21.15.8:0/2891450665 conn(0x7f37100fb850 msgr2=0x7f3710106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.696+0000 7f3716bff700 1 -- 172.21.15.8:0/2891450665 shutdown_connections 2023-11-27T15:27:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:27:58.696+0000 7f3716bff700 1 -- 172.21.15.8:0/2891450665 wait complete. 2023-11-27T15:27:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:59 smithi060 ceph-mon[131518]: pgmap v422: 129 pgs: 129 active+clean; 20 MiB data, 162 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2023-11-27T15:27:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:27:59 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:27:59.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:59 smithi008 ceph-mon[128788]: pgmap v422: 129 pgs: 129 active+clean; 20 MiB data, 162 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2023-11-27T15:27:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:27:59 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:27:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:59 smithi029 ceph-mon[131166]: pgmap v422: 129 pgs: 129 active+clean; 20 MiB data, 162 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2023-11-27T15:27:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:27:59 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:28:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:01 smithi060 ceph-mon[131518]: pgmap v423: 129 pgs: 129 active+clean; 16 MiB data, 167 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2023-11-27T15:28:01.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:01 smithi008 ceph-mon[128788]: pgmap v423: 129 pgs: 129 active+clean; 16 MiB data, 167 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2023-11-27T15:28:01.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:01 smithi029 ceph-mon[131166]: pgmap v423: 129 pgs: 129 active+clean; 16 MiB data, 167 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 189 op/s 2023-11-27T15:28:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:03 smithi060 ceph-mon[131518]: pgmap v424: 129 pgs: 129 active+clean; 18 MiB data, 173 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2023-11-27T15:28:03.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:03 smithi008 ceph-mon[128788]: pgmap v424: 129 pgs: 129 active+clean; 18 MiB data, 173 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2023-11-27T15:28:03.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:03 smithi029 ceph-mon[131166]: pgmap v424: 129 pgs: 129 active+clean; 18 MiB data, 173 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 283 op/s 2023-11-27T15:28:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:04 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:28:04.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:04 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:04.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:04 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:28:04.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:04 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:04 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:28:04.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:04 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:05 smithi060 ceph-mon[131518]: pgmap v425: 129 pgs: 129 active+clean; 18 MiB data, 185 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 289 op/s 2023-11-27T15:28:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:05 smithi008 ceph-mon[128788]: pgmap v425: 129 pgs: 129 active+clean; 18 MiB data, 185 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 289 op/s 2023-11-27T15:28:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:05 smithi029 ceph-mon[131166]: pgmap v425: 129 pgs: 129 active+clean; 18 MiB data, 185 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 289 op/s 2023-11-27T15:28:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:06.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:06 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:28:06.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:06 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:28:06.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:06 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:28:06.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:06 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:06.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:06 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:28:06.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:06 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:28:06.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:06 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:28:06.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:06 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:06.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:06 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:28:06.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:06 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:28:06.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:06 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:28:06.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:06 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:28:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:07 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:28:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:07 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:28:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:07 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:28:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:07 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:28:07.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:07 smithi060 ceph-mon[131518]: pgmap v426: 129 pgs: 129 active+clean; 22 MiB data, 189 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 291 op/s 2023-11-27T15:28:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:07 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:28:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:07 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:28:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:07 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:28:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:07 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:28:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:07 smithi008 ceph-mon[128788]: pgmap v426: 129 pgs: 129 active+clean; 22 MiB data, 189 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 291 op/s 2023-11-27T15:28:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:07 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:28:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:07 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:28:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:07 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:28:07.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:07 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:28:07.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:07 smithi029 ceph-mon[131166]: pgmap v426: 129 pgs: 129 active+clean; 22 MiB data, 189 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 291 op/s 2023-11-27T15:28:09.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:09 smithi060 ceph-mon[131518]: pgmap v427: 129 pgs: 129 active+clean; 26 MiB data, 203 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 260 op/s 2023-11-27T15:28:09.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:09 smithi008 ceph-mon[128788]: pgmap v427: 129 pgs: 129 active+clean; 26 MiB data, 203 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 260 op/s 2023-11-27T15:28:09.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:09 smithi029 ceph-mon[131166]: pgmap v427: 129 pgs: 129 active+clean; 26 MiB data, 203 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 260 op/s 2023-11-27T15:28:11.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:11 smithi060 ceph-mon[131518]: pgmap v428: 129 pgs: 129 active+clean; 26 MiB data, 210 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-11-27T15:28:11.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:11 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:28:11.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:11 smithi008 ceph-mon[128788]: pgmap v428: 129 pgs: 129 active+clean; 26 MiB data, 210 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-11-27T15:28:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:11 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:28:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:11 smithi029 ceph-mon[131166]: pgmap v428: 129 pgs: 129 active+clean; 26 MiB data, 210 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 184 op/s 2023-11-27T15:28:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:11 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:28:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.896+0000 7f603d1ff700 1 -- 172.21.15.8:0/1530155760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60380feab0 msgr2=0x7f6038100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.896+0000 7f603d1ff700 1 --2- 172.21.15.8:0/1530155760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60380feab0 0x7f6038100ed0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6028009f90 tx=0x7f602800b330 comp rx=0 tx=0).stop 2023-11-27T15:28:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.896+0000 7f603d1ff700 1 -- 172.21.15.8:0/1530155760 shutdown_connections 2023-11-27T15:28:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.896+0000 7f603d1ff700 1 --2- 172.21.15.8:0/1530155760 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6038103dd0 0x7f6038104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.896+0000 7f603d1ff700 1 --2- 172.21.15.8:0/1530155760 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60380feab0 0x7f6038100ed0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.896+0000 7f603d1ff700 1 --2- 172.21.15.8:0/1530155760 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6038101410 0x7f6038103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.896+0000 7f603d1ff700 1 -- 172.21.15.8:0/1530155760 >> 172.21.15.8:0/1530155760 conn(0x7f60380fa720 msgr2=0x7f60380fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.896+0000 7f603d1ff700 1 -- 172.21.15.8:0/1530155760 shutdown_connections 2023-11-27T15:28:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.896+0000 7f603d1ff700 1 -- 172.21.15.8:0/1530155760 wait complete. 2023-11-27T15:28:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.897+0000 7f603d1ff700 1 Processor -- start 2023-11-27T15:28:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.898+0000 7f603d1ff700 1 -- start start 2023-11-27T15:28:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.898+0000 7f603d1ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60380feab0 0x7f603819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.898+0000 7f603d1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6038101410 0x7f603819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.898+0000 7f603d1ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6038103dd0 0x7f60381a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.898+0000 7f603d1ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60381a4170 con 0x7f60380feab0 2023-11-27T15:28:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.898+0000 7f603d1ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60381a42b0 con 0x7f6038103dd0 2023-11-27T15:28:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.898+0000 7f603d1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60381a4440 con 0x7f6038101410 2023-11-27T15:28:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.899+0000 7f603759e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6038103dd0 0x7f60381a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.899+0000 7f6036d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60380feab0 0x7f603819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.899+0000 7f603759e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6038103dd0 0x7f60381a3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:42126/0 (socket says 172.21.15.8:42126) 2023-11-27T15:28:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.899+0000 7f602e59c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6038101410 0x7f603819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.899+0000 7f603759e700 1 -- 172.21.15.8:0/2717338469 learned_addr learned my addr 172.21.15.8:0/2717338469 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.899+0000 7f6036d9d700 1 -- 172.21.15.8:0/2717338469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6038101410 msgr2=0x7f603819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.899+0000 7f6036d9d700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6038101410 0x7f603819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.899+0000 7f6036d9d700 1 -- 172.21.15.8:0/2717338469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6038103dd0 msgr2=0x7f60381a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.899+0000 7f6036d9d700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6038103dd0 0x7f60381a3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.900+0000 7f6036d9d700 1 -- 172.21.15.8:0/2717338469 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6020009520 con 0x7f60380feab0 2023-11-27T15:28:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.900+0000 7f6036d9d700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60380feab0 0x7f603819c5f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6028006070 tx=0x7f6028004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.900+0000 7f6034d99700 1 -- 172.21.15.8:0/2717338469 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800e040 con 0x7f60380feab0 2023-11-27T15:28:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.900+0000 7f6034d99700 1 -- 172.21.15.8:0/2717338469 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f602800be00 con 0x7f60380feab0 2023-11-27T15:28:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.900+0000 7f603d1ff700 1 -- 172.21.15.8:0/2717338469 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6028009c40 con 0x7f60380feab0 2023-11-27T15:28:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.900+0000 7f6034d99700 1 -- 172.21.15.8:0/2717338469 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60280052a0 con 0x7f60380feab0 2023-11-27T15:28:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.900+0000 7f603d1ff700 1 -- 172.21.15.8:0/2717338469 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60381121e0 con 0x7f60380feab0 2023-11-27T15:28:11.909 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.906+0000 7f6034d99700 1 -- 172.21.15.8:0/2717338469 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f602801d470 con 0x7f60380feab0 2023-11-27T15:28:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.907+0000 7f6050a98b80 1 Processor -- start 2023-11-27T15:28:11.910 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.907+0000 7f6050a98b80 1 -- start start 2023-11-27T15:28:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.907+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.907+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fcb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.907+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fcbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.907+0000 7f6050a98b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560979ed0000 con 0x560979fcadb0 2023-11-27T15:28:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.907+0000 7f6050a98b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560979f22840 con 0x560979fcbb10 2023-11-27T15:28:11.911 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.907+0000 7f6050a98b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560979f1bcd0 con 0x560979e266b0 2023-11-27T15:28:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.908+0000 7f602e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fcbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.908+0000 7f602e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fcbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:42132/0 (socket says 172.21.15.8:42132) 2023-11-27T15:28:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.908+0000 7f602e59c700 1 -- 172.21.15.8:0/904568776 learned_addr learned my addr 172.21.15.8:0/904568776 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.908+0000 7f603759e700 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.908+0000 7f6036d9d700 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fcb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.909+0000 7f602e59c700 1 -- 172.21.15.8:0/904568776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 msgr2=0x560979fca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:11.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.909+0000 7f602e59c700 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.909+0000 7f602e59c700 1 -- 172.21.15.8:0/904568776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 msgr2=0x560979fcb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.909+0000 7f602e59c700 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fcb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.909+0000 7f602e59c700 1 -- 172.21.15.8:0/904568776 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560979d9d5d0 con 0x560979fcbb10 2023-11-27T15:28:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.909+0000 7f603759e700 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.909+0000 7f6036d9d700 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fcb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.909+0000 7f602e59c700 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fcbf80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6020009f10 tx=0x7f6020011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57da685ba88a9f4f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.910+0000 7f6034d99700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60240a11d0 0x7f60240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.910+0000 7f6034d99700 1 -- 172.21.15.8:0/2717338469 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f60280ce1b0 con 0x7f60380feab0 2023-11-27T15:28:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.910+0000 7f603759e700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60240a11d0 0x7f60240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.910+0000 7f602d59a700 1 -- 172.21.15.8:0/904568776 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020019630 con 0x560979fcbb10 2023-11-27T15:28:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.910+0000 7f602d59a700 1 -- 172.21.15.8:0/904568776 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6020019c50 con 0x560979fcbb10 2023-11-27T15:28:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.910+0000 7f602d59a700 1 -- 172.21.15.8:0/904568776 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602000ece0 con 0x560979fcbb10 2023-11-27T15:28:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.911+0000 7f6050a98b80 1 -- 172.21.15.8:0/904568776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 msgr2=0x560979fcbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.911+0000 7f6050a98b80 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fcbf80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6020009f10 tx=0x7f6020011720 comp rx=0 tx=0).stop 2023-11-27T15:28:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.912+0000 7f603759e700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60240a11d0 0x7f60240a3680 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6030009ec0 tx=0x7f6030009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.912+0000 7f6050a98b80 1 -- 172.21.15.8:0/904568776 shutdown_connections 2023-11-27T15:28:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.912+0000 7f6050a98b80 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.912+0000 7f6050a98b80 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fcbf80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.915 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.912+0000 7f6050a98b80 1 --2- 172.21.15.8:0/904568776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fcb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.912+0000 7f6050a98b80 1 -- 172.21.15.8:0/904568776 >> 172.21.15.8:0/904568776 conn(0x560979fbff50 msgr2=0x560979fc5100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.913+0000 7f6050a98b80 1 -- 172.21.15.8:0/904568776 shutdown_connections 2023-11-27T15:28:11.916 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.913+0000 7f6050a98b80 1 -- 172.21.15.8:0/904568776 wait complete. 2023-11-27T15:28:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.913+0000 7f6050a98b80 1 -- start start 2023-11-27T15:28:11.917 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.914+0000 7f6050a98b80 1 Processor -- start 2023-11-27T15:28:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.914+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fd3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.914+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcadb0 0x560979fd4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.914+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 0x560979fdb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.914+0000 7f6050a98b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560979ebd880 con 0x560979fcbb10 2023-11-27T15:28:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.914+0000 7f6050a98b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560979ebd520 con 0x560979fcadb0 2023-11-27T15:28:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.914+0000 7f6050a98b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560979d9f8c0 con 0x560979e266b0 2023-11-27T15:28:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.914+0000 7f6050a98b80 10 client.0 _ll_register_callbacks cb 0x560979edb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.915+0000 7f602e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcadb0 0x560979fd4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.915+0000 7f603759e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 0x560979fdb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.915+0000 7f602e59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcadb0 0x560979fd4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:42146/0 (socket says 172.21.15.8:42146) 2023-11-27T15:28:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.915+0000 7f602e59c700 1 -- 172.21.15.8:0/3610868169 learned_addr learned my addr 172.21.15.8:0/3610868169 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.915+0000 7f602e59c700 1 -- 172.21.15.8:0/3610868169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 msgr2=0x560979fd3d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:28:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.915+0000 7f602e59c700 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fd3d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.915+0000 7f602e59c700 1 -- 172.21.15.8:0/3610868169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 msgr2=0x560979fdb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.915+0000 7f602e59c700 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 0x560979fdb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.915+0000 7f602e59c700 1 -- 172.21.15.8:0/3610868169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6020009b70 con 0x560979fcadb0 2023-11-27T15:28:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.915+0000 7f603759e700 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 0x560979fdb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.916+0000 7f602e59c700 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcadb0 0x560979fd4260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f602001c040 tx=0x7f602000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acf210a55c803eee server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.916+0000 7f601e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:28:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.916+0000 7f601e7fc700 1 -- 172.21.15.8:0/3610868169 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020019630 con 0x560979fcadb0 2023-11-27T15:28:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.916+0000 7f601e7fc700 1 -- 172.21.15.8:0/3610868169 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6020019790 con 0x560979fcadb0 2023-11-27T15:28:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.916+0000 7f6050a98b80 1 -- 172.21.15.8:0/3610868169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560979fbd710 con 0x560979fcadb0 2023-11-27T15:28:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.917+0000 7f601e7fc700 1 -- 172.21.15.8:0/3610868169 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6020019950 con 0x560979fcadb0 2023-11-27T15:28:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.918+0000 7f6050a98b80 10 client.25556 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.918+0000 7f6050a98b80 1 -- 172.21.15.8:0/3610868169 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560979ee2130 con 0x560979fcadb0 2023-11-27T15:28:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.919+0000 7f601e7fc700 1 -- 172.21.15.8:0/3610868169 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560979fbd710 con 0x560979fcadb0 2023-11-27T15:28:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.919+0000 7f601e7fc700 1 -- 172.21.15.8:0/3610868169 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f60200049b0 con 0x560979fcadb0 2023-11-27T15:28:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.919+0000 7f6050a98b80 10 client.25556 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.919+0000 7f6050a98b80 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:28:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.919+0000 7f6050a98b80 20 client.25556 populate_metadata read hostname 'smithi008' 2023-11-27T15:28:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.919+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560979fdf960 0x560979fe1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.919+0000 7f6050a98b80 4 client.25556 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.919+0000 7f6050a98b80 1 -- 172.21.15.8:0/3610868169 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560979ede040 con 0x560979fdf960 2023-11-27T15:28:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.919+0000 7f6036d9d700 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560979fdf960 0x560979fe1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.920+0000 7f6036d9d700 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560979fdf960 0x560979fe1d70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78b244d21cfa4e6 server_cookie=8dcc91e3abcbf341 in_seq=0 out_seq=0 2023-11-27T15:28:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.920+0000 7f601e7fc700 10 client.25556 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.921+0000 7f601e7fc700 1 -- 172.21.15.8:0/3610868169 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560979ede040 con 0x560979fdf960 2023-11-27T15:28:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.921+0000 7f601e7fc700 10 client.25556 handle_command_reply: tid=0 2023-11-27T15:28:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.921+0000 7f6050a98b80 1 client.25556 shutdown 2023-11-27T15:28:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.922+0000 7f6050a98b80 1 -- 172.21.15.8:0/3610868169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcadb0 msgr2=0x560979fd4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.922+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcadb0 0x560979fd4260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f602001c040 tx=0x7f602000ef60 comp rx=0 tx=0).stop 2023-11-27T15:28:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.923+0000 7f6050a98b80 1 -- 172.21.15.8:0/3610868169 shutdown_connections 2023-11-27T15:28:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.923+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x560979fdf960 0x560979fe1d70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.923+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fd3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.923+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 0x560979fdb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.923+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3610868169 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcadb0 0x560979fd4260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.923+0000 7f6050a98b80 1 -- 172.21.15.8:0/3610868169 >> 172.21.15.8:0/3610868169 conn(0x560979fbff50 msgr2=0x560979fc47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.923+0000 7f601e7fc700 0 client.25556 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.924+0000 7f6050a98b80 1 -- 172.21.15.8:0/3610868169 shutdown_connections 2023-11-27T15:28:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.924+0000 7f6050a98b80 1 -- 172.21.15.8:0/3610868169 wait complete. 2023-11-27T15:28:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.924+0000 7f6050a98b80 20 client.25556 trim_cache size 0 max 16384 2023-11-27T15:28:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.947+0000 7f6050a98b80 1 Processor -- start 2023-11-27T15:28:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.947+0000 7f6050a98b80 1 -- start start 2023-11-27T15:28:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.947+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fd3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.947+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fdb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.947+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fd4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.948+0000 7f6050a98b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x560979fcb450 con 0x560979fcadb0 2023-11-27T15:28:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.948+0000 7f6050a98b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x560979fbfb70 con 0x560979fcbb10 2023-11-27T15:28:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.948+0000 7f6050a98b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x560979fc6bf0 con 0x560979e266b0 2023-11-27T15:28:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.948+0000 7f602e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fdb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.948+0000 7f602e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fdb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51642/0 (socket says 172.21.15.8:51642) 2023-11-27T15:28:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.948+0000 7f602e59c700 1 -- 172.21.15.8:0/4191453089 learned_addr learned my addr 172.21.15.8:0/4191453089 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.949+0000 7f6036d9d700 1 --2- 172.21.15.8:0/4191453089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fd3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.949+0000 7f602e59c700 1 -- 172.21.15.8:0/4191453089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 msgr2=0x560979fd3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.949+0000 7f602e59c700 1 --2- 172.21.15.8:0/4191453089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fd3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.949+0000 7f602e59c700 1 -- 172.21.15.8:0/4191453089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 msgr2=0x560979fd4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.949+0000 7f603759e700 1 --2- 172.21.15.8:0/4191453089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fd4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.950+0000 7f602e59c700 1 --2- 172.21.15.8:0/4191453089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fd4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.950+0000 7f602e59c700 1 -- 172.21.15.8:0/4191453089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560979ee1e80 con 0x560979fcadb0 2023-11-27T15:28:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.950+0000 7f603759e700 1 --2- 172.21.15.8:0/4191453089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fd4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.951+0000 7f602e59c700 1 --2- 172.21.15.8:0/4191453089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fdb260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6020005850 tx=0x7f60200213e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32491fdbf4a967e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.951+0000 7f602dd9b700 1 -- 172.21.15.8:0/4191453089 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020022400 con 0x560979fcadb0 2023-11-27T15:28:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.951+0000 7f602dd9b700 1 -- 172.21.15.8:0/4191453089 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6020019dc0 con 0x560979fcadb0 2023-11-27T15:28:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.951+0000 7f602dd9b700 1 -- 172.21.15.8:0/4191453089 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60200198f0 con 0x560979fcadb0 2023-11-27T15:28:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.952+0000 7f6050a98b80 1 -- 172.21.15.8:0/4191453089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 msgr2=0x560979fdb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.952+0000 7f6050a98b80 1 --2- 172.21.15.8:0/4191453089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fdb260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6020005850 tx=0x7f60200213e0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.952+0000 7f6050a98b80 1 -- 172.21.15.8:0/4191453089 shutdown_connections 2023-11-27T15:28:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.952+0000 7f6050a98b80 1 --2- 172.21.15.8:0/4191453089 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979e266b0 0x560979fd3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.952+0000 7f6050a98b80 1 --2- 172.21.15.8:0/4191453089 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979fcbb10 0x560979fd4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.952+0000 7f6050a98b80 1 --2- 172.21.15.8:0/4191453089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcadb0 0x560979fdb260 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.952+0000 7f6050a98b80 1 -- 172.21.15.8:0/4191453089 >> 172.21.15.8:0/4191453089 conn(0x560979fbff50 msgr2=0x560979fd2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.953+0000 7f6050a98b80 1 -- 172.21.15.8:0/4191453089 shutdown_connections 2023-11-27T15:28:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.953+0000 7f6050a98b80 1 -- 172.21.15.8:0/4191453089 wait complete. 2023-11-27T15:28:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.953+0000 7f6050a98b80 1 -- start start 2023-11-27T15:28:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.953+0000 7f6050a98b80 1 Processor -- start 2023-11-27T15:28:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979e266b0 0x560979fd9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979fcadb0 0x56097a02f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6050a98b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 0x56097a031dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6050a98b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097a032510 con 0x560979fcbb10 2023-11-27T15:28:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6050a98b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097a032680 con 0x560979e266b0 2023-11-27T15:28:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6050a98b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097a032840 con 0x560979fcadb0 2023-11-27T15:28:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6050a98b80 10 client.0 _ll_register_callbacks cb 0x7f6020009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6036d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979e266b0 0x560979fd9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6036d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979e266b0 0x560979fd9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:42168/0 (socket says 172.21.15.8:42168) 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6036d9d700 1 -- 172.21.15.8:0/3628231879 learned_addr learned my addr 172.21.15.8:0/3628231879 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f603759e700 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 0x56097a031dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6036d9d700 1 -- 172.21.15.8:0/3628231879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979fcadb0 msgr2=0x56097a02f7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f602e59c700 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979fcadb0 0x56097a02f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.954+0000 7f6036d9d700 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979fcadb0 0x56097a02f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f6036d9d700 1 -- 172.21.15.8:0/3628231879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 msgr2=0x56097a031dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f6036d9d700 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 0x56097a031dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f6036d9d700 1 -- 172.21.15.8:0/3628231879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560979ee2130 con 0x560979e266b0 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f603759e700 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 0x56097a031dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f6036d9d700 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979e266b0 0x560979fd9200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6028097d70 tx=0x7f6028004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b44477a9a5091998 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f601e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:28:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f601e7fc700 1 -- 172.21.15.8:0/3628231879 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602809ac60 con 0x560979e266b0 2023-11-27T15:28:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f601e7fc700 1 -- 172.21.15.8:0/3628231879 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f602802d2f0 con 0x560979e266b0 2023-11-27T15:28:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f6050a98b80 1 -- 172.21.15.8:0/3628231879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6020019c50 con 0x560979e266b0 2023-11-27T15:28:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f601e7fc700 1 -- 172.21.15.8:0/3628231879 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f602809b300 con 0x560979e266b0 2023-11-27T15:28:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f6050a98b80 10 client.25559 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.955+0000 7f6050a98b80 1 -- 172.21.15.8:0/3628231879 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560979d32c00 con 0x560979e266b0 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.956+0000 7f601e7fc700 1 -- 172.21.15.8:0/3628231879 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6020019c50 con 0x560979e266b0 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.956+0000 7f601e7fc700 1 -- 172.21.15.8:0/3628231879 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f602801d720 con 0x560979e266b0 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.956+0000 7f6050a98b80 10 client.25559 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.956+0000 7f6050a98b80 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.956+0000 7f6050a98b80 20 client.25559 populate_metadata read hostname 'smithi008' 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.956+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56097a033fc0 0x56097a0363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.956+0000 7f6050a98b80 4 client.25559 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.956+0000 7f6050a98b80 1 -- 172.21.15.8:0/3628231879 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560979eea600 con 0x56097a033fc0 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.957+0000 7f602e59c700 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56097a033fc0 0x56097a0363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.957+0000 7f602e59c700 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56097a033fc0 0x56097a0363d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc9316f1c77db14 server_cookie=7bb4c82d44b2ecca in_seq=0 out_seq=0 2023-11-27T15:28:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:11.958+0000 7f601e7fc700 10 client.25559 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:13 smithi060 ceph-mon[131518]: pgmap v429: 129 pgs: 129 active+clean; 31 MiB data, 226 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 130 op/s 2023-11-27T15:28:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:13 smithi008 ceph-mon[128788]: pgmap v429: 129 pgs: 129 active+clean; 31 MiB data, 226 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 130 op/s 2023-11-27T15:28:13.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:13 smithi029 ceph-mon[131166]: pgmap v429: 129 pgs: 129 active+clean; 31 MiB data, 226 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 130 op/s 2023-11-27T15:28:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.067+0000 7f601e7fc700 1 -- 172.21.15.8:0/3628231879 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560979eea600 con 0x56097a033fc0 2023-11-27T15:28:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.067+0000 7f601e7fc700 10 client.25559 handle_command_reply: tid=0 2023-11-27T15:28:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.067+0000 7f6050a98b80 1 client.25559 shutdown 2023-11-27T15:28:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f6050a98b80 1 -- 172.21.15.8:0/3628231879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979e266b0 msgr2=0x560979fd9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979e266b0 0x560979fd9200 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6028097d70 tx=0x7f6028004f60 comp rx=0 tx=0).stop 2023-11-27T15:28:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f6050a98b80 1 -- 172.21.15.8:0/3628231879 shutdown_connections 2023-11-27T15:28:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x560979fcbb10 0x56097a031dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x560979e266b0 0x560979fd9200 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56097a033fc0 0x56097a0363d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f6050a98b80 1 --2- 172.21.15.8:0/3628231879 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x560979fcadb0 0x56097a02f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f6050a98b80 1 -- 172.21.15.8:0/3628231879 >> 172.21.15.8:0/3628231879 conn(0x560979fbff50 msgr2=0x560979fc9550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f601e7fc700 0 client.25559 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f6050a98b80 1 -- 172.21.15.8:0/3628231879 shutdown_connections 2023-11-27T15:28:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.068+0000 7f6050a98b80 1 -- 172.21.15.8:0/3628231879 wait complete. 2023-11-27T15:28:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f6050a98b80 20 client.25559 trim_cache size 0 max 16384 2023-11-27T15:28:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:28:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:28:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:28:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:28:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f603d1ff700 1 -- 172.21.15.8:0/2717338469 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60240a11d0 msgr2=0x7f60240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f603d1ff700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60240a11d0 0x7f60240a3680 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6030009ec0 tx=0x7f6030009310 comp rx=0 tx=0).stop 2023-11-27T15:28:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f603d1ff700 1 -- 172.21.15.8:0/2717338469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60380feab0 msgr2=0x7f603819c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f603d1ff700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60380feab0 0x7f603819c5f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6028006070 tx=0x7f6028004ce0 comp rx=0 tx=0).stop 2023-11-27T15:28:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f603d1ff700 1 -- 172.21.15.8:0/2717338469 shutdown_connections 2023-11-27T15:28:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f603d1ff700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6038103dd0 0x7f60381a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f603d1ff700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60380feab0 0x7f603819c5f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f603d1ff700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f60240a11d0 0x7f60240a3680 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f603d1ff700 1 --2- 172.21.15.8:0/2717338469 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6038101410 0x7f603819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.069+0000 7f603d1ff700 1 -- 172.21.15.8:0/2717338469 >> 172.21.15.8:0/2717338469 conn(0x7f60380fa720 msgr2=0x7f60381028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.070+0000 7f603d1ff700 1 -- 172.21.15.8:0/2717338469 shutdown_connections 2023-11-27T15:28:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:14.070+0000 7f603d1ff700 1 -- 172.21.15.8:0/2717338469 wait complete. 2023-11-27T15:28:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:15 smithi060 ceph-mon[131518]: pgmap v430: 129 pgs: 129 active+clean; 31 MiB data, 239 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2023-11-27T15:28:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:15 smithi008 ceph-mon[128788]: pgmap v430: 129 pgs: 129 active+clean; 31 MiB data, 239 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2023-11-27T15:28:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:15 smithi029 ceph-mon[131166]: pgmap v430: 129 pgs: 129 active+clean; 31 MiB data, 239 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2023-11-27T15:28:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:17 smithi060 ceph-mon[131518]: pgmap v431: 129 pgs: 129 active+clean; 42 MiB data, 278 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2023-11-27T15:28:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:17 smithi008 ceph-mon[128788]: pgmap v431: 129 pgs: 129 active+clean; 42 MiB data, 278 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2023-11-27T15:28:17.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:17 smithi029 ceph-mon[131166]: pgmap v431: 129 pgs: 129 active+clean; 42 MiB data, 278 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2023-11-27T15:28:19.574 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:19 smithi060 ceph-mon[131518]: pgmap v432: 129 pgs: 129 active+clean; 39 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 423 op/s 2023-11-27T15:28:19.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:19 smithi008 ceph-mon[128788]: pgmap v432: 129 pgs: 129 active+clean; 39 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 423 op/s 2023-11-27T15:28:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:19 smithi029 ceph-mon[131166]: pgmap v432: 129 pgs: 129 active+clean; 39 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 423 op/s 2023-11-27T15:28:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:21 smithi060 ceph-mon[131518]: pgmap v433: 129 pgs: 129 active+clean; 41 MiB data, 298 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 425 op/s 2023-11-27T15:28:21.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:21 smithi008 ceph-mon[128788]: pgmap v433: 129 pgs: 129 active+clean; 41 MiB data, 298 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 425 op/s 2023-11-27T15:28:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:21 smithi029 ceph-mon[131166]: pgmap v433: 129 pgs: 129 active+clean; 41 MiB data, 298 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 425 op/s 2023-11-27T15:28:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:23 smithi060 ceph-mon[131518]: pgmap v434: 129 pgs: 129 active+clean; 41 MiB data, 316 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 436 op/s 2023-11-27T15:28:23.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:23 smithi008 ceph-mon[128788]: pgmap v434: 129 pgs: 129 active+clean; 41 MiB data, 316 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 436 op/s 2023-11-27T15:28:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:23 smithi029 ceph-mon[131166]: pgmap v434: 129 pgs: 129 active+clean; 41 MiB data, 316 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 436 op/s 2023-11-27T15:28:25.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:25 smithi060 ceph-mon[131518]: pgmap v435: 129 pgs: 129 active+clean; 49 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 446 op/s 2023-11-27T15:28:25.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:25 smithi008 ceph-mon[128788]: pgmap v435: 129 pgs: 129 active+clean; 49 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 446 op/s 2023-11-27T15:28:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:25 smithi029 ceph-mon[131166]: pgmap v435: 129 pgs: 129 active+clean; 49 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 446 op/s 2023-11-27T15:28:27.256 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.252+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/390139105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e81035e0 msgr2=0x7fe4e81059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.252+0000 7fe4ee5ff700 1 --2- 172.21.15.8:0/390139105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e81035e0 0x7fe4e81059c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0098f0 tx=0x7fe4dc009c00 comp rx=0 tx=0).stop 2023-11-27T15:28:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.252+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/390139105 shutdown_connections 2023-11-27T15:28:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.252+0000 7fe4ee5ff700 1 --2- 172.21.15.8:0/390139105 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4e8100cc0 0x7fe4e81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.252+0000 7fe4ee5ff700 1 --2- 172.21.15.8:0/390139105 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e8105f00 0x7fe4e8108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.252+0000 7fe4ee5ff700 1 --2- 172.21.15.8:0/390139105 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e81035e0 0x7fe4e81059c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.252+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/390139105 >> 172.21.15.8:0/390139105 conn(0x7fe4e80fa720 msgr2=0x7fe4e80fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.252+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/390139105 shutdown_connections 2023-11-27T15:28:27.257 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.252+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/390139105 wait complete. 2023-11-27T15:28:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.253+0000 7fe4ee5ff700 1 Processor -- start 2023-11-27T15:28:27.258 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4ee5ff700 1 -- start start 2023-11-27T15:28:27.259 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4ee5ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e8100cc0 0x7fe4e8198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4ee5ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4e81035e0 0x7fe4e8198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4ee5ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e8105f00 0x7fe4e819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4ee5ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e819fdb0 con 0x7fe4e8105f00 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4ee5ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e819fef0 con 0x7fe4e8100cc0 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4ee5ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e81a0080 con 0x7fe4e81035e0 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4e7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e8100cc0 0x7fe4e8198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4e7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e8100cc0 0x7fe4e8198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:59750/0 (socket says 172.21.15.8:59750) 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4e7fff700 1 -- 172.21.15.8:0/3341339361 learned_addr learned my addr 172.21.15.8:0/3341339361 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4e77fe700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4e81035e0 0x7fe4e8198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.254+0000 7fe4ecb9c700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e8105f00 0x7fe4e819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4e7fff700 1 -- 172.21.15.8:0/3341339361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4e81035e0 msgr2=0x7fe4e8198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4e81035e0 0x7fe4e8198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4e7fff700 1 -- 172.21.15.8:0/3341339361 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e8105f00 msgr2=0x7fe4e819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e8105f00 0x7fe4e819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4e7fff700 1 -- 172.21.15.8:0/3341339361 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc009520 con 0x7fe4e8100cc0 2023-11-27T15:28:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e8100cc0 0x7fe4e8198230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800e9c0 tx=0x7fe4d800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4e57fa700 1 -- 172.21.15.8:0/3341339361 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d800cbd0 con 0x7fe4e8100cc0 2023-11-27T15:28:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4e57fa700 1 -- 172.21.15.8:0/3341339361 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d800eeb0 con 0x7fe4e8100cc0 2023-11-27T15:28:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4e57fa700 1 -- 172.21.15.8:0/3341339361 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8005580 con 0x7fe4e8100cc0 2023-11-27T15:28:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/3341339361 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e810bc50 con 0x7fe4e8100cc0 2023-11-27T15:28:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.255+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/3341339361 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e810bac0 con 0x7fe4e8100cc0 2023-11-27T15:28:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.259+0000 7fe4e57fa700 1 -- 172.21.15.8:0/3341339361 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe4d8004750 con 0x7fe4e8100cc0 2023-11-27T15:28:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.260+0000 7fe501e81b80 1 Processor -- start 2023-11-27T15:28:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.261+0000 7fe501e81b80 1 -- start start 2023-11-27T15:28:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.261+0000 7fe4e57fa700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4d00a11d0 0x7fe4d00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.261+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 0x559192705870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.261+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x5591927065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.261+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192706f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.261+0000 7fe501e81b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919260b000 con 0x5591925616b0 2023-11-27T15:28:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.261+0000 7fe501e81b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919265d840 con 0x559192705db0 2023-11-27T15:28:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.261+0000 7fe501e81b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559192656cd0 con 0x559192706b10 2023-11-27T15:28:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.261+0000 7fe4e57fa700 1 -- 172.21.15.8:0/3341339361 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe4d8014070 con 0x7fe4e8100cc0 2023-11-27T15:28:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.261+0000 7fe4ecb9c700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4d00a11d0 0x7fe4d00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.262+0000 7fe4ecb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192706f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.262+0000 7fe4ecb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192706f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:42704/0 (socket says 172.21.15.8:42704) 2023-11-27T15:28:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.262+0000 7fe4ecb9c700 1 -- 172.21.15.8:0/3337680421 learned_addr learned my addr 172.21.15.8:0/3337680421 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.262+0000 7fe4e77fe700 1 --2- 172.21.15.8:0/3337680421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x5591927065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.262+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/3337680421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 0x559192705870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.262+0000 7fe4e77fe700 1 -- 172.21.15.8:0/3337680421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 msgr2=0x559192706f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.262+0000 7fe4e77fe700 1 --2- 172.21.15.8:0/3337680421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192706f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.262+0000 7fe4e77fe700 1 -- 172.21.15.8:0/3337680421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 msgr2=0x559192705870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.262+0000 7fe4e77fe700 1 --2- 172.21.15.8:0/3337680421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 0x559192705870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.262+0000 7fe4e77fe700 1 -- 172.21.15.8:0/3337680421 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591924d85d0 con 0x559192705db0 2023-11-27T15:28:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.263+0000 7fe4ecb9c700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4d00a11d0 0x7fe4d00a3680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009550 tx=0x7fe4e0009cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.263+0000 7fe4e77fe700 1 --2- 172.21.15.8:0/3337680421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x5591927065d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009230 tx=0x7fe4dc005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f55d4cbbab40302a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.263+0000 7fe4ce7fc700 1 -- 172.21.15.8:0/3337680421 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc00f720 con 0x559192705db0 2023-11-27T15:28:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.263+0000 7fe4ce7fc700 1 -- 172.21.15.8:0/3337680421 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4dc024e10 con 0x559192705db0 2023-11-27T15:28:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.263+0000 7fe4ce7fc700 1 -- 172.21.15.8:0/3337680421 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc005040 con 0x559192705db0 2023-11-27T15:28:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.264+0000 7fe501e81b80 1 -- 172.21.15.8:0/3337680421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 msgr2=0x5591927065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.264+0000 7fe501e81b80 1 --2- 172.21.15.8:0/3337680421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x5591927065d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009230 tx=0x7fe4dc005dc0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.264+0000 7fe501e81b80 1 -- 172.21.15.8:0/3337680421 shutdown_connections 2023-11-27T15:28:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.264+0000 7fe501e81b80 1 --2- 172.21.15.8:0/3337680421 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192706f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.264+0000 7fe501e81b80 1 --2- 172.21.15.8:0/3337680421 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 0x559192705870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.264+0000 7fe501e81b80 1 --2- 172.21.15.8:0/3337680421 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x5591927065d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.264+0000 7fe501e81b80 1 -- 172.21.15.8:0/3337680421 >> 172.21.15.8:0/3337680421 conn(0x5591926faf50 msgr2=0x559192700100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.264+0000 7fe501e81b80 1 -- 172.21.15.8:0/3337680421 shutdown_connections 2023-11-27T15:28:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.264+0000 7fe501e81b80 1 -- 172.21.15.8:0/3337680421 wait complete. 2023-11-27T15:28:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.265+0000 7fe501e81b80 1 -- start start 2023-11-27T15:28:27.271 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.265+0000 7fe501e81b80 1 Processor -- start 2023-11-27T15:28:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.266+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591925616b0 0x55919270ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.266+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x55919270f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.266+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559192706b10 0x5591927161d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.266+0000 7fe501e81b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591925f8880 con 0x559192706b10 2023-11-27T15:28:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.266+0000 7fe501e81b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591925f8520 con 0x559192705db0 2023-11-27T15:28:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.266+0000 7fe501e81b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591924da8c0 con 0x5591925616b0 2023-11-27T15:28:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe501e81b80 10 client.0 _ll_register_callbacks cb 0x559192616720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe4e7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591925616b0 0x55919270ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe4e7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591925616b0 0x55919270ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:42706/0 (socket says 172.21.15.8:42706) 2023-11-27T15:28:27.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe4e7fff700 1 -- 172.21.15.8:0/4010745842 learned_addr learned my addr 172.21.15.8:0/4010745842 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe4e7fff700 1 -- 172.21.15.8:0/4010745842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 msgr2=0x55919270f190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:28:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x55919270f190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe4e7fff700 1 -- 172.21.15.8:0/4010745842 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559192706b10 msgr2=0x5591927161d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:28:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559192706b10 0x5591927161d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe4e7fff700 1 -- 172.21.15.8:0/4010745842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e8108f50 con 0x5591925616b0 2023-11-27T15:28:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591925616b0 0x55919270ec50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8099dd0 tx=0x7fe4d809de70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25ca082a4e948433 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.267+0000 7fe4c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:28:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.268+0000 7fe4c77fe700 1 -- 172.21.15.8:0/4010745842 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8099330 con 0x5591925616b0 2023-11-27T15:28:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.268+0000 7fe4c77fe700 1 -- 172.21.15.8:0/4010745842 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe4d8099490 con 0x5591925616b0 2023-11-27T15:28:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.268+0000 7fe501e81b80 1 -- 172.21.15.8:0/4010745842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591926f8710 con 0x5591925616b0 2023-11-27T15:28:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.268+0000 7fe4c77fe700 1 -- 172.21.15.8:0/4010745842 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d80b8250 con 0x5591925616b0 2023-11-27T15:28:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.268+0000 7fe501e81b80 10 client.25585 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.268+0000 7fe501e81b80 1 -- 172.21.15.8:0/4010745842 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55919261d130 con 0x5591925616b0 2023-11-27T15:28:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.269+0000 7fe4c77fe700 1 -- 172.21.15.8:0/4010745842 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d80c8070 con 0x5591925616b0 2023-11-27T15:28:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.269+0000 7fe4c77fe700 1 -- 172.21.15.8:0/4010745842 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe4d8095c30 con 0x5591925616b0 2023-11-27T15:28:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.269+0000 7fe501e81b80 10 client.25585 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.269+0000 7fe501e81b80 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:28:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.269+0000 7fe501e81b80 20 client.25585 populate_metadata read hostname 'smithi008' 2023-11-27T15:28:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.269+0000 7fe501e81b80 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55919271ab30 0x55919271cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.269+0000 7fe501e81b80 4 client.25585 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.269+0000 7fe501e81b80 1 -- 172.21.15.8:0/4010745842 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559192619040 con 0x55919271ab30 2023-11-27T15:28:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.270+0000 7fe4e77fe700 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55919271ab30 0x55919271cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.271+0000 7fe4c77fe700 10 client.25585 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.271+0000 7fe4e77fe700 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55919271ab30 0x55919271cf40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1360a2508cfe724c server_cookie=4a870a552553ce9d in_seq=0 out_seq=0 2023-11-27T15:28:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.272+0000 7fe4c77fe700 1 -- 172.21.15.8:0/4010745842 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559192619040 con 0x55919271ab30 2023-11-27T15:28:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.272+0000 7fe4c77fe700 10 client.25585 handle_command_reply: tid=0 2023-11-27T15:28:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.272+0000 7fe501e81b80 1 client.25585 shutdown 2023-11-27T15:28:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.273+0000 7fe501e81b80 1 -- 172.21.15.8:0/4010745842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591925616b0 msgr2=0x55919270ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.273+0000 7fe501e81b80 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591925616b0 0x55919270ec50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8099dd0 tx=0x7fe4d809de70 comp rx=0 tx=0).stop 2023-11-27T15:28:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.273+0000 7fe501e81b80 1 -- 172.21.15.8:0/4010745842 shutdown_connections 2023-11-27T15:28:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.273+0000 7fe501e81b80 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55919271ab30 0x55919271cf40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.273+0000 7fe501e81b80 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5591925616b0 0x55919270ec50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.273+0000 7fe501e81b80 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559192706b10 0x5591927161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.273+0000 7fe501e81b80 1 --2- 172.21.15.8:0/4010745842 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x55919270f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.273+0000 7fe501e81b80 1 -- 172.21.15.8:0/4010745842 >> 172.21.15.8:0/4010745842 conn(0x5591926faf50 msgr2=0x5591926ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.273+0000 7fe4c77fe700 0 client.25585 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.274+0000 7fe501e81b80 1 -- 172.21.15.8:0/4010745842 shutdown_connections 2023-11-27T15:28:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.274+0000 7fe501e81b80 1 -- 172.21.15.8:0/4010745842 wait complete. 2023-11-27T15:28:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.275+0000 7fe501e81b80 20 client.25585 trim_cache size 0 max 16384 2023-11-27T15:28:27.297 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.293+0000 7fe501e81b80 1 Processor -- start 2023-11-27T15:28:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.293+0000 7fe501e81b80 1 -- start start 2023-11-27T15:28:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.293+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591925616b0 0x55919270d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.293+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192705db0 0x559192716190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.293+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559192706b10 0x55919270cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.293+0000 7fe501e81b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919270cce0 con 0x559192706b10 2023-11-27T15:28:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.293+0000 7fe501e81b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919270a5d0 con 0x5591925616b0 2023-11-27T15:28:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.293+0000 7fe501e81b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559192700dc0 con 0x559192705db0 2023-11-27T15:28:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4e7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591925616b0 0x55919270d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4e7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591925616b0 0x55919270d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:59762/0 (socket says 172.21.15.8:59762) 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4e7fff700 1 -- 172.21.15.8:0/3978685909 learned_addr learned my addr 172.21.15.8:0/3978685909 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4e77fe700 1 --2- 172.21.15.8:0/3978685909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192705db0 0x559192716190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4e7fff700 1 -- 172.21.15.8:0/3978685909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192705db0 msgr2=0x559192716190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4ecb9c700 1 --2- 172.21.15.8:0/3978685909 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559192706b10 0x55919270cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/3978685909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192705db0 0x559192716190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4e7fff700 1 -- 172.21.15.8:0/3978685909 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559192706b10 msgr2=0x55919270cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/3978685909 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559192706b10 0x55919270cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4e7fff700 1 -- 172.21.15.8:0/3978685909 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55919261ce80 con 0x5591925616b0 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4ecb9c700 1 --2- 172.21.15.8:0/3978685909 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559192706b10 0x55919270cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.294+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/3978685909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591925616b0 0x55919270d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009590 tx=0x7fe4d809a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe13376946ecd7c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.295+0000 7fe4ceffd700 1 -- 172.21.15.8:0/3978685909 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80ca2e0 con 0x5591925616b0 2023-11-27T15:28:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.295+0000 7fe4ceffd700 1 -- 172.21.15.8:0/3978685909 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4d80c8070 con 0x5591925616b0 2023-11-27T15:28:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.295+0000 7fe4ceffd700 1 -- 172.21.15.8:0/3978685909 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8099330 con 0x5591925616b0 2023-11-27T15:28:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.295+0000 7fe501e81b80 1 -- 172.21.15.8:0/3978685909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591925616b0 msgr2=0x55919270d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:27.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.295+0000 7fe501e81b80 1 --2- 172.21.15.8:0/3978685909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591925616b0 0x55919270d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009590 tx=0x7fe4d809a380 comp rx=0 tx=0).stop 2023-11-27T15:28:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.295+0000 7fe501e81b80 1 -- 172.21.15.8:0/3978685909 shutdown_connections 2023-11-27T15:28:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.295+0000 7fe501e81b80 1 --2- 172.21.15.8:0/3978685909 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559192706b10 0x55919270cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.295+0000 7fe501e81b80 1 --2- 172.21.15.8:0/3978685909 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5591925616b0 0x55919270d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.295+0000 7fe501e81b80 1 --2- 172.21.15.8:0/3978685909 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192705db0 0x559192716190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.296+0000 7fe501e81b80 1 -- 172.21.15.8:0/3978685909 >> 172.21.15.8:0/3978685909 conn(0x5591926faf50 msgr2=0x5591927218a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.296+0000 7fe501e81b80 1 -- 172.21.15.8:0/3978685909 shutdown_connections 2023-11-27T15:28:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.296+0000 7fe501e81b80 1 -- 172.21.15.8:0/3978685909 wait complete. 2023-11-27T15:28:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.296+0000 7fe501e81b80 1 -- start start 2023-11-27T15:28:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.296+0000 7fe501e81b80 1 Processor -- start 2023-11-27T15:28:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 0x559192711be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x559192712120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe501e81b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192715140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe501e81b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559192715680 con 0x5591925616b0 2023-11-27T15:28:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe501e81b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591927157f0 con 0x559192705db0 2023-11-27T15:28:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe501e81b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591927159b0 con 0x559192706b10 2023-11-27T15:28:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe501e81b80 10 client.0 _ll_register_callbacks cb 0x7fe4d800cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe4ecb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192715140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe4e77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x559192712120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe4ecb9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192715140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:42732/0 (socket says 172.21.15.8:42732) 2023-11-27T15:28:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.297+0000 7fe4ecb9c700 1 -- 172.21.15.8:0/1822577585 learned_addr learned my addr 172.21.15.8:0/1822577585 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.298+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 0x559192711be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.298+0000 7fe4ecb9c700 1 -- 172.21.15.8:0/1822577585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 msgr2=0x559192712120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.298+0000 7fe4ecb9c700 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x559192712120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.298+0000 7fe4ecb9c700 1 -- 172.21.15.8:0/1822577585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 msgr2=0x559192711be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.298+0000 7fe4ecb9c700 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 0x559192711be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.298+0000 7fe4ecb9c700 1 -- 172.21.15.8:0/1822577585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55919261d130 con 0x559192706b10 2023-11-27T15:28:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.298+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 0x559192711be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.298+0000 7fe4ecb9c700 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192715140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00078f0 tx=0x7fe4e001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b138495687cf397a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.298+0000 7fe4c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:28:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.299+0000 7fe4c77fe700 1 -- 172.21.15.8:0/1822577585 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e001be20 con 0x559192706b10 2023-11-27T15:28:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.299+0000 7fe4c77fe700 1 -- 172.21.15.8:0/1822577585 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe4e0023050 con 0x559192706b10 2023-11-27T15:28:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.299+0000 7fe501e81b80 1 -- 172.21.15.8:0/1822577585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4dc024e10 con 0x559192706b10 2023-11-27T15:28:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.299+0000 7fe4c77fe700 1 -- 172.21.15.8:0/1822577585 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4e0009a20 con 0x559192706b10 2023-11-27T15:28:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.299+0000 7fe501e81b80 10 client.25591 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.299+0000 7fe501e81b80 1 -- 172.21.15.8:0/1822577585 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55919246dc00 con 0x559192706b10 2023-11-27T15:28:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.299+0000 7fe4c77fe700 1 -- 172.21.15.8:0/1822577585 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4dc024e10 con 0x559192706b10 2023-11-27T15:28:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.300+0000 7fe4c77fe700 1 -- 172.21.15.8:0/1822577585 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe4e0025070 con 0x559192706b10 2023-11-27T15:28:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.300+0000 7fe501e81b80 10 client.25591 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.300+0000 7fe501e81b80 10 client.25591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:28:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.300+0000 7fe501e81b80 20 client.25591 populate_metadata read hostname 'smithi008' 2023-11-27T15:28:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.300+0000 7fe501e81b80 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55919276f720 0x559192771b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.300+0000 7fe501e81b80 4 client.25591 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:28:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.300+0000 7fe501e81b80 1 -- 172.21.15.8:0/1822577585 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559192625600 con 0x55919276f720 2023-11-27T15:28:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.301+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55919276f720 0x559192771b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.302+0000 7fe4c77fe700 10 client.25591 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:27.302+0000 7fe4e7fff700 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55919276f720 0x559192771b30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7dc913724147c65 server_cookie=a3eb71305cfcfd34 in_seq=0 out_seq=0 2023-11-27T15:28:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:27 smithi060 ceph-mon[131518]: pgmap v436: 129 pgs: 129 active+clean; 56 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2023-11-27T15:28:27.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:27 smithi008 ceph-mon[128788]: pgmap v436: 129 pgs: 129 active+clean; 56 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2023-11-27T15:28:27.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:27 smithi029 ceph-mon[131166]: pgmap v436: 129 pgs: 129 active+clean; 56 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2023-11-27T15:28:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:29 smithi008 ceph-mon[128788]: pgmap v437: 129 pgs: 129 active+clean; 62 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2023-11-27T15:28:29.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:29 smithi029 ceph-mon[131166]: pgmap v437: 129 pgs: 129 active+clean; 62 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2023-11-27T15:28:29.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:29 smithi060 ceph-mon[131518]: pgmap v437: 129 pgs: 129 active+clean; 62 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2023-11-27T15:28:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:31 smithi060 ceph-mon[131518]: pgmap v438: 129 pgs: 129 active+clean; 62 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 138 op/s 2023-11-27T15:28:31.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:31 smithi008 ceph-mon[128788]: pgmap v438: 129 pgs: 129 active+clean; 62 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 138 op/s 2023-11-27T15:28:31.901 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:31 smithi029 ceph-mon[131166]: pgmap v438: 129 pgs: 129 active+clean; 62 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 138 op/s 2023-11-27T15:28:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.360+0000 7fe4c77fe700 1 -- 172.21.15.8:0/1822577585 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559192625600 con 0x55919276f720 2023-11-27T15:28:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.360+0000 7fe4c77fe700 10 client.25591 handle_command_reply: tid=0 2023-11-27T15:28:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.360+0000 7fe501e81b80 1 client.25591 shutdown 2023-11-27T15:28:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.361+0000 7fe501e81b80 1 -- 172.21.15.8:0/1822577585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 msgr2=0x559192715140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.361+0000 7fe501e81b80 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192715140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00078f0 tx=0x7fe4e001b000 comp rx=0 tx=0).stop 2023-11-27T15:28:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.361+0000 7fe501e81b80 1 -- 172.21.15.8:0/1822577585 shutdown_connections 2023-11-27T15:28:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.361+0000 7fe501e81b80 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55919276f720 0x559192771b30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.361+0000 7fe4c77fe700 0 client.25591 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.361+0000 7fe501e81b80 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559192706b10 0x559192715140 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.362+0000 7fe501e81b80 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5591925616b0 0x559192711be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.362+0000 7fe501e81b80 1 --2- 172.21.15.8:0/1822577585 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559192705db0 0x559192712120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.362+0000 7fe501e81b80 1 -- 172.21.15.8:0/1822577585 >> 172.21.15.8:0/1822577585 conn(0x5591926faf50 msgr2=0x55919270d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.363+0000 7fe501e81b80 1 -- 172.21.15.8:0/1822577585 shutdown_connections 2023-11-27T15:28:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.363+0000 7fe501e81b80 1 -- 172.21.15.8:0/1822577585 wait complete. 2023-11-27T15:28:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.363+0000 7fe501e81b80 20 client.25591 trim_cache size 0 max 16384 2023-11-27T15:28:33.367 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:28:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:28:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:28:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:28:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.364+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/3341339361 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4d00a11d0 msgr2=0x7fe4d00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.364+0000 7fe4ee5ff700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4d00a11d0 0x7fe4d00a3680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009550 tx=0x7fe4e0009cd0 comp rx=0 tx=0).stop 2023-11-27T15:28:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.364+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/3341339361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e8100cc0 msgr2=0x7fe4e8198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.364+0000 7fe4ee5ff700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e8100cc0 0x7fe4e8198230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800e9c0 tx=0x7fe4d800ecd0 comp rx=0 tx=0).stop 2023-11-27T15:28:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.364+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/3341339361 shutdown_connections 2023-11-27T15:28:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.364+0000 7fe4ee5ff700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe4e8105f00 0x7fe4e819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.364+0000 7fe4ee5ff700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe4e8100cc0 0x7fe4e8198230 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.364+0000 7fe4ee5ff700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe4d00a11d0 0x7fe4d00a3680 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.364+0000 7fe4ee5ff700 1 --2- 172.21.15.8:0/3341339361 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe4e81035e0 0x7fe4e8198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.365+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/3341339361 >> 172.21.15.8:0/3341339361 conn(0x7fe4e80fa720 msgr2=0x7fe4e80ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.365+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/3341339361 shutdown_connections 2023-11-27T15:28:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:33.365+0000 7fe4ee5ff700 1 -- 172.21.15.8:0/3341339361 wait complete. 2023-11-27T15:28:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:33 smithi060 ceph-mon[131518]: pgmap v439: 129 pgs: 129 active+clean; 70 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 1.10k op/s 2023-11-27T15:28:33.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:33 smithi008 ceph-mon[128788]: pgmap v439: 129 pgs: 129 active+clean; 70 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 1.10k op/s 2023-11-27T15:28:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:33 smithi029 ceph-mon[131166]: pgmap v439: 129 pgs: 129 active+clean; 70 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 1.10k op/s 2023-11-27T15:28:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:35 smithi060 ceph-mon[131518]: pgmap v440: 129 pgs: 129 active+clean; 71 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 1.67k op/s 2023-11-27T15:28:35.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:35 smithi008 ceph-mon[128788]: pgmap v440: 129 pgs: 129 active+clean; 71 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 1.67k op/s 2023-11-27T15:28:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:35 smithi029 ceph-mon[131166]: pgmap v440: 129 pgs: 129 active+clean; 71 MiB data, 532 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 1.67k op/s 2023-11-27T15:28:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:37 smithi060 ceph-mon[131518]: pgmap v441: 129 pgs: 129 active+clean; 68 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 1.98k op/s 2023-11-27T15:28:37.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:37 smithi008 ceph-mon[128788]: pgmap v441: 129 pgs: 129 active+clean; 68 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 1.98k op/s 2023-11-27T15:28:37.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:37 smithi029 ceph-mon[131166]: pgmap v441: 129 pgs: 129 active+clean; 68 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 1.98k op/s 2023-11-27T15:28:39.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:39 smithi008 ceph-mon[128788]: pgmap v442: 129 pgs: 129 active+clean; 73 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 2.16k op/s 2023-11-27T15:28:39.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:39 smithi029 ceph-mon[131166]: pgmap v442: 129 pgs: 129 active+clean; 73 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 2.16k op/s 2023-11-27T15:28:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:39 smithi060 ceph-mon[131518]: pgmap v442: 129 pgs: 129 active+clean; 73 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 2.16k op/s 2023-11-27T15:28:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:41 smithi060 ceph-mon[131518]: pgmap v443: 129 pgs: 129 active+clean; 76 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 2.15k op/s 2023-11-27T15:28:41.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:41 smithi008 ceph-mon[128788]: pgmap v443: 129 pgs: 129 active+clean; 76 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 2.15k op/s 2023-11-27T15:28:41.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:41 smithi029 ceph-mon[131166]: pgmap v443: 129 pgs: 129 active+clean; 76 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 2.15k op/s 2023-11-27T15:28:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:43 smithi060 ceph-mon[131518]: pgmap v444: 129 pgs: 129 active+clean; 88 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 2.15k op/s 2023-11-27T15:28:43.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:43 smithi008 ceph-mon[128788]: pgmap v444: 129 pgs: 129 active+clean; 88 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 2.15k op/s 2023-11-27T15:28:43.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:43 smithi029 ceph-mon[131166]: pgmap v444: 129 pgs: 129 active+clean; 88 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.2 MiB/s wr, 2.15k op/s 2023-11-27T15:28:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:45 smithi060 ceph-mon[131518]: pgmap v445: 129 pgs: 129 active+clean; 88 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 1.21k op/s 2023-11-27T15:28:45.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:45 smithi008 ceph-mon[128788]: pgmap v445: 129 pgs: 129 active+clean; 88 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 1.21k op/s 2023-11-27T15:28:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:45 smithi029 ceph-mon[131166]: pgmap v445: 129 pgs: 129 active+clean; 88 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 1.21k op/s 2023-11-27T15:28:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.582+0000 7f9e289fe700 1 -- 172.21.15.8:0/2448095327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e24103c80 msgr2=0x7f9e24104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.582+0000 7f9e289fe700 1 --2- 172.21.15.8:0/2448095327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e24103c80 0x7f9e24104100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009f90 tx=0x7f9e1c00b330 comp rx=0 tx=0).stop 2023-11-27T15:28:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.582+0000 7f9e289fe700 1 -- 172.21.15.8:0/2448095327 shutdown_connections 2023-11-27T15:28:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.582+0000 7f9e289fe700 1 --2- 172.21.15.8:0/2448095327 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e24103c80 0x7f9e24104100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.582+0000 7f9e289fe700 1 --2- 172.21.15.8:0/2448095327 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e240fe960 0x7f9e24100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.582+0000 7f9e289fe700 1 --2- 172.21.15.8:0/2448095327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e241012c0 0x7f9e24103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.582+0000 7f9e289fe700 1 -- 172.21.15.8:0/2448095327 >> 172.21.15.8:0/2448095327 conn(0x7f9e240fa550 msgr2=0x7f9e240fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.582+0000 7f9e289fe700 1 -- 172.21.15.8:0/2448095327 shutdown_connections 2023-11-27T15:28:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.582+0000 7f9e289fe700 1 -- 172.21.15.8:0/2448095327 wait complete. 2023-11-27T15:28:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.584+0000 7f9e289fe700 1 Processor -- start 2023-11-27T15:28:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.584+0000 7f9e289fe700 1 -- start start 2023-11-27T15:28:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.584+0000 7f9e289fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e240fe960 0x7f9e2419c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.585+0000 7f9e289fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e241012c0 0x7f9e2419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.585+0000 7f9e289fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24103c80 0x7f9e241a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.585+0000 7f9e289fe700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241a4060 con 0x7f9e24103c80 2023-11-27T15:28:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.585+0000 7f9e289fe700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241a41a0 con 0x7f9e241012c0 2023-11-27T15:28:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.585+0000 7f9e289fe700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241a4330 con 0x7f9e240fe960 2023-11-27T15:28:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.585+0000 7f9e22d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e240fe960 0x7f9e2419c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.585+0000 7f9e2259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e241012c0 0x7f9e2419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.585+0000 7f9e2259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e241012c0 0x7f9e2419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:57624/0 (socket says 172.21.15.8:57624) 2023-11-27T15:28:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.585+0000 7f9e2259c700 1 -- 172.21.15.8:0/2121521429 learned_addr learned my addr 172.21.15.8:0/2121521429 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.585+0000 7f9e2359e700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24103c80 0x7f9e241a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.586+0000 7f9e2259c700 1 -- 172.21.15.8:0/2121521429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e240fe960 msgr2=0x7f9e2419c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.586+0000 7f9e2259c700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e240fe960 0x7f9e2419c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.586+0000 7f9e2259c700 1 -- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24103c80 msgr2=0x7f9e241a3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.586+0000 7f9e2259c700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24103c80 0x7f9e241a3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.586+0000 7f9e2259c700 1 -- 172.21.15.8:0/2121521429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c009c40 con 0x7f9e241012c0 2023-11-27T15:28:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.586+0000 7f9e2359e700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24103c80 0x7f9e241a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.586+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e240fe960 0x7f9e2419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.586+0000 7f9e2259c700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e241012c0 0x7f9e2419ca20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800d950 tx=0x7f9e1800dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.587+0000 7f9e0bfff700 1 -- 172.21.15.8:0/2121521429 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18009b40 con 0x7f9e241012c0 2023-11-27T15:28:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.587+0000 7f9e0bfff700 1 -- 172.21.15.8:0/2121521429 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e18009ca0 con 0x7f9e241012c0 2023-11-27T15:28:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.587+0000 7f9e0bfff700 1 -- 172.21.15.8:0/2121521429 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1800f620 con 0x7f9e241012c0 2023-11-27T15:28:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.587+0000 7f9e289fe700 1 -- 172.21.15.8:0/2121521429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e24112010 con 0x7f9e241012c0 2023-11-27T15:28:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.587+0000 7f9e289fe700 1 -- 172.21.15.8:0/2121521429 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e24111e80 con 0x7f9e241012c0 2023-11-27T15:28:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.592+0000 7f9e0bfff700 1 -- 172.21.15.8:0/2121521429 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9e18010460 con 0x7f9e241012c0 2023-11-27T15:28:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.593+0000 7f9e3ca8db80 1 Processor -- start 2023-11-27T15:28:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.593+0000 7f9e3ca8db80 1 -- start start 2023-11-27T15:28:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.594+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819f86e6b0 0x55819fa12a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.597 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.594+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 0x55819fa13820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.594+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819fa13d60 0x55819fa14200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.594+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819f918000 con 0x55819fa12fb0 2023-11-27T15:28:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.594+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819f96a840 con 0x55819fa13d60 2023-11-27T15:28:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.594+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819f963cd0 con 0x55819f86e6b0 2023-11-27T15:28:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.594+0000 7f9e2359e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819f86e6b0 0x55819fa12a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.594+0000 7f9e0bfff700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e0c0a11d0 0x7f9e0c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.594+0000 7f9e2359e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819f86e6b0 0x55819fa12a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58254/0 (socket says 172.21.15.8:58254) 2023-11-27T15:28:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.594+0000 7f9e2359e700 1 -- 172.21.15.8:0/3225289447 learned_addr learned my addr 172.21.15.8:0/3225289447 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.595+0000 7f9e2259c700 1 --2- 172.21.15.8:0/3225289447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819fa13d60 0x55819fa14200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.595+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/3225289447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 0x55819fa13820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.595+0000 7f9e0bfff700 1 -- 172.21.15.8:0/2121521429 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9e180ccc80 con 0x7f9e241012c0 2023-11-27T15:28:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.595+0000 7f9e2359e700 1 -- 172.21.15.8:0/3225289447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819fa13d60 msgr2=0x55819fa14200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.595+0000 7f9e2359e700 1 --2- 172.21.15.8:0/3225289447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819fa13d60 0x55819fa14200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.595+0000 7f9e2359e700 1 -- 172.21.15.8:0/3225289447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 msgr2=0x55819fa13820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.595+0000 7f9e2359e700 1 --2- 172.21.15.8:0/3225289447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 0x55819fa13820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.595+0000 7f9e2359e700 1 -- 172.21.15.8:0/3225289447 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55819f7e55d0 con 0x55819f86e6b0 2023-11-27T15:28:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.595+0000 7f9e2359e700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e0c0a11d0 0x7f9e0c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.595+0000 7f9e2359e700 1 --2- 172.21.15.8:0/3225289447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819f86e6b0 0x55819fa12a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009470 tx=0x7f9e1c0094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27302ca17ac436ae server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.596+0000 7f9e097fa700 1 -- 172.21.15.8:0/3225289447 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00e070 con 0x55819f86e6b0 2023-11-27T15:28:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.596+0000 7f9e097fa700 1 -- 172.21.15.8:0/3225289447 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e1c00bbf0 con 0x55819f86e6b0 2023-11-27T15:28:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.596+0000 7f9e097fa700 1 -- 172.21.15.8:0/3225289447 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c01ddd0 con 0x55819f86e6b0 2023-11-27T15:28:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.597+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/3225289447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819f86e6b0 msgr2=0x55819fa12a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.597+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/3225289447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819f86e6b0 0x55819fa12a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009470 tx=0x7f9e1c0094a0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.597+0000 7f9e2359e700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e0c0a11d0 0x7f9e0c0a3680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c029000 tx=0x7f9e1c026000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.597+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/3225289447 shutdown_connections 2023-11-27T15:28:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.597+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/3225289447 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819f86e6b0 0x55819fa12a70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.604 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.597+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/3225289447 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819fa13d60 0x55819fa14200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.597+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/3225289447 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 0x55819fa13820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.597+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/3225289447 >> 172.21.15.8:0/3225289447 conn(0x55819fa081a0 msgr2=0x55819fa0d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.597+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/3225289447 shutdown_connections 2023-11-27T15:28:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.597+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/3225289447 wait complete. 2023-11-27T15:28:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.598+0000 7f9e3ca8db80 1 -- start start 2023-11-27T15:28:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.598+0000 7f9e3ca8db80 1 Processor -- start 2023-11-27T15:28:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.599+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.605 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.599+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 0x55819fa1c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.599+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 0x55819fa234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.599+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819f905880 con 0x55819fa13d60 2023-11-27T15:28:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.599+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819f905520 con 0x55819f86e6b0 2023-11-27T15:28:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.599+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819f7e78c0 con 0x55819fa12fb0 2023-11-27T15:28:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.599+0000 7f9e3ca8db80 10 client.0 _ll_register_callbacks cb 0x55819f923720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.599+0000 7f9e2259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 0x55819fa1c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.599+0000 7f9e2259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 0x55819fa1c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58262/0 (socket says 172.21.15.8:58262) 2023-11-27T15:28:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.599+0000 7f9e2259c700 1 -- 172.21.15.8:0/722004518 learned_addr learned my addr 172.21.15.8:0/722004518 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e2259c700 1 -- 172.21.15.8:0/722004518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 msgr2=0x55819fa1bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e2259c700 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e2259c700 1 -- 172.21.15.8:0/722004518 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 msgr2=0x55819fa234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e2359e700 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 0x55819fa234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e2259c700 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 0x55819fa234d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e2259c700 1 -- 172.21.15.8:0/722004518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e24104bf0 con 0x55819fa12fb0 2023-11-27T15:28:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:46.607 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e2359e700 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 0x55819fa234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e2259c700 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 0x55819fa1c490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e180967d0 tx=0x7f9e18021350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4b44b8f1f0e0c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:46.608 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.600+0000 7f9e06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:28:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.602+0000 7f9e06ff5700 1 -- 172.21.15.8:0/722004518 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e180d1040 con 0x55819fa12fb0 2023-11-27T15:28:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.602+0000 7f9e06ff5700 1 -- 172.21.15.8:0/722004518 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9e1809dd90 con 0x55819fa12fb0 2023-11-27T15:28:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.602+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/722004518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55819fa05960 con 0x55819fa12fb0 2023-11-27T15:28:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.602+0000 7f9e06ff5700 1 -- 172.21.15.8:0/722004518 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e180a21b0 con 0x55819fa12fb0 2023-11-27T15:28:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.602+0000 7f9e3ca8db80 10 client.25603 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.602+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/722004518 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55819f92a130 con 0x55819fa12fb0 2023-11-27T15:28:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.602+0000 7f9e06ff5700 1 -- 172.21.15.8:0/722004518 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e180b6070 con 0x55819fa12fb0 2023-11-27T15:28:46.611 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.603+0000 7f9e06ff5700 1 -- 172.21.15.8:0/722004518 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9e180a1d90 con 0x55819fa12fb0 2023-11-27T15:28:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.603+0000 7f9e3ca8db80 10 client.25603 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.603+0000 7f9e3ca8db80 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:28:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.603+0000 7f9e3ca8db80 20 client.25603 populate_metadata read hostname 'smithi008' 2023-11-27T15:28:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.603+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55819fa27cd0 0x55819fa2a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.603+0000 7f9e3ca8db80 4 client.25603 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:46.612 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.603+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/722004518 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55819f926040 con 0x55819fa27cd0 2023-11-27T15:28:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.603+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55819fa27cd0 0x55819fa2a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.604+0000 7f9e06ff5700 10 client.25603 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.605+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55819fa27cd0 0x55819fa2a0e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69f472eeb7afcd0 server_cookie=8aa9b1ed6b1c50ee in_seq=0 out_seq=0 2023-11-27T15:28:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e06ff5700 1 -- 172.21.15.8:0/722004518 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55819f926040 con 0x55819fa27cd0 2023-11-27T15:28:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e06ff5700 10 client.25603 handle_command_reply: tid=0 2023-11-27T15:28:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e3ca8db80 1 client.25603 shutdown 2023-11-27T15:28:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/722004518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 msgr2=0x55819fa1c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.613 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 0x55819fa1c490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e180967d0 tx=0x7f9e18021350 comp rx=0 tx=0).stop 2023-11-27T15:28:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/722004518 shutdown_connections 2023-11-27T15:28:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 0x55819fa234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55819fa27cd0 0x55819fa2a0e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/722004518 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 0x55819fa1c490 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.614 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/722004518 >> 172.21.15.8:0/722004518 conn(0x55819fa081a0 msgr2=0x55819fa0ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.606+0000 7f9e06ff5700 0 client.25603 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.607+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/722004518 shutdown_connections 2023-11-27T15:28:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.607+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/722004518 wait complete. 2023-11-27T15:28:46.615 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.607+0000 7f9e3ca8db80 20 client.25603 trim_cache size 0 max 16384 2023-11-27T15:28:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.633+0000 7f9e3ca8db80 1 Processor -- start 2023-11-27T15:28:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.633+0000 7f9e3ca8db80 1 -- start start 2023-11-27T15:28:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 0x55819fa23490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 0x55819fa1a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819fa1d140 con 0x55819fa13d60 2023-11-27T15:28:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819fa0e1c0 con 0x55819f86e6b0 2023-11-27T15:28:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819fa1b400 con 0x55819fa12fb0 2023-11-27T15:28:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e22d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e22d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:57648/0 (socket says 172.21.15.8:57648) 2023-11-27T15:28:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e22d9d700 1 -- 172.21.15.8:0/3985006162 learned_addr learned my addr 172.21.15.8:0/3985006162 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e2259c700 1 --2- 172.21.15.8:0/3985006162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 0x55819fa23490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.634+0000 7f9e2359e700 1 --2- 172.21.15.8:0/3985006162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 0x55819fa1a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.635+0000 7f9e22d9d700 1 -- 172.21.15.8:0/3985006162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 msgr2=0x55819fa23490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.635+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/3985006162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 0x55819fa23490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.635+0000 7f9e22d9d700 1 -- 172.21.15.8:0/3985006162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 msgr2=0x55819fa1a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.635+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/3985006162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 0x55819fa1a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.635+0000 7f9e22d9d700 1 -- 172.21.15.8:0/3985006162 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55819f929e80 con 0x55819f86e6b0 2023-11-27T15:28:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.635+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/3985006162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1ac70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e14004790 tx=0x7f9e1401dca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c76325d3607992 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.636+0000 7f9e09ffb700 1 -- 172.21.15.8:0/3985006162 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14017040 con 0x55819f86e6b0 2023-11-27T15:28:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.636+0000 7f9e09ffb700 1 -- 172.21.15.8:0/3985006162 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e14021070 con 0x55819f86e6b0 2023-11-27T15:28:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.636+0000 7f9e09ffb700 1 -- 172.21.15.8:0/3985006162 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14003be0 con 0x55819f86e6b0 2023-11-27T15:28:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.637+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/3985006162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 msgr2=0x55819fa1ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.637+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/3985006162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1ac70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e14004790 tx=0x7f9e1401dca0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.637+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/3985006162 shutdown_connections 2023-11-27T15:28:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.638+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/3985006162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa13d60 0x55819fa1a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.638+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/3985006162 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1ac70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.638+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/3985006162 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa12fb0 0x55819fa23490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.641 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.638+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/3985006162 >> 172.21.15.8:0/3985006162 conn(0x55819fa081a0 msgr2=0x55819fa2ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.638+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/3985006162 shutdown_connections 2023-11-27T15:28:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.638+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/3985006162 wait complete. 2023-11-27T15:28:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.638+0000 7f9e3ca8db80 1 -- start start 2023-11-27T15:28:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.639+0000 7f9e3ca8db80 1 Processor -- start 2023-11-27T15:28:46.642 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.639+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.639+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 0x55819fa1f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.643 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.639+0000 7f9e3ca8db80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa13d60 0x55819fa22480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.640+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819fa229c0 con 0x55819fa12fb0 2023-11-27T15:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.640+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819fa22b30 con 0x55819f86e6b0 2023-11-27T15:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.640+0000 7f9e3ca8db80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819fa22cf0 con 0x55819fa13d60 2023-11-27T15:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.640+0000 7f9e3ca8db80 10 client.0 _ll_register_callbacks cb 0x55819f923720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.640+0000 7f9e2359e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa13d60 0x55819fa22480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.640+0000 7f9e2359e700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa13d60 0x55819fa22480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58284/0 (socket says 172.21.15.8:58284) 2023-11-27T15:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.640+0000 7f9e2259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 0x55819fa1f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.644 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.640+0000 7f9e2259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 0x55819fa1f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58342/0 (socket says 172.21.15.8:58342) 2023-11-27T15:28:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.640+0000 7f9e2259c700 1 -- 172.21.15.8:0/2164025119 learned_addr learned my addr 172.21.15.8:0/2164025119 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:28:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.641+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.641+0000 7f9e2359e700 1 -- 172.21.15.8:0/2164025119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 msgr2=0x55819fa1ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.641+0000 7f9e2359e700 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.641+0000 7f9e2359e700 1 -- 172.21.15.8:0/2164025119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 msgr2=0x55819fa1f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.641+0000 7f9e2359e700 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 0x55819fa1f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.641+0000 7f9e2359e700 1 -- 172.21.15.8:0/2164025119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e14009a40 con 0x55819fa13d60 2023-11-27T15:28:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.642+0000 7f9e2259c700 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 0x55819fa1f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.642+0000 7f9e2359e700 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa13d60 0x55819fa22480 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0074d0 tx=0x7f9e1c00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be08e2ef7ab4f7d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.642+0000 7f9e06ff5700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:28:46.646 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.642+0000 7f9e06ff5700 1 -- 172.21.15.8:0/2164025119 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c039070 con 0x55819fa13d60 2023-11-27T15:28:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.642+0000 7f9e06ff5700 1 -- 172.21.15.8:0/2164025119 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9e1c00b9f0 con 0x55819fa13d60 2023-11-27T15:28:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.643+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/2164025119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e1c00bbf0 con 0x55819fa13d60 2023-11-27T15:28:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.643+0000 7f9e06ff5700 1 -- 172.21.15.8:0/2164025119 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e1c00e040 con 0x55819fa13d60 2023-11-27T15:28:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.643+0000 7f9e3ca8db80 10 client.25609 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.643+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/2164025119 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55819f77ac00 con 0x55819fa13d60 2023-11-27T15:28:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.643+0000 7f9e06ff5700 1 -- 172.21.15.8:0/2164025119 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e1c00bbf0 con 0x55819fa13d60 2023-11-27T15:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.644+0000 7f9e06ff5700 1 -- 172.21.15.8:0/2164025119 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9e1c03c030 con 0x55819fa13d60 2023-11-27T15:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.644+0000 7f9e3ca8db80 10 client.25609 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.644+0000 7f9e3ca8db80 10 client.25609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:28:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.644+0000 7f9e3ca8db80 20 client.25609 populate_metadata read hostname 'smithi008' 2023-11-27T15:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.644+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55819fa7c4d0 0x55819fa7e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.644+0000 7f9e3ca8db80 4 client.25609 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.644+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/2164025119 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55819f932600 con 0x55819fa7c4d0 2023-11-27T15:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.645+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55819fa7c4d0 0x55819fa7e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.646+0000 7f9e22d9d700 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55819fa7c4d0 0x55819fa7e8e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f0a8e273fcd6a8a server_cookie=89730095948d043f in_seq=0 out_seq=0 2023-11-27T15:28:46.649 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:46.646+0000 7f9e06ff5700 10 client.25609 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:47 smithi060 ceph-mon[131518]: pgmap v446: 129 pgs: 129 active+clean; 96 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 668 op/s 2023-11-27T15:28:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:47 smithi008 ceph-mon[128788]: pgmap v446: 129 pgs: 129 active+clean; 96 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 668 op/s 2023-11-27T15:28:47.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:47 smithi029 ceph-mon[131166]: pgmap v446: 129 pgs: 129 active+clean; 96 MiB data, 686 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 668 op/s 2023-11-27T15:28:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.477+0000 7f9e06ff5700 1 -- 172.21.15.8:0/2164025119 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55819f932600 con 0x55819fa7c4d0 2023-11-27T15:28:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.477+0000 7f9e06ff5700 10 client.25609 handle_command_reply: tid=0 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.477+0000 7f9e3ca8db80 1 client.25609 shutdown 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.477+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/2164025119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa13d60 msgr2=0x55819fa22480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.477+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa13d60 0x55819fa22480 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0074d0 tx=0x7f9e1c00bf90 comp rx=0 tx=0).stop 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.478+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/2164025119 shutdown_connections 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.478+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55819fa7c4d0 0x55819fa7e8e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.478+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55819fa13d60 0x55819fa22480 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.478+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55819f86e6b0 0x55819fa1ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.478+0000 7f9e3ca8db80 1 --2- 172.21.15.8:0/2164025119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55819fa12fb0 0x55819fa1f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.478+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/2164025119 >> 172.21.15.8:0/2164025119 conn(0x55819fa081a0 msgr2=0x55819fa1ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.478+0000 7f9e06ff5700 0 client.25609 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:28:49.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.478+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/2164025119 shutdown_connections 2023-11-27T15:28:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.478+0000 7f9e3ca8db80 1 -- 172.21.15.8:0/2164025119 wait complete. 2023-11-27T15:28:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.478+0000 7f9e3ca8db80 20 client.25609 trim_cache size 0 max 16384 2023-11-27T15:28:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:28:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:28:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:28:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:28:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.479+0000 7f9e289fe700 1 -- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e0c0a11d0 msgr2=0x7f9e0c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.479+0000 7f9e289fe700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e0c0a11d0 0x7f9e0c0a3680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c029000 tx=0x7f9e1c026000 comp rx=0 tx=0).stop 2023-11-27T15:28:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.479+0000 7f9e289fe700 1 -- 172.21.15.8:0/2121521429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e241012c0 msgr2=0x7f9e2419ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.479+0000 7f9e289fe700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e241012c0 0x7f9e2419ca20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800d950 tx=0x7f9e1800dd10 comp rx=0 tx=0).stop 2023-11-27T15:28:49.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.479+0000 7f9e289fe700 1 -- 172.21.15.8:0/2121521429 shutdown_connections 2023-11-27T15:28:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.479+0000 7f9e289fe700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9e0c0a11d0 0x7f9e0c0a3680 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.479+0000 7f9e289fe700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9e240fe960 0x7f9e2419c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.479+0000 7f9e289fe700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9e24103c80 0x7f9e241a3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.479+0000 7f9e289fe700 1 --2- 172.21.15.8:0/2121521429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9e241012c0 0x7f9e2419ca20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.479+0000 7f9e289fe700 1 -- 172.21.15.8:0/2121521429 >> 172.21.15.8:0/2121521429 conn(0x7f9e240fa550 msgr2=0x7f9e240ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.480+0000 7f9e289fe700 1 -- 172.21.15.8:0/2121521429 shutdown_connections 2023-11-27T15:28:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:28:49.480+0000 7f9e289fe700 1 -- 172.21.15.8:0/2121521429 wait complete. 2023-11-27T15:28:49.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:49 smithi008 ceph-mon[128788]: pgmap v447: 129 pgs: 129 active+clean; 108 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2023-11-27T15:28:49.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:49 smithi029 ceph-mon[131166]: pgmap v447: 129 pgs: 129 active+clean; 108 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2023-11-27T15:28:49.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:49 smithi060 ceph-mon[131518]: pgmap v447: 129 pgs: 129 active+clean; 108 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2023-11-27T15:28:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:51 smithi060 ceph-mon[131518]: pgmap v448: 129 pgs: 129 active+clean; 108 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 448 op/s 2023-11-27T15:28:51.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:51 smithi008 ceph-mon[128788]: pgmap v448: 129 pgs: 129 active+clean; 108 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 448 op/s 2023-11-27T15:28:51.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:51 smithi029 ceph-mon[131166]: pgmap v448: 129 pgs: 129 active+clean; 108 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 448 op/s 2023-11-27T15:28:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:53 smithi060 ceph-mon[131518]: pgmap v449: 129 pgs: 129 active+clean; 100 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 1.22k op/s 2023-11-27T15:28:53.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:53 smithi008 ceph-mon[128788]: pgmap v449: 129 pgs: 129 active+clean; 100 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 1.22k op/s 2023-11-27T15:28:53.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:53 smithi029 ceph-mon[131166]: pgmap v449: 129 pgs: 129 active+clean; 100 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 1.22k op/s 2023-11-27T15:28:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:55 smithi060 ceph-mon[131518]: pgmap v450: 129 pgs: 129 active+clean; 107 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 1.28k op/s 2023-11-27T15:28:55.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:55 smithi008 ceph-mon[128788]: pgmap v450: 129 pgs: 129 active+clean; 107 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 1.28k op/s 2023-11-27T15:28:55.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:55 smithi029 ceph-mon[131166]: pgmap v450: 129 pgs: 129 active+clean; 107 MiB data, 791 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 1.28k op/s 2023-11-27T15:28:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:57 smithi060 ceph-mon[131518]: pgmap v451: 129 pgs: 129 active+clean; 112 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 1.26k op/s 2023-11-27T15:28:57.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:57 smithi008 ceph-mon[128788]: pgmap v451: 129 pgs: 129 active+clean; 112 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 1.26k op/s 2023-11-27T15:28:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:57 smithi029 ceph-mon[131166]: pgmap v451: 129 pgs: 129 active+clean; 112 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 1.26k op/s 2023-11-27T15:28:59.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:28:59 smithi008 ceph-mon[128788]: pgmap v452: 129 pgs: 129 active+clean; 118 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 1.24k op/s 2023-11-27T15:28:59.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:28:59 smithi029 ceph-mon[131166]: pgmap v452: 129 pgs: 129 active+clean; 118 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 1.24k op/s 2023-11-27T15:28:59.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:28:59 smithi060 ceph-mon[131518]: pgmap v452: 129 pgs: 129 active+clean; 118 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 1.24k op/s 2023-11-27T15:29:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:01 smithi060 ceph-mon[131518]: pgmap v453: 129 pgs: 129 active+clean; 122 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 1.16k op/s 2023-11-27T15:29:01.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:01 smithi008 ceph-mon[128788]: pgmap v453: 129 pgs: 129 active+clean; 122 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 1.16k op/s 2023-11-27T15:29:01.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:01 smithi029 ceph-mon[131166]: pgmap v453: 129 pgs: 129 active+clean; 122 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 1.16k op/s 2023-11-27T15:29:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.683+0000 7f1d96fff700 1 -- 172.21.15.8:0/1438314758 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d90101dd0 msgr2=0x7f1d90108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.683+0000 7f1d96fff700 1 --2- 172.21.15.8:0/1438314758 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d90101dd0 0x7f1d90108690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009f90 tx=0x7f1d8800b330 comp rx=0 tx=0).stop 2023-11-27T15:29:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.683+0000 7f1d96fff700 1 -- 172.21.15.8:0/1438314758 shutdown_connections 2023-11-27T15:29:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.683+0000 7f1d96fff700 1 --2- 172.21.15.8:0/1438314758 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d90100250 0x7f1d90100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.683+0000 7f1d96fff700 1 --2- 172.21.15.8:0/1438314758 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d90101dd0 0x7f1d90108690 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.683+0000 7f1d96fff700 1 --2- 172.21.15.8:0/1438314758 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d90101440 0x7f1d90101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.683+0000 7f1d96fff700 1 -- 172.21.15.8:0/1438314758 >> 172.21.15.8:0/1438314758 conn(0x7f1d900fb850 msgr2=0x7f1d900fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.683+0000 7f1d96fff700 1 -- 172.21.15.8:0/1438314758 shutdown_connections 2023-11-27T15:29:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.683+0000 7f1d96fff700 1 -- 172.21.15.8:0/1438314758 wait complete. 2023-11-27T15:29:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.684+0000 7f1d96fff700 1 Processor -- start 2023-11-27T15:29:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d96fff700 1 -- start start 2023-11-27T15:29:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d96fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d90100250 0x7f1d90198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d96fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d90101440 0x7f1d90198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d96fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d90101dd0 0x7f1d9019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d96fff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9019fe80 con 0x7f1d90101dd0 2023-11-27T15:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d96fff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9019ffc0 con 0x7f1d90101440 2023-11-27T15:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d96fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d901a0150 con 0x7f1d90100250 2023-11-27T15:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d94d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d90100250 0x7f1d90198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d94d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d90100250 0x7f1d90198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:49050/0 (socket says 172.21.15.8:49050) 2023-11-27T15:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d94d9b700 1 -- 172.21.15.8:0/3393900396 learned_addr learned my addr 172.21.15.8:0/3393900396 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.685+0000 7f1d9559c700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d90101dd0 0x7f1d9019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.686+0000 7f1d94d9b700 1 -- 172.21.15.8:0/3393900396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d90101440 msgr2=0x7f1d90198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.686+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d90101440 0x7f1d90198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.686+0000 7f1d94d9b700 1 -- 172.21.15.8:0/3393900396 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d90101dd0 msgr2=0x7f1d9019f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.686+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d90101dd0 0x7f1d9019f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.686+0000 7f1d94d9b700 1 -- 172.21.15.8:0/3393900396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009520 con 0x7f1d90100250 2023-11-27T15:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.686+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d90100250 0x7f1d90198300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400ea50 tx=0x7f1d8400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.686+0000 7f1d8dffb700 1 -- 172.21.15.8:0/3393900396 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8400cbd0 con 0x7f1d90100250 2023-11-27T15:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.686+0000 7f1d96fff700 1 -- 172.21.15.8:0/3393900396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d88009c40 con 0x7f1d90100250 2023-11-27T15:29:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.687+0000 7f1d96fff700 1 -- 172.21.15.8:0/3393900396 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9010bc30 con 0x7f1d90100250 2023-11-27T15:29:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.687+0000 7f1d8dffb700 1 -- 172.21.15.8:0/3393900396 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d8400cd30 con 0x7f1d90100250 2023-11-27T15:29:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.687+0000 7f1d8dffb700 1 -- 172.21.15.8:0/3393900396 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84018680 con 0x7f1d90100250 2023-11-27T15:29:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.691+0000 7f1d8dffb700 1 -- 172.21.15.8:0/3393900396 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1d84010430 con 0x7f1d90100250 2023-11-27T15:29:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1daa937b80 1 Processor -- start 2023-11-27T15:29:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1daa937b80 1 -- start start 2023-11-27T15:29:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x559794662870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x5597946635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794663b10 0x559794663f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1daa937b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559794568000 con 0x559794662db0 2023-11-27T15:29:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1daa937b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597945ba840 con 0x5597944be6b0 2023-11-27T15:29:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1daa937b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597945b3cd0 con 0x559794663b10 2023-11-27T15:29:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1d9559c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x559794662870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1d9559c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x559794662870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:57922/0 (socket says 172.21.15.8:57922) 2023-11-27T15:29:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.693+0000 7f1d9559c700 1 -- 172.21.15.8:0/1688436752 learned_addr learned my addr 172.21.15.8:0/1688436752 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.694+0000 7f1d8ffff700 1 --2- 172.21.15.8:0/1688436752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794663b10 0x559794663f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.694+0000 7f1d9559c700 1 -- 172.21.15.8:0/1688436752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794663b10 msgr2=0x559794663f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.694+0000 7f1d9559c700 1 --2- 172.21.15.8:0/1688436752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794663b10 0x559794663f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.694+0000 7f1d8dffb700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d800a1150 0x7f1d800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.694+0000 7f1d9559c700 1 -- 172.21.15.8:0/1688436752 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 msgr2=0x5597946635d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:29:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.694+0000 7f1d9559c700 1 --2- 172.21.15.8:0/1688436752 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x5597946635d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.694+0000 7f1d9559c700 1 -- 172.21.15.8:0/1688436752 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597944355d0 con 0x5597944be6b0 2023-11-27T15:29:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.694+0000 7f1d8dffb700 1 -- 172.21.15.8:0/3393900396 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1d84014070 con 0x7f1d90100250 2023-11-27T15:29:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.695+0000 7f1d9559c700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d800a1150 0x7f1d800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.695+0000 7f1d9559c700 1 --2- 172.21.15.8:0/1688436752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x559794662870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009b40 tx=0x7f1d88009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7befd98e8faa9610 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.695+0000 7f1d7affd700 1 -- 172.21.15.8:0/1688436752 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800e040 con 0x5597944be6b0 2023-11-27T15:29:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.695+0000 7f1d7affd700 1 -- 172.21.15.8:0/1688436752 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d88003e60 con 0x5597944be6b0 2023-11-27T15:29:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.695+0000 7f1d7affd700 1 -- 172.21.15.8:0/1688436752 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88028aa0 con 0x5597944be6b0 2023-11-27T15:29:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.696+0000 7f1daa937b80 1 -- 172.21.15.8:0/1688436752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 msgr2=0x559794662870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.696+0000 7f1daa937b80 1 --2- 172.21.15.8:0/1688436752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x559794662870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009b40 tx=0x7f1d88009b70 comp rx=0 tx=0).stop 2023-11-27T15:29:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.696+0000 7f1d9559c700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d800a1150 0x7f1d800a3600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1d880061e0 tx=0x7f1d8802a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.696+0000 7f1daa937b80 1 -- 172.21.15.8:0/1688436752 shutdown_connections 2023-11-27T15:29:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.696+0000 7f1daa937b80 1 --2- 172.21.15.8:0/1688436752 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794663b10 0x559794663f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.696+0000 7f1daa937b80 1 --2- 172.21.15.8:0/1688436752 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x559794662870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.696+0000 7f1daa937b80 1 --2- 172.21.15.8:0/1688436752 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x5597946635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.696+0000 7f1daa937b80 1 -- 172.21.15.8:0/1688436752 >> 172.21.15.8:0/1688436752 conn(0x559794657f50 msgr2=0x55979465d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.696+0000 7f1daa937b80 1 -- 172.21.15.8:0/1688436752 shutdown_connections 2023-11-27T15:29:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.697+0000 7f1daa937b80 1 -- 172.21.15.8:0/1688436752 wait complete. 2023-11-27T15:29:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.697+0000 7f1daa937b80 1 -- start start 2023-11-27T15:29:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.697+0000 7f1daa937b80 1 Processor -- start 2023-11-27T15:29:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794662db0 0x55979466c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794663b10 0x559794673250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1d94d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1d94d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:57930/0 (socket says 172.21.15.8:57930) 2023-11-27T15:29:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1d94d9b700 1 -- 172.21.15.8:0/955291332 learned_addr learned my addr 172.21.15.8:0/955291332 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559794555880 con 0x559794663b10 2023-11-27T15:29:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x559794555520 con 0x5597944be6b0 2023-11-27T15:29:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597944378c0 con 0x559794662db0 2023-11-27T15:29:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1d8ffff700 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794662db0 0x55979466c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.699+0000 7f1d9559c700 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794663b10 0x559794673250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.700+0000 7f1daa937b80 10 client.0 _ll_register_callbacks cb 0x559794573720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:29:02.707 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.700+0000 7f1d8ffff700 1 -- 172.21.15.8:0/955291332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 msgr2=0x55979466bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.700+0000 7f1d8ffff700 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.700+0000 7f1d8ffff700 1 -- 172.21.15.8:0/955291332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794663b10 msgr2=0x559794673250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.700+0000 7f1d8ffff700 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794663b10 0x559794673250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.700+0000 7f1d8ffff700 1 -- 172.21.15.8:0/955291332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d90108fc0 con 0x559794662db0 2023-11-27T15:29:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.700+0000 7f1d8ffff700 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794662db0 0x55979466c210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c011670 tx=0x7f1d7c011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82b2bb08bd61bb90 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.700+0000 7f1d6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:29:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.700+0000 7f1d6ffff700 1 -- 172.21.15.8:0/955291332 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c004830 con 0x559794662db0 2023-11-27T15:29:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.701+0000 7f1d6ffff700 1 -- 172.21.15.8:0/955291332 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1d7c004990 con 0x559794662db0 2023-11-27T15:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.701+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559794655710 con 0x559794662db0 2023-11-27T15:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.701+0000 7f1d6ffff700 1 -- 172.21.15.8:0/955291332 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d7c004b50 con 0x559794662db0 2023-11-27T15:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.701+0000 7f1daa937b80 10 client.25621 fetch_fsmap learned FSMap version 28 2023-11-27T15:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.701+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55979457a130 con 0x559794662db0 2023-11-27T15:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.701+0000 7f1d6ffff700 1 -- 172.21.15.8:0/955291332 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559794655710 con 0x559794662db0 2023-11-27T15:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.702+0000 7f1d6ffff700 1 -- 172.21.15.8:0/955291332 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1d7c020020 con 0x559794662db0 2023-11-27T15:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.702+0000 7f1daa937b80 10 client.25621 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.702+0000 7f1daa937b80 10 client.25621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.702+0000 7f1daa937b80 20 client.25621 populate_metadata read hostname 'smithi008' 2023-11-27T15:29:02.709 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.702+0000 7f1daa937b80 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559794677cb0 0x55979467a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.702+0000 7f1daa937b80 4 client.25621 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:29:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.702+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559794576040 con 0x559794677cb0 2023-11-27T15:29:02.710 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.703+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559794677cb0 0x55979467a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.703+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559794677cb0 0x55979467a0c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38797cb47def26e7 server_cookie=7c1cca271bc66d79 in_seq=0 out_seq=0 2023-11-27T15:29:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.703+0000 7f1d6ffff700 10 client.25621 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.704+0000 7f1d6ffff700 1 -- 172.21.15.8:0/955291332 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559794576040 con 0x559794677cb0 2023-11-27T15:29:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.704+0000 7f1d6ffff700 10 client.25621 handle_command_reply: tid=0 2023-11-27T15:29:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.704+0000 7f1daa937b80 1 client.25621 shutdown 2023-11-27T15:29:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794662db0 msgr2=0x55979466c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1daa937b80 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794662db0 0x55979466c210 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c011670 tx=0x7f1d7c011980 comp rx=0 tx=0).stop 2023-11-27T15:29:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 shutdown_connections 2023-11-27T15:29:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1daa937b80 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794663b10 0x559794673250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1daa937b80 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1daa937b80 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559794677cb0 0x55979467a0c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1daa937b80 1 --2- 172.21.15.8:0/955291332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794662db0 0x55979466c210 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 >> 172.21.15.8:0/955291332 conn(0x559794657f50 msgr2=0x55979465c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1d6ffff700 0 client.25621 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 shutdown_connections 2023-11-27T15:29:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.705+0000 7f1daa937b80 1 -- 172.21.15.8:0/955291332 wait complete. 2023-11-27T15:29:02.713 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.706+0000 7f1daa937b80 20 client.25621 trim_cache size 0 max 16384 2023-11-27T15:29:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1daa937b80 1 Processor -- start 2023-11-27T15:29:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1daa937b80 1 -- start start 2023-11-27T15:29:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597944be6b0 0x55979466bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x559794673210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559794663b10 0x55979466c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1daa937b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979465ebe0 con 0x559794662db0 2023-11-27T15:29:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1daa937b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979466c950 con 0x559794663b10 2023-11-27T15:29:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1daa937b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597946633c0 con 0x5597944be6b0 2023-11-27T15:29:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1d94d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597944be6b0 0x55979466bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1d9559c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559794663b10 0x55979466c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1d94d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597944be6b0 0x55979466bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:49070/0 (socket says 172.21.15.8:49070) 2023-11-27T15:29:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1d94d9b700 1 -- 172.21.15.8:0/418269284 learned_addr learned my addr 172.21.15.8:0/418269284 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:02.731 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.726+0000 7f1d8ffff700 1 --2- 172.21.15.8:0/418269284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x559794673210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.727+0000 7f1d94d9b700 1 -- 172.21.15.8:0/418269284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559794663b10 msgr2=0x55979466c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.727+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/418269284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559794663b10 0x55979466c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.735 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.727+0000 7f1d94d9b700 1 -- 172.21.15.8:0/418269284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 msgr2=0x559794673210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.727+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/418269284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x559794673210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.727+0000 7f1d94d9b700 1 -- 172.21.15.8:0/418269284 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559794579e80 con 0x5597944be6b0 2023-11-27T15:29:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.727+0000 7f1d8ffff700 1 --2- 172.21.15.8:0/418269284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x559794673210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:29:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.727+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/418269284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597944be6b0 0x55979466bcd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1d84005950 tx=0x7f1d84085040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a0ca81381abf31a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.727+0000 7f1d7b7fe700 1 -- 172.21.15.8:0/418269284 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8409e270 con 0x5597944be6b0 2023-11-27T15:29:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.727+0000 7f1d7b7fe700 1 -- 172.21.15.8:0/418269284 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d840a2690 con 0x5597944be6b0 2023-11-27T15:29:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.727+0000 7f1d7b7fe700 1 -- 172.21.15.8:0/418269284 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840a2200 con 0x5597944be6b0 2023-11-27T15:29:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.728+0000 7f1daa937b80 1 -- 172.21.15.8:0/418269284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597944be6b0 msgr2=0x55979466bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.728+0000 7f1daa937b80 1 --2- 172.21.15.8:0/418269284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597944be6b0 0x55979466bcd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1d84005950 tx=0x7f1d84085040 comp rx=0 tx=0).stop 2023-11-27T15:29:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.728+0000 7f1daa937b80 1 -- 172.21.15.8:0/418269284 shutdown_connections 2023-11-27T15:29:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.728+0000 7f1daa937b80 1 --2- 172.21.15.8:0/418269284 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5597944be6b0 0x55979466bcd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.728+0000 7f1daa937b80 1 --2- 172.21.15.8:0/418269284 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559794663b10 0x55979466c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.728+0000 7f1daa937b80 1 --2- 172.21.15.8:0/418269284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x559794673210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.728+0000 7f1daa937b80 1 -- 172.21.15.8:0/418269284 >> 172.21.15.8:0/418269284 conn(0x559794657f50 msgr2=0x559794669f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.728+0000 7f1daa937b80 1 -- 172.21.15.8:0/418269284 shutdown_connections 2023-11-27T15:29:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.728+0000 7f1daa937b80 1 -- 172.21.15.8:0/418269284 wait complete. 2023-11-27T15:29:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.729+0000 7f1daa937b80 1 -- start start 2023-11-27T15:29:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.729+0000 7f1daa937b80 1 Processor -- start 2023-11-27T15:29:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x55979466f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1daa937b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794663b10 0x559794672240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1daa937b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559794672780 con 0x559794662db0 2023-11-27T15:29:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1daa937b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597946728f0 con 0x5597944be6b0 2023-11-27T15:29:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1daa937b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559794672ab0 con 0x559794663b10 2023-11-27T15:29:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1daa937b80 10 client.0 _ll_register_callbacks cb 0x7f1d7c009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:29:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1d94d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1d94d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:57952/0 (socket says 172.21.15.8:57952) 2023-11-27T15:29:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1d94d9b700 1 -- 172.21.15.8:0/693807401 learned_addr learned my addr 172.21.15.8:0/693807401 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1d8ffff700 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x55979466f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1d9559c700 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794663b10 0x559794672240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1d94d9b700 1 -- 172.21.15.8:0/693807401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794663b10 msgr2=0x559794672240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794663b10 0x559794672240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1d94d9b700 1 -- 172.21.15.8:0/693807401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 msgr2=0x55979466f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x55979466f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.730+0000 7f1d94d9b700 1 -- 172.21.15.8:0/693807401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9010bc30 con 0x5597944be6b0 2023-11-27T15:29:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.731+0000 7f1d94d9b700 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466ece0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d84096cb0 tx=0x7f1d8400c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f3e370f1e2aa3bf server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.731+0000 7f1d6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:29:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.731+0000 7f1d6ffff700 1 -- 172.21.15.8:0/693807401 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840b1070 con 0x5597944be6b0 2023-11-27T15:29:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.731+0000 7f1daa937b80 1 -- 172.21.15.8:0/693807401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d88003e60 con 0x5597944be6b0 2023-11-27T15:29:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.731+0000 7f1d6ffff700 1 -- 172.21.15.8:0/693807401 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1d840caa40 con 0x5597944be6b0 2023-11-27T15:29:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.731+0000 7f1d6ffff700 1 -- 172.21.15.8:0/693807401 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d8409e2f0 con 0x5597944be6b0 2023-11-27T15:29:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.731+0000 7f1daa937b80 10 client.25604 fetch_fsmap learned FSMap version 28 2023-11-27T15:29:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.731+0000 7f1daa937b80 1 -- 172.21.15.8:0/693807401 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5597943cac00 con 0x5597944be6b0 2023-11-27T15:29:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.731+0000 7f1d6ffff700 1 -- 172.21.15.8:0/693807401 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d88003e60 con 0x5597944be6b0 2023-11-27T15:29:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.732+0000 7f1d6ffff700 1 -- 172.21.15.8:0/693807401 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1d8407e0f0 con 0x5597944be6b0 2023-11-27T15:29:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.732+0000 7f1daa937b80 10 client.25604 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:29:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.732+0000 7f1daa937b80 10 client.25604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:29:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.732+0000 7f1daa937b80 20 client.25604 populate_metadata read hostname 'smithi008' 2023-11-27T15:29:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.732+0000 7f1daa937b80 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5597946d4e90 0x5597946d72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.732+0000 7f1daa937b80 4 client.25604 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:29:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.732+0000 7f1daa937b80 1 -- 172.21.15.8:0/693807401 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559794582600 con 0x5597946d4e90 2023-11-27T15:29:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.732+0000 7f1d8ffff700 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5597946d4e90 0x5597946d72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.732+0000 7f1d6ffff700 10 client.25604 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:02.742 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:02.732+0000 7f1d8ffff700 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5597946d4e90 0x5597946d72a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a85ef586431944a1 server_cookie=2e0945fb0d7e5e13 in_seq=0 out_seq=0 2023-11-27T15:29:03.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:03 smithi060 ceph-mon[131518]: pgmap v454: 129 pgs: 129 active+clean; 131 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 966 op/s 2023-11-27T15:29:03.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:03 smithi008 ceph-mon[128788]: pgmap v454: 129 pgs: 129 active+clean; 131 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 966 op/s 2023-11-27T15:29:03.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:03 smithi029 ceph-mon[131166]: pgmap v454: 129 pgs: 129 active+clean; 131 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 966 op/s 2023-11-27T15:29:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1d6ffff700 1 -- 172.21.15.8:0/693807401 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559794582600 con 0x5597946d4e90 2023-11-27T15:29:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1d6ffff700 10 client.25604 handle_command_reply: tid=0 2023-11-27T15:29:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1daa937b80 1 client.25604 shutdown 2023-11-27T15:29:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1daa937b80 1 -- 172.21.15.8:0/693807401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 msgr2=0x55979466ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1daa937b80 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466ece0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1d84096cb0 tx=0x7f1d8400c6e0 comp rx=0 tx=0).stop 2023-11-27T15:29:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1daa937b80 1 -- 172.21.15.8:0/693807401 shutdown_connections 2023-11-27T15:29:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1daa937b80 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5597946d4e90 0x5597946d72a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1daa937b80 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559794663b10 0x559794672240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1daa937b80 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5597944be6b0 0x55979466ece0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1daa937b80 1 --2- 172.21.15.8:0/693807401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559794662db0 0x55979466f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.333+0000 7f1daa937b80 1 -- 172.21.15.8:0/693807401 >> 172.21.15.8:0/693807401 conn(0x559794657f50 msgr2=0x55979466bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.334+0000 7f1d6ffff700 0 client.25604 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.334+0000 7f1daa937b80 1 -- 172.21.15.8:0/693807401 shutdown_connections 2023-11-27T15:29:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.334+0000 7f1daa937b80 1 -- 172.21.15.8:0/693807401 wait complete. 2023-11-27T15:29:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.334+0000 7f1daa937b80 20 client.25604 trim_cache size 0 max 16384 2023-11-27T15:29:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:29:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:29:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:29:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:29:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.335+0000 7f1d96fff700 1 -- 172.21.15.8:0/3393900396 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d800a1150 msgr2=0x7f1d800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.335+0000 7f1d96fff700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d800a1150 0x7f1d800a3600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1d880061e0 tx=0x7f1d8802a000 comp rx=0 tx=0).stop 2023-11-27T15:29:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.335+0000 7f1d96fff700 1 -- 172.21.15.8:0/3393900396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d90100250 msgr2=0x7f1d90198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.335+0000 7f1d96fff700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d90100250 0x7f1d90198300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400ea50 tx=0x7f1d8400ed60 comp rx=0 tx=0).stop 2023-11-27T15:29:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.335+0000 7f1d96fff700 1 -- 172.21.15.8:0/3393900396 shutdown_connections 2023-11-27T15:29:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.335+0000 7f1d96fff700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1d800a1150 0x7f1d800a3600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.335+0000 7f1d96fff700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1d90100250 0x7f1d90198300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.335+0000 7f1d96fff700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1d90101dd0 0x7f1d9019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.335+0000 7f1d96fff700 1 --2- 172.21.15.8:0/3393900396 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1d90101440 0x7f1d90198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.335+0000 7f1d96fff700 1 -- 172.21.15.8:0/3393900396 >> 172.21.15.8:0/3393900396 conn(0x7f1d900fb850 msgr2=0x7f1d900fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.336+0000 7f1d96fff700 1 -- 172.21.15.8:0/3393900396 shutdown_connections 2023-11-27T15:29:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:05.336+0000 7f1d96fff700 1 -- 172.21.15.8:0/3393900396 wait complete. 2023-11-27T15:29:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:05 smithi060 ceph-mon[131518]: pgmap v455: 129 pgs: 129 active+clean; 139 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2023-11-27T15:29:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:05 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:29:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:05 smithi008 ceph-mon[128788]: pgmap v455: 129 pgs: 129 active+clean; 139 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2023-11-27T15:29:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:05 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:29:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:05 smithi029 ceph-mon[131166]: pgmap v455: 129 pgs: 129 active+clean; 139 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2023-11-27T15:29:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:05 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:29:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:07 smithi060 ceph-mon[131518]: pgmap v456: 129 pgs: 129 active+clean; 133 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 519 op/s 2023-11-27T15:29:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:07 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:29:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:07 smithi008 ceph-mon[128788]: pgmap v456: 129 pgs: 129 active+clean; 133 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 519 op/s 2023-11-27T15:29:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:07 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:29:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:07 smithi029 ceph-mon[131166]: pgmap v456: 129 pgs: 129 active+clean; 133 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 519 op/s 2023-11-27T15:29:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:07 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:29:08.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:08 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:29:08.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:08 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:29:08.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:08 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:29:08.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:29:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:29:08.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:29:08.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:08 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:29:08.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:08 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:29:08.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:08 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:29:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:09 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:29:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:09 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:29:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:09 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:29:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:09 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:29:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:09 smithi060 ceph-mon[131518]: pgmap v457: 129 pgs: 129 active+clean; 131 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 944 op/s 2023-11-27T15:29:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:09 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:29:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:09 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:29:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:09 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:29:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:09 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:29:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:09 smithi008 ceph-mon[128788]: pgmap v457: 129 pgs: 129 active+clean; 131 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 944 op/s 2023-11-27T15:29:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:09 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:29:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:09 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:29:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:09 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:29:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:09 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:29:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:09 smithi029 ceph-mon[131166]: pgmap v457: 129 pgs: 129 active+clean; 131 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.0 MiB/s wr, 944 op/s 2023-11-27T15:29:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:10 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:29:10.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:10 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:29:10.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:10 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:29:11.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:11 smithi060 ceph-mon[131518]: pgmap v458: 129 pgs: 129 active+clean; 135 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 1.02k op/s 2023-11-27T15:29:11.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:11 smithi008 ceph-mon[128788]: pgmap v458: 129 pgs: 129 active+clean; 135 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 1.02k op/s 2023-11-27T15:29:11.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:11 smithi029 ceph-mon[131166]: pgmap v458: 129 pgs: 129 active+clean; 135 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.7 MiB/s wr, 1.02k op/s 2023-11-27T15:29:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:13 smithi060 ceph-mon[131518]: pgmap v459: 129 pgs: 129 active+clean; 142 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 1.03k op/s 2023-11-27T15:29:13.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:13 smithi008 ceph-mon[128788]: pgmap v459: 129 pgs: 129 active+clean; 142 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 1.03k op/s 2023-11-27T15:29:13.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:13 smithi029 ceph-mon[131166]: pgmap v459: 129 pgs: 129 active+clean; 142 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 1.03k op/s 2023-11-27T15:29:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:15 smithi060 ceph-mon[131518]: pgmap v460: 129 pgs: 129 active+clean; 147 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 1.01k op/s 2023-11-27T15:29:15.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:15 smithi008 ceph-mon[128788]: pgmap v460: 129 pgs: 129 active+clean; 147 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 1.01k op/s 2023-11-27T15:29:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:15 smithi029 ceph-mon[131166]: pgmap v460: 129 pgs: 129 active+clean; 147 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 1.01k op/s 2023-11-27T15:29:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:17 smithi060 ceph-mon[131518]: pgmap v461: 129 pgs: 129 active+clean; 147 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 917 op/s 2023-11-27T15:29:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:17 smithi008 ceph-mon[128788]: pgmap v461: 129 pgs: 129 active+clean; 147 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 917 op/s 2023-11-27T15:29:17.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:17 smithi029 ceph-mon[131166]: pgmap v461: 129 pgs: 129 active+clean; 147 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 917 op/s 2023-11-27T15:29:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.552+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1122765024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9c4103490 msgr2=0x7fe9c4105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.552+0000 7fe9c89fe700 1 --2- 172.21.15.8:0/1122765024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9c4103490 0x7fe9c4105870 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009990 tx=0x7fe9b8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.553+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1122765024 shutdown_connections 2023-11-27T15:29:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.553+0000 7fe9c89fe700 1 --2- 172.21.15.8:0/1122765024 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9c4100b70 0x7fe9c4102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.553+0000 7fe9c89fe700 1 --2- 172.21.15.8:0/1122765024 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9c4105db0 0x7fe9c4108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.553+0000 7fe9c89fe700 1 --2- 172.21.15.8:0/1122765024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9c4103490 0x7fe9c4105870 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.553+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1122765024 >> 172.21.15.8:0/1122765024 conn(0x7fe9c40fa550 msgr2=0x7fe9c40fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.553+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1122765024 shutdown_connections 2023-11-27T15:29:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.553+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1122765024 wait complete. 2023-11-27T15:29:18.557 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.554+0000 7fe9c89fe700 1 Processor -- start 2023-11-27T15:29:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.554+0000 7fe9c89fe700 1 -- start start 2023-11-27T15:29:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.555+0000 7fe9c89fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9c4100b70 0x7fe9c406f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.555+0000 7fe9c89fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9c4103490 0x7fe9c406f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.555+0000 7fe9c89fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9c4105db0 0x7fe9c41a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.555+0000 7fe9c89fe700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40729b0 con 0x7fe9c4100b70 2023-11-27T15:29:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.555+0000 7fe9c89fe700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4072b20 con 0x7fe9c4105db0 2023-11-27T15:29:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.555+0000 7fe9c89fe700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c41a6510 con 0x7fe9c4103490 2023-11-27T15:29:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.555+0000 7fe9c359e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9c4105db0 0x7fe9c41a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.555+0000 7fe9c359e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9c4105db0 0x7fe9c41a5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:43388/0 (socket says 172.21.15.8:43388) 2023-11-27T15:29:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.555+0000 7fe9c359e700 1 -- 172.21.15.8:0/1076289098 learned_addr learned my addr 172.21.15.8:0/1076289098 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.555+0000 7fe9c259c700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9c4103490 0x7fe9c406f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9c4100b70 0x7fe9c406f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9c359e700 1 -- 172.21.15.8:0/1076289098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9c4103490 msgr2=0x7fe9c406f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9c359e700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9c4103490 0x7fe9c406f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9c359e700 1 -- 172.21.15.8:0/1076289098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9c4100b70 msgr2=0x7fe9c406f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9c359e700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9c4100b70 0x7fe9c406f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9c359e700 1 -- 172.21.15.8:0/1076289098 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc009c40 con 0x7fe9c4105db0 2023-11-27T15:29:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9c359e700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9c4105db0 0x7fe9c41a5fd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00c3a0 tx=0x7fe9bc00c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9abfff700 1 -- 172.21.15.8:0/1076289098 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc017070 con 0x7fe9c4105db0 2023-11-27T15:29:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1076289098 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b80095f0 con 0x7fe9c4105db0 2023-11-27T15:29:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9abfff700 1 -- 172.21.15.8:0/1076289098 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9bc00f040 con 0x7fe9c4105db0 2023-11-27T15:29:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.556+0000 7fe9abfff700 1 -- 172.21.15.8:0/1076289098 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00e050 con 0x7fe9c4105db0 2023-11-27T15:29:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.557+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1076289098 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c410b9e0 con 0x7fe9c4105db0 2023-11-27T15:29:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.560+0000 7fe9abfff700 1 -- 172.21.15.8:0/1076289098 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe9bc003a00 con 0x7fe9c4105db0 2023-11-27T15:29:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.562+0000 7fe9abfff700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9ac0a11d0 0x7fe9ac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.562+0000 7fe9dcaf2b80 1 Processor -- start 2023-11-27T15:29:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.562+0000 7fe9abfff700 1 -- 172.21.15.8:0/1076289098 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe9bc0cc500 con 0x7fe9c4105db0 2023-11-27T15:29:18.565 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.562+0000 7fe9dcaf2b80 1 -- start start 2023-11-27T15:29:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.562+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9ac0a11d0 0x7fe9ac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.562+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292cf9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.562+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfa3c0 0x55f292cfac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.562+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfb170 0x55f292cfb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9dcaf2b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292bfb000 con 0x55f292b516b0 2023-11-27T15:29:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9dcaf2b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292c4d840 con 0x55f292cfa3c0 2023-11-27T15:29:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9dcaf2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292c46cd0 con 0x55f292cfb170 2023-11-27T15:29:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfa3c0 0x55f292cfac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c259c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfa3c0 0x55f292cfac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:43390/0 (socket says 172.21.15.8:43390) 2023-11-27T15:29:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c259c700 1 -- 172.21.15.8:0/1918097119 learned_addr learned my addr 172.21.15.8:0/1918097119 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c359e700 1 --2- 172.21.15.8:0/1918097119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfb170 0x55f292cfb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/1918097119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292cf9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c259c700 1 -- 172.21.15.8:0/1918097119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfb170 msgr2=0x55f292cfb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c259c700 1 --2- 172.21.15.8:0/1918097119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfb170 0x55f292cfb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c259c700 1 -- 172.21.15.8:0/1918097119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 msgr2=0x55f292cf9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c259c700 1 --2- 172.21.15.8:0/1918097119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292cf9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c259c700 1 -- 172.21.15.8:0/1918097119 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f292ac85d0 con 0x55f292cfa3c0 2023-11-27T15:29:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.563+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/1918097119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292cf9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:29:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.564+0000 7fe9c259c700 1 --2- 172.21.15.8:0/1918097119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfa3c0 0x55f292cfac30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8015000 tx=0x7fe9b8005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf203781bbba62d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.564+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9ac0a11d0 0x7fe9ac0a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55f292cf0ec0 tx=0x7fe9b40098c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.564+0000 7fe9a97fa700 1 -- 172.21.15.8:0/1918097119 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b801d070 con 0x55f292cfa3c0 2023-11-27T15:29:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.564+0000 7fe9a97fa700 1 -- 172.21.15.8:0/1918097119 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9b801e040 con 0x55f292cfa3c0 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.564+0000 7fe9a97fa700 1 -- 172.21.15.8:0/1918097119 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8004970 con 0x55f292cfa3c0 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.565+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/1918097119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfa3c0 msgr2=0x55f292cfac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.565+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/1918097119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfa3c0 0x55f292cfac30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8015000 tx=0x7fe9b8005ea0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.565+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/1918097119 shutdown_connections 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.565+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/1918097119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfb170 0x55f292cfb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.565+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/1918097119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292cf9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.565+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/1918097119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfa3c0 0x55f292cfac30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.565+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/1918097119 >> 172.21.15.8:0/1918097119 conn(0x55f292ceb1a0 msgr2=0x55f292cf02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.565+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/1918097119 shutdown_connections 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.565+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/1918097119 wait complete. 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.566+0000 7fe9dcaf2b80 1 -- start start 2023-11-27T15:29:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.566+0000 7fe9dcaf2b80 1 Processor -- start 2023-11-27T15:29:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d03290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfb170 0x55f292d0a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9dcaf2b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292be8880 con 0x55f292b516b0 2023-11-27T15:29:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9dcaf2b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292be8520 con 0x55f292cfb170 2023-11-27T15:29:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9dcaf2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292aca8c0 con 0x55f292cfa3c0 2023-11-27T15:29:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9dcaf2b80 10 client.0 _ll_register_callbacks cb 0x55f292c06720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:29:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9c2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d03290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9c2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d03290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60112/0 (socket says 172.21.15.8:60112) 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9c2d9d700 1 -- 172.21.15.8:0/626424097 learned_addr learned my addr 172.21.15.8:0/626424097 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9c259c700 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.567+0000 7fe9c359e700 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfb170 0x55f292d0a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.568+0000 7fe9c2d9d700 1 -- 172.21.15.8:0/626424097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 msgr2=0x55f292d037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.568+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.568+0000 7fe9c2d9d700 1 -- 172.21.15.8:0/626424097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfb170 msgr2=0x55f292d0a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.568+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfb170 0x55f292d0a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.568+0000 7fe9c2d9d700 1 -- 172.21.15.8:0/626424097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c4108db0 con 0x55f292b516b0 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.568+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d03290 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40045d0 tx=0x7fe9b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6fb5029febf24d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.568+0000 7fe9a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.568+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/626424097 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4003b90 con 0x55f292b516b0 2023-11-27T15:29:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.568+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/626424097 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe9b4003cf0 con 0x55f292b516b0 2023-11-27T15:29:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.568+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/626424097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f292ce8960 con 0x55f292b516b0 2023-11-27T15:29:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.569+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/626424097 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9b4004830 con 0x55f292b516b0 2023-11-27T15:29:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.569+0000 7fe9dcaf2b80 10 client.15588 fetch_fsmap learned FSMap version 30 2023-11-27T15:29:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.569+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/626424097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f292c0d130 con 0x55f292b516b0 2023-11-27T15:29:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.569+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/626424097 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f292ce8960 con 0x55f292b516b0 2023-11-27T15:29:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.570+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/626424097 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe9b4021430 con 0x55f292b516b0 2023-11-27T15:29:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.570+0000 7fe9dcaf2b80 10 client.15588 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:29:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.570+0000 7fe9dcaf2b80 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:29:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.570+0000 7fe9dcaf2b80 20 client.15588 populate_metadata read hostname 'smithi008' 2023-11-27T15:29:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.570+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f292d0f010 0x55f292d11420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.570+0000 7fe9dcaf2b80 4 client.15588 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:29:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.570+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/626424097 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f292c09040 con 0x55f292d0f010 2023-11-27T15:29:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.767+0000 7fe9c259c700 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f292d0f010 0x55f292d11420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.767+0000 7fe9a6ff5700 10 client.15588 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:18.771 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.767+0000 7fe9c259c700 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f292d0f010 0x55f292d11420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48870cc812c06fc server_cookie=af7f7cad77a993dc in_seq=0 out_seq=0 2023-11-27T15:29:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.771+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/626424097 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f292c09040 con 0x55f292d0f010 2023-11-27T15:29:18.774 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.771+0000 7fe9a6ff5700 10 client.15588 handle_command_reply: tid=0 2023-11-27T15:29:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.771+0000 7fe9dcaf2b80 1 client.15588 shutdown 2023-11-27T15:29:18.775 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.772+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/626424097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 msgr2=0x55f292d03290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.772+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d03290 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40045d0 tx=0x7fe9b4009260 comp rx=0 tx=0).stop 2023-11-27T15:29:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.772+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/626424097 shutdown_connections 2023-11-27T15:29:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.772+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfb170 0x55f292d0a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.772+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d03290 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.772+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f292d0f010 0x55f292d11420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.772+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/626424097 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.772+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/626424097 >> 172.21.15.8:0/626424097 conn(0x55f292ceb1a0 msgr2=0x55f292cefa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:18.776 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.772+0000 7fe9a6ff5700 0 client.15588 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.773+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/626424097 shutdown_connections 2023-11-27T15:29:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.773+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/626424097 wait complete. 2023-11-27T15:29:18.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.773+0000 7fe9dcaf2b80 20 client.15588 trim_cache size 0 max 16384 2023-11-27T15:29:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.799+0000 7fe9dcaf2b80 1 Processor -- start 2023-11-27T15:29:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.799+0000 7fe9dcaf2b80 1 -- start start 2023-11-27T15:29:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.799+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d15da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.799+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d0a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.800+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfb170 0x55f292cfe080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.800+0000 7fe9c259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d0a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.800+0000 7fe9c259c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d0a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:44416/0 (socket says 172.21.15.8:44416) 2023-11-27T15:29:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.800+0000 7fe9c259c700 1 -- 172.21.15.8:0/2289095679 learned_addr learned my addr 172.21.15.8:0/2289095679 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.800+0000 7fe9c359e700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfb170 0x55f292cfe080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.800+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/2289095679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292cf11c0 con 0x55f292b516b0 2023-11-27T15:29:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.800+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/2289095679 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292d02740 con 0x55f292cfb170 2023-11-27T15:29:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.800+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/2289095679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292cf1dd0 con 0x55f292cfa3c0 2023-11-27T15:29:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.800+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/2289095679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d15da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.801+0000 7fe9c259c700 1 -- 172.21.15.8:0/2289095679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfb170 msgr2=0x55f292cfe080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.801+0000 7fe9c259c700 1 --2- 172.21.15.8:0/2289095679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfb170 0x55f292cfe080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.801+0000 7fe9c259c700 1 -- 172.21.15.8:0/2289095679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 msgr2=0x55f292d15da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.801+0000 7fe9c259c700 1 --2- 172.21.15.8:0/2289095679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d15da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.801+0000 7fe9c259c700 1 -- 172.21.15.8:0/2289095679 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f292c0ce80 con 0x55f292cfa3c0 2023-11-27T15:29:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.801+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/2289095679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d15da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.802+0000 7fe9c259c700 1 --2- 172.21.15.8:0/2289095679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d0a7d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009990 tx=0x7fe9b801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55143386ee3657e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.802+0000 7fe9a9ffb700 1 -- 172.21.15.8:0/2289095679 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b801d040 con 0x55f292cfa3c0 2023-11-27T15:29:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.802+0000 7fe9a9ffb700 1 -- 172.21.15.8:0/2289095679 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9b800be20 con 0x55f292cfa3c0 2023-11-27T15:29:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.802+0000 7fe9a9ffb700 1 -- 172.21.15.8:0/2289095679 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8004500 con 0x55f292cfa3c0 2023-11-27T15:29:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.803+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/2289095679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 msgr2=0x55f292d0a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.803+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/2289095679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d0a7d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009990 tx=0x7fe9b801a040 comp rx=0 tx=0).stop 2023-11-27T15:29:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.803+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/2289095679 shutdown_connections 2023-11-27T15:29:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.803+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/2289095679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292cfb170 0x55f292cfe080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.803+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/2289095679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292b516b0 0x55f292d15da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.803+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/2289095679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d0a7d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.803+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/2289095679 >> 172.21.15.8:0/2289095679 conn(0x55f292ceb1a0 msgr2=0x55f292cf02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.803+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/2289095679 shutdown_connections 2023-11-27T15:29:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.803+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/2289095679 wait complete. 2023-11-27T15:29:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.804+0000 7fe9dcaf2b80 1 -- start start 2023-11-27T15:29:18.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.804+0000 7fe9dcaf2b80 1 Processor -- start 2023-11-27T15:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.805+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292b516b0 0x55f292d08a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.805+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d08f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.805+0000 7fe9dcaf2b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292cfb170 0x55f292d09490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.805+0000 7fe9dcaf2b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292d09b70 con 0x55f292cfb170 2023-11-27T15:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.805+0000 7fe9dcaf2b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292d09cb0 con 0x55f292b516b0 2023-11-27T15:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.805+0000 7fe9dcaf2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f292d09e70 con 0x55f292cfa3c0 2023-11-27T15:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.805+0000 7fe9dcaf2b80 10 client.0 _ll_register_callbacks cb 0x7fe9b40043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.805+0000 7fe9c2d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292b516b0 0x55f292d08a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.805+0000 7fe9c2d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292b516b0 0x55f292d08a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:43406/0 (socket says 172.21.15.8:43406) 2023-11-27T15:29:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.805+0000 7fe9c2d9d700 1 -- 172.21.15.8:0/3242700596 learned_addr learned my addr 172.21.15.8:0/3242700596 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.806+0000 7fe9c2d9d700 1 -- 172.21.15.8:0/3242700596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 msgr2=0x55f292d08f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:29:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.806+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d08f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.806+0000 7fe9c2d9d700 1 -- 172.21.15.8:0/3242700596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292cfb170 msgr2=0x55f292d09490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:29:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.806+0000 7fe9c359e700 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292cfb170 0x55f292d09490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.807+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292cfb170 0x55f292d09490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.807+0000 7fe9c2d9d700 1 -- 172.21.15.8:0/3242700596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f292c0d130 con 0x55f292b516b0 2023-11-27T15:29:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.807+0000 7fe9c359e700 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292cfb170 0x55f292d09490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:29:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.807+0000 7fe9c2d9d700 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292b516b0 0x55f292d08a10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4013fd0 tx=0x7fe9b40036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d7614071063178c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.807+0000 7fe9a6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:29:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.807+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/3242700596 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4003b90 con 0x55f292b516b0 2023-11-27T15:29:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.807+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/3242700596 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe9b4003cf0 con 0x55f292b516b0 2023-11-27T15:29:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.807+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/3242700596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9b801e040 con 0x55f292b516b0 2023-11-27T15:29:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.808+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/3242700596 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9b4005040 con 0x55f292b516b0 2023-11-27T15:29:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.808+0000 7fe9dcaf2b80 10 client.25622 fetch_fsmap learned FSMap version 30 2023-11-27T15:29:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.808+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/3242700596 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f292ac85d0 con 0x55f292b516b0 2023-11-27T15:29:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.808+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/3242700596 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9b801e040 con 0x55f292b516b0 2023-11-27T15:29:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.809+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/3242700596 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe9b4004990 con 0x55f292b516b0 2023-11-27T15:29:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.809+0000 7fe9dcaf2b80 10 client.25622 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:29:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.809+0000 7fe9dcaf2b80 10 client.25622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:29:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.809+0000 7fe9dcaf2b80 20 client.25622 populate_metadata read hostname 'smithi008' 2023-11-27T15:29:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.809+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f292d63a70 0x55f292d65e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.809+0000 7fe9dcaf2b80 4 client.25622 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:29:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.809+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/3242700596 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f292c15600 con 0x55f292d63a70 2023-11-27T15:29:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.810+0000 7fe9c259c700 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f292d63a70 0x55f292d65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.811+0000 7fe9c259c700 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f292d63a70 0x55f292d65e80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f7f787f15adecc server_cookie=2255e73be228f037 in_seq=0 out_seq=0 2023-11-27T15:29:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:18.811+0000 7fe9a6ff5700 10 client.25622 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:19.615 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:19 smithi008 ceph-mon[128788]: pgmap v462: 129 pgs: 129 active+clean; 155 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 569 op/s 2023-11-27T15:29:19.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:19 smithi029 ceph-mon[131166]: pgmap v462: 129 pgs: 129 active+clean; 155 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 569 op/s 2023-11-27T15:29:19.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:19 smithi060 ceph-mon[131518]: pgmap v462: 129 pgs: 129 active+clean; 155 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 569 op/s 2023-11-27T15:29:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.179+0000 7fe9a6ff5700 1 -- 172.21.15.8:0/3242700596 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f292c15600 con 0x55f292d63a70 2023-11-27T15:29:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.179+0000 7fe9a6ff5700 10 client.25622 handle_command_reply: tid=0 2023-11-27T15:29:20.183 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.179+0000 7fe9dcaf2b80 1 client.25622 shutdown 2023-11-27T15:29:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.179+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/3242700596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292b516b0 msgr2=0x55f292d08a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.179+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292b516b0 0x55f292d08a10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4013fd0 tx=0x7fe9b40036f0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.179+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/3242700596 shutdown_connections 2023-11-27T15:29:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.179+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55f292cfb170 0x55f292d09490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.184 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.180+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55f292b516b0 0x55f292d08a10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.180+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55f292d63a70 0x55f292d65e80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.180+0000 7fe9dcaf2b80 1 --2- 172.21.15.8:0/3242700596 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55f292cfa3c0 0x55f292d08f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.180+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/3242700596 >> 172.21.15.8:0/3242700596 conn(0x55f292ceb1a0 msgr2=0x55f292d15da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.180+0000 7fe9a6ff5700 0 client.25622 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.180+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/3242700596 shutdown_connections 2023-11-27T15:29:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.180+0000 7fe9dcaf2b80 1 -- 172.21.15.8:0/3242700596 wait complete. 2023-11-27T15:29:20.185 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.180+0000 7fe9dcaf2b80 20 client.25622 trim_cache size 0 max 16384 2023-11-27T15:29:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:29:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:29:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:29:20.186 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:29:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.181+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1076289098 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9ac0a11d0 msgr2=0x7fe9ac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.181+0000 7fe9c89fe700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9ac0a11d0 0x7fe9ac0a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55f292cf0ec0 tx=0x7fe9b40098c0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.181+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1076289098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9c4105db0 msgr2=0x7fe9c41a5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.181+0000 7fe9c89fe700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9c4105db0 0x7fe9c41a5fd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00c3a0 tx=0x7fe9bc00c6b0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.181+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1076289098 shutdown_connections 2023-11-27T15:29:20.187 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.181+0000 7fe9c89fe700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe9c4105db0 0x7fe9c41a5fd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.181+0000 7fe9c89fe700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe9c4100b70 0x7fe9c406f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.181+0000 7fe9c89fe700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe9ac0a11d0 0x7fe9ac0a3680 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.181+0000 7fe9c89fe700 1 --2- 172.21.15.8:0/1076289098 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe9c4103490 0x7fe9c406f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.181+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1076289098 >> 172.21.15.8:0/1076289098 conn(0x7fe9c40fa550 msgr2=0x7fe9c4101850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.182+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1076289098 shutdown_connections 2023-11-27T15:29:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:20.182+0000 7fe9c89fe700 1 -- 172.21.15.8:0/1076289098 wait complete. 2023-11-27T15:29:20.657 INFO:tasks.workunit.client.0.smithi008.stdout:created hierarchy, now cleaning up 2023-11-27T15:29:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:21 smithi060 ceph-mon[131518]: pgmap v463: 129 pgs: 129 active+clean; 162 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 164 op/s 2023-11-27T15:29:21.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:21 smithi008 ceph-mon[128788]: pgmap v463: 129 pgs: 129 active+clean; 162 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 164 op/s 2023-11-27T15:29:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:21 smithi029 ceph-mon[131166]: pgmap v463: 129 pgs: 129 active+clean; 162 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 164 op/s 2023-11-27T15:29:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:23 smithi060 ceph-mon[131518]: pgmap v464: 129 pgs: 129 active+clean; 156 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 521 op/s 2023-11-27T15:29:23.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:23 smithi008 ceph-mon[128788]: pgmap v464: 129 pgs: 129 active+clean; 156 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 521 op/s 2023-11-27T15:29:23.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:23 smithi029 ceph-mon[131166]: pgmap v464: 129 pgs: 129 active+clean; 156 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 521 op/s 2023-11-27T15:29:25.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:25 smithi060 ceph-mon[131518]: pgmap v465: 129 pgs: 129 active+clean; 151 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 731 op/s 2023-11-27T15:29:25.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:25 smithi008 ceph-mon[128788]: pgmap v465: 129 pgs: 129 active+clean; 151 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 731 op/s 2023-11-27T15:29:25.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:25 smithi029 ceph-mon[131166]: pgmap v465: 129 pgs: 129 active+clean; 151 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 731 op/s 2023-11-27T15:29:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:27 smithi060 ceph-mon[131518]: pgmap v466: 129 pgs: 129 active+clean; 154 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 758 op/s 2023-11-27T15:29:27.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:27 smithi008 ceph-mon[128788]: pgmap v466: 129 pgs: 129 active+clean; 154 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 758 op/s 2023-11-27T15:29:27.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:27 smithi029 ceph-mon[131166]: pgmap v466: 129 pgs: 129 active+clean; 154 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 758 op/s 2023-11-27T15:29:29.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:29 smithi008 ceph-mon[128788]: pgmap v467: 129 pgs: 129 active+clean; 155 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 805 op/s 2023-11-27T15:29:29.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:29 smithi029 ceph-mon[131166]: pgmap v467: 129 pgs: 129 active+clean; 155 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 805 op/s 2023-11-27T15:29:29.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:29 smithi060 ceph-mon[131518]: pgmap v467: 129 pgs: 129 active+clean; 155 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 805 op/s 2023-11-27T15:29:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:31 smithi060 ceph-mon[131518]: pgmap v468: 129 pgs: 129 active+clean; 160 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 829 op/s 2023-11-27T15:29:31.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:31 smithi008 ceph-mon[128788]: pgmap v468: 129 pgs: 129 active+clean; 160 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 829 op/s 2023-11-27T15:29:31.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:31 smithi029 ceph-mon[131166]: pgmap v468: 129 pgs: 129 active+clean; 160 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 829 op/s 2023-11-27T15:29:33.403 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.399+0000 7f4f577ff700 1 -- 172.21.15.8:0/2848943655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f50107d60 msgr2=0x7f4f50108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.399+0000 7f4f577ff700 1 --2- 172.21.15.8:0/2848943655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f50107d60 0x7f4f50108170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400b0a0 tx=0x7f4f4400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.399+0000 7f4f577ff700 1 -- 172.21.15.8:0/2848943655 shutdown_connections 2023-11-27T15:29:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.399+0000 7f4f577ff700 1 --2- 172.21.15.8:0/2848943655 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f500721e0 0x7f4f50117a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.399+0000 7f4f577ff700 1 --2- 172.21.15.8:0/2848943655 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f50107d60 0x7f4f50108170 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.399+0000 7f4f577ff700 1 --2- 172.21.15.8:0/2848943655 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f50071830 0x7f4f50071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.399+0000 7f4f577ff700 1 -- 172.21.15.8:0/2848943655 >> 172.21.15.8:0/2848943655 conn(0x7f4f5006d480 msgr2=0x7f4f5006f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:33.404 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.400+0000 7f4f577ff700 1 -- 172.21.15.8:0/2848943655 shutdown_connections 2023-11-27T15:29:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.400+0000 7f4f577ff700 1 -- 172.21.15.8:0/2848943655 wait complete. 2023-11-27T15:29:33.405 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.401+0000 7f4f577ff700 1 Processor -- start 2023-11-27T15:29:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.401+0000 7f4f577ff700 1 -- start start 2023-11-27T15:29:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.402+0000 7f4f577ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f50071830 0x7f4f501a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.402+0000 7f4f577ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f500721e0 0x7f4f501a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.402+0000 7f4f577ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f50107d60 0x7f4f501ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.402+0000 7f4f577ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501ac9b0 con 0x7f4f50107d60 2023-11-27T15:29:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.402+0000 7f4f577ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501acaf0 con 0x7f4f50071830 2023-11-27T15:29:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.402+0000 7f4f577ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501acc80 con 0x7f4f500721e0 2023-11-27T15:29:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.402+0000 7f4f54d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f500721e0 0x7f4f501a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.402+0000 7f4f54d9a700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f500721e0 0x7f4f501a5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:49312/0 (socket says 172.21.15.8:49312) 2023-11-27T15:29:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.402+0000 7f4f54d9a700 1 -- 172.21.15.8:0/1455976537 learned_addr learned my addr 172.21.15.8:0/1455976537 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.403+0000 7f4f55d9c700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f50107d60 0x7f4f501ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.403+0000 7f4f5559b700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f50071830 0x7f4f501a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.403+0000 7f4f54d9a700 1 -- 172.21.15.8:0/1455976537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f50071830 msgr2=0x7f4f501a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.403+0000 7f4f54d9a700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f50071830 0x7f4f501a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.403+0000 7f4f54d9a700 1 -- 172.21.15.8:0/1455976537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f50107d60 msgr2=0x7f4f501ac3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.403+0000 7f4f54d9a700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f50107d60 0x7f4f501ac3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.403+0000 7f4f54d9a700 1 -- 172.21.15.8:0/1455976537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44009d00 con 0x7f4f500721e0 2023-11-27T15:29:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.404+0000 7f4f54d9a700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f500721e0 0x7f4f501a5370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b550 tx=0x7f4f4c00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.404+0000 7f4f427fc700 1 -- 172.21.15.8:0/1455976537 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c011830 con 0x7f4f500721e0 2023-11-27T15:29:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.404+0000 7f4f427fc700 1 -- 172.21.15.8:0/1455976537 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f4c011e50 con 0x7f4f500721e0 2023-11-27T15:29:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.404+0000 7f4f427fc700 1 -- 172.21.15.8:0/1455976537 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00f2e0 con 0x7f4f500721e0 2023-11-27T15:29:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.404+0000 7f4f577ff700 1 -- 172.21.15.8:0/1455976537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f5011aa00 con 0x7f4f500721e0 2023-11-27T15:29:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.405+0000 7f4f577ff700 1 -- 172.21.15.8:0/1455976537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5011a870 con 0x7f4f500721e0 2023-11-27T15:29:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.409+0000 7f4f427fc700 1 -- 172.21.15.8:0/1455976537 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4f4c011990 con 0x7f4f500721e0 2023-11-27T15:29:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.413+0000 7f4f427fc700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4f3c0a11d0 0x7f4f3c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.413+0000 7f4f427fc700 1 -- 172.21.15.8:0/1455976537 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4f4c0cc6a0 con 0x7f4f500721e0 2023-11-27T15:29:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.414+0000 7f4f6b052b80 1 Processor -- start 2023-11-27T15:29:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.415+0000 7f4f6b052b80 1 -- start start 2023-11-27T15:29:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.415+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 0x56477b07d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.415+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b07e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.415+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b07ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.415+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477af83000 con 0x56477b07ddb0 2023-11-27T15:29:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.415+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477afd5840 con 0x56477b07eb10 2023-11-27T15:29:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.415+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477afcecd0 con 0x56477aed96b0 2023-11-27T15:29:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.416+0000 7f4f5559b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 0x56477b07d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.416+0000 7f4f5559b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 0x56477b07d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:49318/0 (socket says 172.21.15.8:49318) 2023-11-27T15:29:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.416+0000 7f4f5559b700 1 -- 172.21.15.8:0/935122755 learned_addr learned my addr 172.21.15.8:0/935122755 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.416+0000 7f4f5559b700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4f3c0a11d0 0x7f4f3c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.417+0000 7f4f54d9a700 1 --2- 172.21.15.8:0/935122755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b07e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.417+0000 7f4f55d9c700 1 --2- 172.21.15.8:0/935122755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b07ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.417+0000 7f4f5559b700 1 -- 172.21.15.8:0/935122755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 msgr2=0x56477b07ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.417+0000 7f4f5559b700 1 --2- 172.21.15.8:0/935122755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b07ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.417+0000 7f4f5559b700 1 -- 172.21.15.8:0/935122755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 msgr2=0x56477b07e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.417+0000 7f4f5559b700 1 --2- 172.21.15.8:0/935122755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b07e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.417+0000 7f4f5559b700 1 -- 172.21.15.8:0/935122755 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56477ae505d0 con 0x56477aed96b0 2023-11-27T15:29:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.418+0000 7f4f54d9a700 1 --2- 172.21.15.8:0/935122755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b07e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:29:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.419+0000 7f4f5559b700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4f3c0a11d0 0x7f4f3c0a3680 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009cd0 tx=0x7f4f44013000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.420+0000 7f4f5559b700 1 --2- 172.21.15.8:0/935122755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 0x56477b07d870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400b0a0 tx=0x7f4f44004b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5692d6ea2a6d7eb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.420+0000 7f4f3b7fe700 1 -- 172.21.15.8:0/935122755 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400e040 con 0x56477aed96b0 2023-11-27T15:29:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.420+0000 7f4f3b7fe700 1 -- 172.21.15.8:0/935122755 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f44007870 con 0x56477aed96b0 2023-11-27T15:29:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.421+0000 7f4f3b7fe700 1 -- 172.21.15.8:0/935122755 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4401e690 con 0x56477aed96b0 2023-11-27T15:29:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.422+0000 7f4f6b052b80 1 -- 172.21.15.8:0/935122755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 msgr2=0x56477b07d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.422+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/935122755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 0x56477b07d870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400b0a0 tx=0x7f4f44004b10 comp rx=0 tx=0).stop 2023-11-27T15:29:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.423+0000 7f4f6b052b80 1 -- 172.21.15.8:0/935122755 shutdown_connections 2023-11-27T15:29:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.423+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/935122755 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 0x56477b07d870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.423+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/935122755 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b07ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.423+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/935122755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b07e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.423+0000 7f4f6b052b80 1 -- 172.21.15.8:0/935122755 >> 172.21.15.8:0/935122755 conn(0x56477b072f50 msgr2=0x56477b078100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.424+0000 7f4f6b052b80 1 -- 172.21.15.8:0/935122755 shutdown_connections 2023-11-27T15:29:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.424+0000 7f4f6b052b80 1 -- 172.21.15.8:0/935122755 wait complete. 2023-11-27T15:29:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.425+0000 7f4f6b052b80 1 -- start start 2023-11-27T15:29:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.425+0000 7f4f6b052b80 1 Processor -- start 2023-11-27T15:29:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.427+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477aed96b0 0x56477b086c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.427+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b087150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.427+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07eb10 0x56477b08e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.427+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477af70880 con 0x56477b07ddb0 2023-11-27T15:29:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.427+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477af70520 con 0x56477aed96b0 2023-11-27T15:29:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.427+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477ae528c0 con 0x56477b07eb10 2023-11-27T15:29:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.427+0000 7f4f6b052b80 10 client.0 _ll_register_callbacks cb 0x56477af8e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:29:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.428+0000 7f4f55d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07eb10 0x56477b08e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.428+0000 7f4f5559b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477aed96b0 0x56477b086c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.428+0000 7f4f55d9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07eb10 0x56477b08e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:49330/0 (socket says 172.21.15.8:49330) 2023-11-27T15:29:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.428+0000 7f4f55d9c700 1 -- 172.21.15.8:0/3877615683 learned_addr learned my addr 172.21.15.8:0/3877615683 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.428+0000 7f4f55d9c700 1 -- 172.21.15.8:0/3877615683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477aed96b0 msgr2=0x56477b086c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.428+0000 7f4f55d9c700 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477aed96b0 0x56477b086c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.428+0000 7f4f55d9c700 1 -- 172.21.15.8:0/3877615683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 msgr2=0x56477b087150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:29:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.428+0000 7f4f55d9c700 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b087150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.428+0000 7f4f55d9c700 1 -- 172.21.15.8:0/3877615683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f50117f70 con 0x56477b07eb10 2023-11-27T15:29:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.428+0000 7f4f5559b700 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477aed96b0 0x56477b086c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:29:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.429+0000 7f4f55d9c700 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07eb10 0x56477b08e190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f48003930 tx=0x7f4f48003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa96c8707853045 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.429+0000 7f4f38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:29:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.429+0000 7f4f38ff9700 1 -- 172.21.15.8:0/3877615683 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48004210 con 0x56477b07eb10 2023-11-27T15:29:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.429+0000 7f4f38ff9700 1 -- 172.21.15.8:0/3877615683 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4f4801b440 con 0x56477b07eb10 2023-11-27T15:29:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.429+0000 7f4f6b052b80 1 -- 172.21.15.8:0/3877615683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56477b070710 con 0x56477b07eb10 2023-11-27T15:29:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.430+0000 7f4f38ff9700 1 -- 172.21.15.8:0/3877615683 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f4801b600 con 0x56477b07eb10 2023-11-27T15:29:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.430+0000 7f4f6b052b80 10 client.25651 fetch_fsmap learned FSMap version 30 2023-11-27T15:29:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.430+0000 7f4f6b052b80 1 -- 172.21.15.8:0/3877615683 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56477af95130 con 0x56477b07eb10 2023-11-27T15:29:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.430+0000 7f4f38ff9700 1 -- 172.21.15.8:0/3877615683 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56477b070710 con 0x56477b07eb10 2023-11-27T15:29:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.431+0000 7f4f38ff9700 1 -- 172.21.15.8:0/3877615683 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4f48011940 con 0x56477b07eb10 2023-11-27T15:29:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.431+0000 7f4f6b052b80 10 client.25651 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:29:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.431+0000 7f4f6b052b80 10 client.25651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:29:33.436 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.431+0000 7f4f6b052b80 20 client.25651 populate_metadata read hostname 'smithi008' 2023-11-27T15:29:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.431+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56477b0929c0 0x56477b094dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.431+0000 7f4f6b052b80 4 client.25651 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:29:33.437 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.431+0000 7f4f6b052b80 1 -- 172.21.15.8:0/3877615683 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56477af91040 con 0x56477b0929c0 2023-11-27T15:29:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.432+0000 7f4f5559b700 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56477b0929c0 0x56477b094dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.434+0000 7f4f5559b700 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56477b0929c0 0x56477b094dd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f58f08092c7ea19 server_cookie=a85edadf356e1b5b in_seq=0 out_seq=0 2023-11-27T15:29:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.434+0000 7f4f38ff9700 10 client.25651 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.447+0000 7f4f38ff9700 1 -- 172.21.15.8:0/3877615683 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56477af91040 con 0x56477b0929c0 2023-11-27T15:29:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.447+0000 7f4f38ff9700 10 client.25651 handle_command_reply: tid=0 2023-11-27T15:29:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.447+0000 7f4f6b052b80 1 client.25651 shutdown 2023-11-27T15:29:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.448+0000 7f4f6b052b80 1 -- 172.21.15.8:0/3877615683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07eb10 msgr2=0x56477b08e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.448+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07eb10 0x56477b08e190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f48003930 tx=0x7f4f48003cf0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.448+0000 7f4f6b052b80 1 -- 172.21.15.8:0/3877615683 shutdown_connections 2023-11-27T15:29:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.448+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56477b0929c0 0x56477b094dd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.448+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07eb10 0x56477b08e190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.448+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477aed96b0 0x56477b086c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.448+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/3877615683 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b087150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.448+0000 7f4f6b052b80 1 -- 172.21.15.8:0/3877615683 >> 172.21.15.8:0/3877615683 conn(0x56477b072f50 msgr2=0x56477b0777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.448+0000 7f4f38ff9700 0 client.25651 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.449+0000 7f4f6b052b80 1 -- 172.21.15.8:0/3877615683 shutdown_connections 2023-11-27T15:29:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.449+0000 7f4f6b052b80 1 -- 172.21.15.8:0/3877615683 wait complete. 2023-11-27T15:29:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.449+0000 7f4f6b052b80 20 client.25651 trim_cache size 0 max 16384 2023-11-27T15:29:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.470+0000 7f4f6b052b80 1 Processor -- start 2023-11-27T15:29:33.474 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.471+0000 7f4f6b052b80 1 -- start start 2023-11-27T15:29:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.471+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 0x56477b084e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.471+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b08e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.471+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b085360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.471+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477b0860c0 con 0x56477b07ddb0 2023-11-27T15:29:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.471+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477b086780 con 0x56477b07eb10 2023-11-27T15:29:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.471+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477b078aa0 con 0x56477aed96b0 2023-11-27T15:29:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.471+0000 7f4f55d9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b085360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.471+0000 7f4f55d9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b085360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:34448/0 (socket says 172.21.15.8:34448) 2023-11-27T15:29:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.471+0000 7f4f55d9c700 1 -- 172.21.15.8:0/4114307670 learned_addr learned my addr 172.21.15.8:0/4114307670 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.472+0000 7f4f54d9a700 1 --2- 172.21.15.8:0/4114307670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b08e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.472+0000 7f4f5559b700 1 --2- 172.21.15.8:0/4114307670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 0x56477b084e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.472+0000 7f4f55d9c700 1 -- 172.21.15.8:0/4114307670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 msgr2=0x56477b084e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.472+0000 7f4f55d9c700 1 --2- 172.21.15.8:0/4114307670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 0x56477b084e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.472+0000 7f4f55d9c700 1 -- 172.21.15.8:0/4114307670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 msgr2=0x56477b08e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.472+0000 7f4f55d9c700 1 --2- 172.21.15.8:0/4114307670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b08e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.472+0000 7f4f55d9c700 1 -- 172.21.15.8:0/4114307670 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56477af94e80 con 0x56477b07eb10 2023-11-27T15:29:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.472+0000 7f4f55d9c700 1 --2- 172.21.15.8:0/4114307670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b085360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f48003900 tx=0x7f4f4800ba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f6b5df746209afd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.472+0000 7f4f3bfff700 1 -- 172.21.15.8:0/4114307670 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48005930 con 0x56477b07eb10 2023-11-27T15:29:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.473+0000 7f4f3bfff700 1 -- 172.21.15.8:0/4114307670 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f48005c80 con 0x56477b07eb10 2023-11-27T15:29:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.473+0000 7f4f3bfff700 1 -- 172.21.15.8:0/4114307670 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48004680 con 0x56477b07eb10 2023-11-27T15:29:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.473+0000 7f4f6b052b80 1 -- 172.21.15.8:0/4114307670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 msgr2=0x56477b085360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.473+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/4114307670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b085360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4f48003900 tx=0x7f4f4800ba80 comp rx=0 tx=0).stop 2023-11-27T15:29:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.474+0000 7f4f6b052b80 1 -- 172.21.15.8:0/4114307670 shutdown_connections 2023-11-27T15:29:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.474+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/4114307670 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477aed96b0 0x56477b084e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.474+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/4114307670 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b085360 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.474+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/4114307670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477b07ddb0 0x56477b08e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.474+0000 7f4f6b052b80 1 -- 172.21.15.8:0/4114307670 >> 172.21.15.8:0/4114307670 conn(0x56477b072f50 msgr2=0x56477b0996f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:33.479 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.474+0000 7f4f6b052b80 1 -- 172.21.15.8:0/4114307670 shutdown_connections 2023-11-27T15:29:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.474+0000 7f4f6b052b80 1 -- 172.21.15.8:0/4114307670 wait complete. 2023-11-27T15:29:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.475+0000 7f4f6b052b80 1 -- start start 2023-11-27T15:29:33.480 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.475+0000 7f4f6b052b80 1 Processor -- start 2023-11-27T15:29:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.475+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477aed96b0 0x56477b089bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.475+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07ddb0 0x56477b08a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.475+0000 7f4f6b052b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b08d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.475+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477b08d650 con 0x56477aed96b0 2023-11-27T15:29:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.475+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477b08d7c0 con 0x56477b07eb10 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.475+0000 7f4f6b052b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477b08d930 con 0x56477b07ddb0 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f6b052b80 10 client.0 _ll_register_callbacks cb 0x7f4f48004400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f5559b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477aed96b0 0x56477b089bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f5559b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477aed96b0 0x56477b089bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58694/0 (socket says 172.21.15.8:58694) 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f5559b700 1 -- 172.21.15.8:0/715177515 learned_addr learned my addr 172.21.15.8:0/715177515 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f5559b700 1 -- 172.21.15.8:0/715177515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07ddb0 msgr2=0x56477b08a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f5559b700 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07ddb0 0x56477b08a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f55d9c700 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b08d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f5559b700 1 -- 172.21.15.8:0/715177515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 msgr2=0x56477b08d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f5559b700 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b08d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f5559b700 1 -- 172.21.15.8:0/715177515 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56477af95130 con 0x56477aed96b0 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.476+0000 7f4f55d9c700 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b08d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.477+0000 7f4f5559b700 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477aed96b0 0x56477b089bb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009fd0 tx=0x7f4f440099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda4941f8962c5ea server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.477+0000 7f4f38ff9700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.477+0000 7f4f38ff9700 1 -- 172.21.15.8:0/715177515 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400e040 con 0x56477aed96b0 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.477+0000 7f4f38ff9700 1 -- 172.21.15.8:0/715177515 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4f4401e4b0 con 0x56477aed96b0 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.477+0000 7f4f6b052b80 1 -- 172.21.15.8:0/715177515 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f44007870 con 0x56477aed96b0 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f38ff9700 1 -- 172.21.15.8:0/715177515 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f44005a70 con 0x56477aed96b0 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f6b052b80 10 client.15603 fetch_fsmap learned FSMap version 30 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f6b052b80 1 -- 172.21.15.8:0/715177515 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56477ade5c00 con 0x56477aed96b0 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f38ff9700 1 -- 172.21.15.8:0/715177515 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f44007870 con 0x56477aed96b0 2023-11-27T15:29:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f38ff9700 1 -- 172.21.15.8:0/715177515 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4f440052b0 con 0x56477aed96b0 2023-11-27T15:29:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f6b052b80 10 client.15603 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:29:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f6b052b80 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:29:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f6b052b80 20 client.15603 populate_metadata read hostname 'smithi008' 2023-11-27T15:29:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56477b0f8310 0x56477b0fa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f6b052b80 4 client.15603 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:29:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.478+0000 7f4f6b052b80 1 -- 172.21.15.8:0/715177515 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56477af9d600 con 0x56477b0f8310 2023-11-27T15:29:33.485 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.479+0000 7f4f54d9a700 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56477b0f8310 0x56477b0fa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.479+0000 7f4f54d9a700 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56477b0f8310 0x56477b0fa720 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2496ec764ea164f0 server_cookie=59ca363eee374de9 in_seq=0 out_seq=0 2023-11-27T15:29:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:33.479+0000 7f4f38ff9700 10 client.15603 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:33 smithi060 ceph-mon[131518]: pgmap v469: 129 pgs: 129 active+clean; 171 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 898 op/s 2023-11-27T15:29:33.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:33 smithi008 ceph-mon[128788]: pgmap v469: 129 pgs: 129 active+clean; 171 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 898 op/s 2023-11-27T15:29:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:33 smithi029 ceph-mon[131166]: pgmap v469: 129 pgs: 129 active+clean; 171 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 898 op/s 2023-11-27T15:29:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.984+0000 7f4f38ff9700 1 -- 172.21.15.8:0/715177515 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56477af9d600 con 0x56477b0f8310 2023-11-27T15:29:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.984+0000 7f4f38ff9700 10 client.15603 handle_command_reply: tid=0 2023-11-27T15:29:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.984+0000 7f4f6b052b80 1 client.15603 shutdown 2023-11-27T15:29:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.984+0000 7f4f6b052b80 1 -- 172.21.15.8:0/715177515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477aed96b0 msgr2=0x56477b089bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.984+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477aed96b0 0x56477b089bb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009fd0 tx=0x7f4f440099e0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.984+0000 7f4f6b052b80 1 -- 172.21.15.8:0/715177515 shutdown_connections 2023-11-27T15:29:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.984+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56477b07eb10 0x56477b08d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.984+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56477aed96b0 0x56477b089bb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.984+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56477b0f8310 0x56477b0fa720 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.985+0000 7f4f6b052b80 1 --2- 172.21.15.8:0/715177515 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56477b07ddb0 0x56477b08a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.985+0000 7f4f6b052b80 1 -- 172.21.15.8:0/715177515 >> 172.21.15.8:0/715177515 conn(0x56477b072f50 msgr2=0x56477b084e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.985+0000 7f4f38ff9700 0 client.15603 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.985+0000 7f4f6b052b80 1 -- 172.21.15.8:0/715177515 shutdown_connections 2023-11-27T15:29:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.985+0000 7f4f6b052b80 1 -- 172.21.15.8:0/715177515 wait complete. 2023-11-27T15:29:34.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.985+0000 7f4f6b052b80 20 client.15603 trim_cache size 0 max 16384 2023-11-27T15:29:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:29:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:29:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:29:34.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:29:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 -- 172.21.15.8:0/1455976537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4f3c0a11d0 msgr2=0x7f4f3c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4f3c0a11d0 0x7f4f3c0a3680 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009cd0 tx=0x7f4f44013000 comp rx=0 tx=0).stop 2023-11-27T15:29:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 -- 172.21.15.8:0/1455976537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f500721e0 msgr2=0x7f4f501a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f500721e0 0x7f4f501a5370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00b550 tx=0x7f4f4c00b860 comp rx=0 tx=0).stop 2023-11-27T15:29:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 -- 172.21.15.8:0/1455976537 shutdown_connections 2023-11-27T15:29:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f50107d60 0x7f4f501ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4f50071830 0x7f4f501a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f4f3c0a11d0 0x7f4f3c0a3680 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 --2- 172.21.15.8:0/1455976537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4f500721e0 0x7f4f501a5370 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 -- 172.21.15.8:0/1455976537 >> 172.21.15.8:0/1455976537 conn(0x7f4f5006d480 msgr2=0x7f4f5006f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 -- 172.21.15.8:0/1455976537 shutdown_connections 2023-11-27T15:29:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:34.986+0000 7f4f577ff700 1 -- 172.21.15.8:0/1455976537 wait complete. 2023-11-27T15:29:35.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:35 smithi060 ceph-mon[131518]: pgmap v470: 129 pgs: 129 active+clean; 171 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 488 op/s 2023-11-27T15:29:35.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:35 smithi008 ceph-mon[128788]: pgmap v470: 129 pgs: 129 active+clean; 171 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 488 op/s 2023-11-27T15:29:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:35 smithi029 ceph-mon[131166]: pgmap v470: 129 pgs: 129 active+clean; 171 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 488 op/s 2023-11-27T15:29:36.548 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: *** Caught signal (Segmentation fault) ** 2023-11-27T15:29:36.548 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: in thread 7f0c05583700 thread_name:md_log_replay 2023-11-27T15:29:36.548 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-27T15:29:36.548 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f0c16132cf0] 2023-11-27T15:29:36.548 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2: (operator<<(std::ostream&, CInode const&)+0xd37) [0x56373c6d0b07] 2023-11-27T15:29:36.548 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 3: (CInode::maybe_export_pin(bool)+0x1d4) [0x56373c6ecf94] 2023-11-27T15:29:36.549 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 4: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x249) [0x56373c7f1709] 2023-11-27T15:29:36.549 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 5: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x8de) [0x56373c7f645e] 2023-11-27T15:29:36.549 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 6: (EUpdate::replay(MDSRank*)+0x61) [0x56373c7fc631] 2023-11-27T15:29:36.549 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 7: (MDLog::_replay_thread()+0x66d) [0x56373c783ebd] 2023-11-27T15:29:36.549 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 8: (MDLog::ReplayThread::entry()+0x11) [0x56373c433481] 2023-11-27T15:29:36.549 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f0c161281ca] 2023-11-27T15:29:36.549 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 10: clone() 2023-11-27T15:29:36.549 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2023-11-27T15:29:36.123+0000 7f0c05583700 -1 *** Caught signal (Segmentation fault) ** 2023-11-27T15:29:36.549 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: in thread 7f0c05583700 thread_name:md_log_replay 2023-11-27T15:29:36.549 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2023-11-27T15:29:36.550 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-27T15:29:36.550 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f0c16132cf0] 2023-11-27T15:29:36.550 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2: (operator<<(std::ostream&, CInode const&)+0xd37) [0x56373c6d0b07] 2023-11-27T15:29:36.550 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 3: (CInode::maybe_export_pin(bool)+0x1d4) [0x56373c6ecf94] 2023-11-27T15:29:36.550 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 4: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x249) [0x56373c7f1709] 2023-11-27T15:29:36.550 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 5: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x8de) [0x56373c7f645e] 2023-11-27T15:29:36.550 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 6: (EUpdate::replay(MDSRank*)+0x61) [0x56373c7fc631] 2023-11-27T15:29:36.550 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 7: (MDLog::_replay_thread()+0x66d) [0x56373c783ebd] 2023-11-27T15:29:36.550 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 8: (MDLog::ReplayThread::entry()+0x11) [0x56373c433481] 2023-11-27T15:29:36.551 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f0c161281ca] 2023-11-27T15:29:36.551 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 10: clone() 2023-11-27T15:29:36.551 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-27T15:29:36.551 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2023-11-27T15:29:36.551 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 0> 2023-11-27T15:29:36.123+0000 7f0c05583700 -1 *** Caught signal (Segmentation fault) ** 2023-11-27T15:29:36.552 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: in thread 7f0c05583700 thread_name:md_log_replay 2023-11-27T15:29:36.552 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2023-11-27T15:29:36.552 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-27T15:29:36.552 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f0c16132cf0] 2023-11-27T15:29:36.552 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2: (operator<<(std::ostream&, CInode const&)+0xd37) [0x56373c6d0b07] 2023-11-27T15:29:36.552 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 3: (CInode::maybe_export_pin(bool)+0x1d4) [0x56373c6ecf94] 2023-11-27T15:29:36.552 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 4: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x249) [0x56373c7f1709] 2023-11-27T15:29:36.552 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 5: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x8de) [0x56373c7f645e] 2023-11-27T15:29:36.552 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 6: (EUpdate::replay(MDSRank*)+0x61) [0x56373c7fc631] 2023-11-27T15:29:36.552 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 7: (MDLog::_replay_thread()+0x66d) [0x56373c783ebd] 2023-11-27T15:29:36.553 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 8: (MDLog::ReplayThread::entry()+0x11) [0x56373c433481] 2023-11-27T15:29:36.553 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f0c161281ca] 2023-11-27T15:29:36.553 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 10: clone() 2023-11-27T15:29:36.553 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-27T15:29:36.553 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2023-11-27T15:29:36.553 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 0> 2023-11-27T15:29:36.123+0000 7f0c05583700 -1 *** Caught signal (Segmentation fault) ** 2023-11-27T15:29:36.553 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: in thread 7f0c05583700 thread_name:md_log_replay 2023-11-27T15:29:36.553 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2023-11-27T15:29:36.553 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-27T15:29:36.554 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f0c16132cf0] 2023-11-27T15:29:36.554 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2: (operator<<(std::ostream&, CInode const&)+0xd37) [0x56373c6d0b07] 2023-11-27T15:29:36.554 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 3: (CInode::maybe_export_pin(bool)+0x1d4) [0x56373c6ecf94] 2023-11-27T15:29:36.554 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 4: (EMetaBlob::fullbit::update_inode(MDSRank*, CInode*)+0x249) [0x56373c7f1709] 2023-11-27T15:29:36.554 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 5: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x8de) [0x56373c7f645e] 2023-11-27T15:29:36.554 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 6: (EUpdate::replay(MDSRank*)+0x61) [0x56373c7fc631] 2023-11-27T15:29:36.554 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 7: (MDLog::_replay_thread()+0x66d) [0x56373c783ebd] 2023-11-27T15:29:36.554 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 8: (MDLog::ReplayThread::entry()+0x11) [0x56373c433481] 2023-11-27T15:29:36.554 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 9: /lib64/libpthread.so.0(+0x81ca) [0x7f0c161281ca] 2023-11-27T15:29:36.555 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 10: clone() 2023-11-27T15:29:36.555 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-27T15:29:36.555 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[157804]: 2023-11-27T15:29:36.555 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i.service: Main process exited, code=exited, status=139/n/a 2023-11-27T15:29:36.824 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:36 smithi060 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i.service: Failed with result 'exit-code'. 2023-11-27T15:29:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:37 smithi060 ceph-mon[131518]: pgmap v471: 129 pgs: 129 active+clean; 173 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 371 op/s 2023-11-27T15:29:37.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:37 smithi008 ceph-mon[128788]: pgmap v471: 129 pgs: 129 active+clean; 173 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 371 op/s 2023-11-27T15:29:37.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:37 smithi029 ceph-mon[131166]: pgmap v471: 129 pgs: 129 active+clean; 173 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 371 op/s 2023-11-27T15:29:39.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:39 smithi008 ceph-mon[128788]: pgmap v472: 129 pgs: 129 active+clean; 175 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 476 op/s 2023-11-27T15:29:39.646 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:39 smithi029 ceph-mon[131166]: pgmap v472: 129 pgs: 129 active+clean; 175 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 476 op/s 2023-11-27T15:29:39.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:39 smithi060 ceph-mon[131518]: pgmap v472: 129 pgs: 129 active+clean; 175 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 476 op/s 2023-11-27T15:29:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:41 smithi060 ceph-mon[131518]: pgmap v473: 129 pgs: 129 active+clean; 174 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 505 op/s 2023-11-27T15:29:41.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:41 smithi008 ceph-mon[128788]: pgmap v473: 129 pgs: 129 active+clean; 174 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 505 op/s 2023-11-27T15:29:41.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:41 smithi029 ceph-mon[131166]: pgmap v473: 129 pgs: 129 active+clean; 174 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 505 op/s 2023-11-27T15:29:43.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:43 smithi060 ceph-mon[131518]: pgmap v474: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 555 op/s 2023-11-27T15:29:43.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:43 smithi008 ceph-mon[128788]: pgmap v474: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 555 op/s 2023-11-27T15:29:43.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:43 smithi029 ceph-mon[131166]: pgmap v474: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 555 op/s 2023-11-27T15:29:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:45 smithi060 ceph-mon[131518]: pgmap v475: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 510 op/s 2023-11-27T15:29:45.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:45 smithi008 ceph-mon[128788]: pgmap v475: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 510 op/s 2023-11-27T15:29:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:45 smithi029 ceph-mon[131166]: pgmap v475: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 510 op/s 2023-11-27T15:29:47.167 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:46 smithi060 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i.service: Service RestartSec=10s expired, scheduling restart. 2023-11-27T15:29:47.167 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:46 smithi060 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i.service: Scheduled restart job, restart counter is at 1. 2023-11-27T15:29:47.167 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:46 smithi060 systemd[1]: Stopped Ceph mds.i for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:29:47.167 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:46 smithi060 systemd[1]: Starting Ceph mds.i for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:29:47.429 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:47 smithi060 podman[160560]: 2023-11-27T15:29:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:47 smithi060 ceph-mon[131518]: pgmap v476: 129 pgs: 129 active+clean; 191 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 531 op/s 2023-11-27T15:29:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:47 smithi060 ceph-mon[131518]: daemon mds.i restarted 2023-11-27T15:29:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:47 smithi060 ceph-mon[131518]: osdmap e93: 12 total, 12 up, 12 in 2023-11-27T15:29:47.824 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:47 smithi060 bash[160560]: fcf858c8330553484fecc7c3fadbf221a40aacb5f8b63c8e4aca9195045da2b3 2023-11-27T15:29:47.824 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:47 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[160576]: starting mds.i at 2023-11-27T15:29:47.824 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:47 smithi060 systemd[1]: Started Ceph mds.i for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:29:47.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:47 smithi008 ceph-mon[128788]: pgmap v476: 129 pgs: 129 active+clean; 191 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 531 op/s 2023-11-27T15:29:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:47 smithi008 ceph-mon[128788]: daemon mds.i restarted 2023-11-27T15:29:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:47 smithi008 ceph-mon[128788]: osdmap e93: 12 total, 12 up, 12 in 2023-11-27T15:29:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:47 smithi029 ceph-mon[131166]: pgmap v476: 129 pgs: 129 active+clean; 191 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 531 op/s 2023-11-27T15:29:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:47 smithi029 ceph-mon[131166]: daemon mds.i restarted 2023-11-27T15:29:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:47 smithi029 ceph-mon[131166]: osdmap e93: 12 total, 12 up, 12 in 2023-11-27T15:29:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.150+0000 7f3220bff700 1 -- 172.21.15.8:0/930356332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321c100230 msgr2=0x7f321c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.150+0000 7f3220bff700 1 --2- 172.21.15.8:0/930356332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321c100230 0x7f321c100640 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f320c009990 tx=0x7f320c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.151+0000 7f3220bff700 1 -- 172.21.15.8:0/930356332 shutdown_connections 2023-11-27T15:29:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.151+0000 7f3220bff700 1 --2- 172.21.15.8:0/930356332 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321c100230 0x7f321c100640 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.151+0000 7f3220bff700 1 --2- 172.21.15.8:0/930356332 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321c101e70 0x7f321c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.151+0000 7f3220bff700 1 --2- 172.21.15.8:0/930356332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f321c1014c0 0x7f321c101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.151+0000 7f3220bff700 1 -- 172.21.15.8:0/930356332 >> 172.21.15.8:0/930356332 conn(0x7f321c0fb810 msgr2=0x7f321c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.151+0000 7f3220bff700 1 -- 172.21.15.8:0/930356332 shutdown_connections 2023-11-27T15:29:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.151+0000 7f3220bff700 1 -- 172.21.15.8:0/930356332 wait complete. 2023-11-27T15:29:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.152+0000 7f3220bff700 1 Processor -- start 2023-11-27T15:29:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.153+0000 7f3220bff700 1 -- start start 2023-11-27T15:29:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.153+0000 7f3220bff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321c100230 0x7f321c10f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.153+0000 7f3220bff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321c1014c0 0x7f321c10f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.153+0000 7f3220bff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f321c101e70 0x7f321c1146e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.153+0000 7f3220bff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c114cb0 con 0x7f321c101e70 2023-11-27T15:29:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.153+0000 7f3220bff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c114e20 con 0x7f321c1014c0 2023-11-27T15:29:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.153+0000 7f3220bff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c114fe0 con 0x7f321c100230 2023-11-27T15:29:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.154+0000 7f321ad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f321c101e70 0x7f321c1146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.154+0000 7f3211d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321c1014c0 0x7f321c10f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.154+0000 7f321ad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f321c101e70 0x7f321c1146e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47074/0 (socket says 172.21.15.8:47074) 2023-11-27T15:29:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.154+0000 7f321ad9d700 1 -- 172.21.15.8:0/422198302 learned_addr learned my addr 172.21.15.8:0/422198302 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.154+0000 7f321ad9d700 1 -- 172.21.15.8:0/422198302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321c100230 msgr2=0x7f321c10f140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-27T15:29:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.154+0000 7f321ad9d700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321c100230 0x7f321c10f140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.154+0000 7f321ad9d700 1 -- 172.21.15.8:0/422198302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321c1014c0 msgr2=0x7f321c10f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.154+0000 7f321ad9d700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321c1014c0 0x7f321c10f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.154+0000 7f321ad9d700 1 -- 172.21.15.8:0/422198302 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c0095f0 con 0x7f321c101e70 2023-11-27T15:29:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.155+0000 7f321ad9d700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f321c101e70 0x7f321c1146e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f321400e940 tx=0x7f321400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.155+0000 7f3213fff700 1 -- 172.21.15.8:0/422198302 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321400cbd0 con 0x7f321c101e70 2023-11-27T15:29:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.155+0000 7f3213fff700 1 -- 172.21.15.8:0/422198302 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f321400ee90 con 0x7f321c101e70 2023-11-27T15:29:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.155+0000 7f3220bff700 1 -- 172.21.15.8:0/422198302 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c109a50 con 0x7f321c101e70 2023-11-27T15:29:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.155+0000 7f3220bff700 1 -- 172.21.15.8:0/422198302 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c1098c0 con 0x7f321c101e70 2023-11-27T15:29:48.159 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.155+0000 7f3213fff700 1 -- 172.21.15.8:0/422198302 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32140055a0 con 0x7f321c101e70 2023-11-27T15:29:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.159+0000 7f3213fff700 1 -- 172.21.15.8:0/422198302 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f321400cd30 con 0x7f321c101e70 2023-11-27T15:29:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.161+0000 7f32345e1b80 1 Processor -- start 2023-11-27T15:29:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.161+0000 7f32345e1b80 1 -- start start 2023-11-27T15:29:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba1827870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1827db0 0x559ba18285d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 0x559ba1828f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f32345e1b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba172d000 con 0x559ba1828b10 2023-11-27T15:29:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f32345e1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba177f840 con 0x559ba16836b0 2023-11-27T15:29:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f32345e1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba1778cd0 con 0x559ba1827db0 2023-11-27T15:29:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f3213fff700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f31fc0a11d0 0x7f31fc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f3211d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba1827870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f3211d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba1827870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:60120/0 (socket says 172.21.15.8:60120) 2023-11-27T15:29:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f3211d9b700 1 -- 172.21.15.8:0/2571297914 learned_addr learned my addr 172.21.15.8:0/2571297914 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f3213fff700 1 -- 172.21.15.8:0/422198302 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3214014070 con 0x7f321c101e70 2023-11-27T15:29:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.162+0000 7f3211d9b700 1 --2- 172.21.15.8:0/2571297914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 0x559ba1828f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.163+0000 7f321a59c700 1 --2- 172.21.15.8:0/2571297914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1827db0 0x559ba18285d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.163+0000 7f3211d9b700 1 -- 172.21.15.8:0/2571297914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1827db0 msgr2=0x559ba18285d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.163+0000 7f3211d9b700 1 --2- 172.21.15.8:0/2571297914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1827db0 0x559ba18285d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.163+0000 7f3211d9b700 1 -- 172.21.15.8:0/2571297914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 msgr2=0x559ba1827870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.163+0000 7f3211d9b700 1 --2- 172.21.15.8:0/2571297914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba1827870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.163+0000 7f3211d9b700 1 -- 172.21.15.8:0/2571297914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ba15fa5d0 con 0x559ba1828b10 2023-11-27T15:29:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.163+0000 7f321ad9d700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f31fc0a11d0 0x7f31fc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.163+0000 7f3211d9b700 1 --2- 172.21.15.8:0/2571297914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 0x559ba1828f80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f320400ba60 tx=0x7f32040096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55b66acd1f94e4a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.164+0000 7f3210d99700 1 -- 172.21.15.8:0/2571297914 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204015a10 con 0x559ba1828b10 2023-11-27T15:29:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.164+0000 7f321ad9d700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f31fc0a11d0 0x7f31fc0a3680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3214099e50 tx=0x7f3214009640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.164+0000 7f3210d99700 1 -- 172.21.15.8:0/2571297914 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f320400be20 con 0x559ba1828b10 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.165+0000 7f3210d99700 1 -- 172.21.15.8:0/2571297914 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204015a10 con 0x559ba1828b10 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.165+0000 7f32345e1b80 1 -- 172.21.15.8:0/2571297914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 msgr2=0x559ba1828f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.165+0000 7f32345e1b80 1 --2- 172.21.15.8:0/2571297914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 0x559ba1828f80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f320400ba60 tx=0x7f32040096d0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.165+0000 7f32345e1b80 1 -- 172.21.15.8:0/2571297914 shutdown_connections 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.165+0000 7f32345e1b80 1 --2- 172.21.15.8:0/2571297914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 0x559ba1828f80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.165+0000 7f32345e1b80 1 --2- 172.21.15.8:0/2571297914 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba1827870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.165+0000 7f32345e1b80 1 --2- 172.21.15.8:0/2571297914 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1827db0 0x559ba18285d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.165+0000 7f32345e1b80 1 -- 172.21.15.8:0/2571297914 >> 172.21.15.8:0/2571297914 conn(0x559ba181cf50 msgr2=0x559ba1822100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.165+0000 7f32345e1b80 1 -- 172.21.15.8:0/2571297914 shutdown_connections 2023-11-27T15:29:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.165+0000 7f32345e1b80 1 -- 172.21.15.8:0/2571297914 wait complete. 2023-11-27T15:29:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.166+0000 7f32345e1b80 1 -- start start 2023-11-27T15:29:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.166+0000 7f32345e1b80 1 Processor -- start 2023-11-27T15:29:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.167+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba16836b0 0x559ba1830ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.167+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba1831220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.167+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1828b10 0x559ba1838260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.167+0000 7f32345e1b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba171a880 con 0x559ba16836b0 2023-11-27T15:29:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.167+0000 7f32345e1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba171a520 con 0x559ba1827db0 2023-11-27T15:29:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.167+0000 7f32345e1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba15fc8c0 con 0x559ba1828b10 2023-11-27T15:29:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.167+0000 7f32345e1b80 10 client.0 _ll_register_callbacks cb 0x559ba1738720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:29:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.168+0000 7f3211d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba16836b0 0x559ba1830ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.168+0000 7f321a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba1831220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.168+0000 7f321a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba1831220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:60136/0 (socket says 172.21.15.8:60136) 2023-11-27T15:29:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.168+0000 7f321a59c700 1 -- 172.21.15.8:0/930519718 learned_addr learned my addr 172.21.15.8:0/930519718 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.169+0000 7f3211d9b700 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1828b10 0x559ba1838260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.169+0000 7f3211d9b700 1 -- 172.21.15.8:0/930519718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1828b10 msgr2=0x559ba1838260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.169+0000 7f3211d9b700 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1828b10 0x559ba1838260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.169+0000 7f3211d9b700 1 -- 172.21.15.8:0/930519718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 msgr2=0x559ba1831220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.169+0000 7f3211d9b700 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba1831220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.169+0000 7f3211d9b700 1 -- 172.21.15.8:0/930519718 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320400b740 con 0x559ba16836b0 2023-11-27T15:29:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.169+0000 7f3211d9b700 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba16836b0 0x559ba1830ce0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f320400bfd0 tx=0x7f3204004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23af1bfcebf23cb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.169+0000 7f3209ffb700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:29:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.170+0000 7f3209ffb700 1 -- 172.21.15.8:0/930519718 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320401ae70 con 0x559ba16836b0 2023-11-27T15:29:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.170+0000 7f3209ffb700 1 -- 172.21.15.8:0/930519718 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3204006e40 con 0x559ba16836b0 2023-11-27T15:29:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.170+0000 7f32345e1b80 1 -- 172.21.15.8:0/930519718 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ba181a710 con 0x559ba16836b0 2023-11-27T15:29:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.170+0000 7f3209ffb700 1 -- 172.21.15.8:0/930519718 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f320400b250 con 0x559ba16836b0 2023-11-27T15:29:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.170+0000 7f32345e1b80 10 client.15612 fetch_fsmap learned FSMap version 32 2023-11-27T15:29:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.170+0000 7f32345e1b80 1 -- 172.21.15.8:0/930519718 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559ba173f130 con 0x559ba16836b0 2023-11-27T15:29:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.170+0000 7f3209ffb700 1 -- 172.21.15.8:0/930519718 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ba181a710 con 0x559ba16836b0 2023-11-27T15:29:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.171+0000 7f3209ffb700 1 -- 172.21.15.8:0/930519718 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 32) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3204008040 con 0x559ba16836b0 2023-11-27T15:29:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.171+0000 7f32345e1b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:29:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.171+0000 7f32345e1b80 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:29:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.171+0000 7f32345e1b80 20 client.15612 populate_metadata read hostname 'smithi008' 2023-11-27T15:29:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.171+0000 7f32345e1b80 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559ba183ca90 0x559ba183eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.171+0000 7f32345e1b80 4 client.15612 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:29:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.171+0000 7f32345e1b80 1 -- 172.21.15.8:0/930519718 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ba173b040 con 0x559ba183ca90 2023-11-27T15:29:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.172+0000 7f321a59c700 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559ba183ca90 0x559ba183eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.172+0000 7f3209ffb700 10 client.15612 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.172+0000 7f321a59c700 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559ba183ca90 0x559ba183eea0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f95a74955ddabdb server_cookie=652b2a20d50a7ee9 in_seq=0 out_seq=0 2023-11-27T15:29:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.177+0000 7f3209ffb700 1 -- 172.21.15.8:0/930519718 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559ba173b040 con 0x559ba183ca90 2023-11-27T15:29:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.177+0000 7f3209ffb700 10 client.15612 handle_command_reply: tid=0 2023-11-27T15:29:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.177+0000 7f32345e1b80 1 client.15612 shutdown 2023-11-27T15:29:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.177+0000 7f32345e1b80 1 -- 172.21.15.8:0/930519718 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba16836b0 msgr2=0x559ba1830ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.177+0000 7f32345e1b80 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba16836b0 0x559ba1830ce0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f320400bfd0 tx=0x7f3204004a90 comp rx=0 tx=0).stop 2023-11-27T15:29:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.178+0000 7f32345e1b80 1 -- 172.21.15.8:0/930519718 shutdown_connections 2023-11-27T15:29:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.178+0000 7f32345e1b80 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559ba183ca90 0x559ba183eea0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.178+0000 7f32345e1b80 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1828b10 0x559ba1838260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.178+0000 7f3209ffb700 0 client.15612 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.178+0000 7f32345e1b80 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba16836b0 0x559ba1830ce0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.178+0000 7f32345e1b80 1 --2- 172.21.15.8:0/930519718 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba1831220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.178+0000 7f32345e1b80 1 -- 172.21.15.8:0/930519718 >> 172.21.15.8:0/930519718 conn(0x559ba181cf50 msgr2=0x559ba18217e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.178+0000 7f32345e1b80 1 -- 172.21.15.8:0/930519718 shutdown_connections 2023-11-27T15:29:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.179+0000 7f32345e1b80 1 -- 172.21.15.8:0/930519718 wait complete. 2023-11-27T15:29:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.179+0000 7f32345e1b80 20 client.15612 trim_cache size 0 max 16384 2023-11-27T15:29:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.200+0000 7f32345e1b80 1 Processor -- start 2023-11-27T15:29:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.200+0000 7f32345e1b80 1 -- start start 2023-11-27T15:29:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.200+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba16836b0 0x559ba182edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.200+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba18381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.200+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 0x559ba182f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.200+0000 7f32345e1b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba1831a70 con 0x559ba1828b10 2023-11-27T15:29:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.200+0000 7f32345e1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba1831ed0 con 0x559ba1827db0 2023-11-27T15:29:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.200+0000 7f32345e1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba1830190 con 0x559ba16836b0 2023-11-27T15:29:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f321a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba18381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f3211d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba16836b0 0x559ba182edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f321a59c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba18381f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:60146/0 (socket says 172.21.15.8:60146) 2023-11-27T15:29:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f3211d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba16836b0 0x559ba182edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:50206/0 (socket says 172.21.15.8:50206) 2023-11-27T15:29:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f321a59c700 1 -- 172.21.15.8:0/3313300340 learned_addr learned my addr 172.21.15.8:0/3313300340 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:48.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f3211d9b700 1 --2- 172.21.15.8:0/3313300340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 0x559ba182f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f321a59c700 1 -- 172.21.15.8:0/3313300340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba16836b0 msgr2=0x559ba182edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f321a59c700 1 --2- 172.21.15.8:0/3313300340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba16836b0 0x559ba182edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f321a59c700 1 -- 172.21.15.8:0/3313300340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 msgr2=0x559ba182f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f321a59c700 1 --2- 172.21.15.8:0/3313300340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 0x559ba182f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f321a59c700 1 -- 172.21.15.8:0/3313300340 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ba173ee80 con 0x559ba1827db0 2023-11-27T15:29:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.201+0000 7f3211d9b700 1 --2- 172.21.15.8:0/3313300340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba16836b0 0x559ba182edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.202+0000 7f321a59c700 1 --2- 172.21.15.8:0/3313300340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba18381f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f320c0098e0 tx=0x7f320c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c64fcc3f2898a31e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.202+0000 7f321159a700 1 -- 172.21.15.8:0/3313300340 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c026c50 con 0x559ba1827db0 2023-11-27T15:29:48.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.202+0000 7f321159a700 1 -- 172.21.15.8:0/3313300340 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f320c01e040 con 0x559ba1827db0 2023-11-27T15:29:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.202+0000 7f321159a700 1 -- 172.21.15.8:0/3313300340 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c005460 con 0x559ba1827db0 2023-11-27T15:29:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.203+0000 7f32345e1b80 1 -- 172.21.15.8:0/3313300340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 msgr2=0x559ba18381f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.203+0000 7f32345e1b80 1 --2- 172.21.15.8:0/3313300340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba18381f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f320c0098e0 tx=0x7f320c00b260 comp rx=0 tx=0).stop 2023-11-27T15:29:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.203+0000 7f32345e1b80 1 -- 172.21.15.8:0/3313300340 shutdown_connections 2023-11-27T15:29:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.203+0000 7f32345e1b80 1 --2- 172.21.15.8:0/3313300340 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba16836b0 0x559ba182edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.203+0000 7f32345e1b80 1 --2- 172.21.15.8:0/3313300340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1828b10 0x559ba182f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.203+0000 7f32345e1b80 1 --2- 172.21.15.8:0/3313300340 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba1827db0 0x559ba18381f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.203+0000 7f32345e1b80 1 -- 172.21.15.8:0/3313300340 >> 172.21.15.8:0/3313300340 conn(0x559ba181cf50 msgr2=0x559ba18437c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.204+0000 7f32345e1b80 1 -- 172.21.15.8:0/3313300340 shutdown_connections 2023-11-27T15:29:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.204+0000 7f32345e1b80 1 -- 172.21.15.8:0/3313300340 wait complete. 2023-11-27T15:29:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.204+0000 7f32345e1b80 1 -- start start 2023-11-27T15:29:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.204+0000 7f32345e1b80 1 Processor -- start 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba18342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1827db0 0x559ba1834800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f32345e1b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1828b10 0x559ba1837820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f32345e1b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba1837d60 con 0x559ba1827db0 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f32345e1b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba1837ed0 con 0x559ba16836b0 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f32345e1b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ba1838090 con 0x559ba1828b10 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f32345e1b80 10 client.0 _ll_register_callbacks cb 0x7f320400b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f3211d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba18342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f3211d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba18342c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:60150/0 (socket says 172.21.15.8:60150) 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f3211d9b700 1 -- 172.21.15.8:0/3437119109 learned_addr learned my addr 172.21.15.8:0/3437119109 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:29:48.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f3211d9b700 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1828b10 0x559ba1837820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.205+0000 7f321a59c700 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1827db0 0x559ba1834800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.206+0000 7f3211d9b700 1 -- 172.21.15.8:0/3437119109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1828b10 msgr2=0x559ba1837820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.206+0000 7f3211d9b700 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1828b10 0x559ba1837820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.206+0000 7f3211d9b700 1 -- 172.21.15.8:0/3437119109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1827db0 msgr2=0x559ba1834800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.206+0000 7f3211d9b700 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1827db0 0x559ba1834800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.206+0000 7f3211d9b700 1 -- 172.21.15.8:0/3437119109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320400f040 con 0x559ba16836b0 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.206+0000 7f3211d9b700 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba18342c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f320400b590 tx=0x7f3204019710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7c88f46a14d372c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.206+0000 7f3209ffb700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.206+0000 7f3209ffb700 1 -- 172.21.15.8:0/3437119109 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204026640 con 0x559ba16836b0 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.206+0000 7f3209ffb700 1 -- 172.21.15.8:0/3437119109 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f320403f060 con 0x559ba16836b0 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.206+0000 7f32345e1b80 1 -- 172.21.15.8:0/3437119109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f320400be20 con 0x559ba16836b0 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f3209ffb700 1 -- 172.21.15.8:0/3437119109 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3204026e60 con 0x559ba16836b0 2023-11-27T15:29:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f32345e1b80 10 client.25658 fetch_fsmap learned FSMap version 32 2023-11-27T15:29:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f32345e1b80 1 -- 172.21.15.8:0/3437119109 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f321c108f80 con 0x559ba16836b0 2023-11-27T15:29:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f3209ffb700 1 -- 172.21.15.8:0/3437119109 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f320400be20 con 0x559ba16836b0 2023-11-27T15:29:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f3209ffb700 1 -- 172.21.15.8:0/3437119109 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 32) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3204030070 con 0x559ba16836b0 2023-11-27T15:29:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f32345e1b80 10 client.25658 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:29:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f32345e1b80 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:29:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f32345e1b80 20 client.25658 populate_metadata read hostname 'smithi008' 2023-11-27T15:29:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f32345e1b80 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559ba1891880 0x559ba1893c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f32345e1b80 4 client.25658 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:29:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.207+0000 7f32345e1b80 1 -- 172.21.15.8:0/3437119109 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559ba1747600 con 0x559ba1891880 2023-11-27T15:29:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.208+0000 7f321a59c700 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559ba1891880 0x559ba1893c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.208+0000 7f321a59c700 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559ba1891880 0x559ba1893c90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=915640b5260db3f1 server_cookie=da3a4b92e98981e in_seq=0 out_seq=0 2023-11-27T15:29:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.208+0000 7f3209ffb700 10 client.25658 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:48.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:48 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.60:6838/871970532,v1:172.21.15.60:6839/871970532] up:boot 2023-11-27T15:29:48.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:48 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:29:48.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:48 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:29:48.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:48 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:29:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f3209ffb700 1 -- 172.21.15.8:0/3437119109 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559ba1747600 con 0x559ba1891880 2023-11-27T15:29:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f3209ffb700 10 client.25658 handle_command_reply: tid=0 2023-11-27T15:29:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f32345e1b80 1 client.25658 shutdown 2023-11-27T15:29:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f32345e1b80 1 -- 172.21.15.8:0/3437119109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 msgr2=0x559ba18342c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f32345e1b80 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba18342c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f320400b590 tx=0x7f3204019710 comp rx=0 tx=0).stop 2023-11-27T15:29:48.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f32345e1b80 1 -- 172.21.15.8:0/3437119109 shutdown_connections 2023-11-27T15:29:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f32345e1b80 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x559ba1891880 0x559ba1893c90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f32345e1b80 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x559ba1828b10 0x559ba1837820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f32345e1b80 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x559ba16836b0 0x559ba18342c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f3209ffb700 0 client.25658 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:29:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.857+0000 7f32345e1b80 1 --2- 172.21.15.8:0/3437119109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x559ba1827db0 0x559ba1834800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.858+0000 7f32345e1b80 1 -- 172.21.15.8:0/3437119109 >> 172.21.15.8:0/3437119109 conn(0x559ba181cf50 msgr2=0x559ba182edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.858+0000 7f32345e1b80 1 -- 172.21.15.8:0/3437119109 shutdown_connections 2023-11-27T15:29:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.858+0000 7f32345e1b80 1 -- 172.21.15.8:0/3437119109 wait complete. 2023-11-27T15:29:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:29:48.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:29:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:29:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:29:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.858+0000 7f32345e1b80 20 client.25658 trim_cache size 0 max 16384 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.858+0000 7f3220bff700 1 -- 172.21.15.8:0/422198302 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f31fc0a11d0 msgr2=0x7f31fc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.858+0000 7f3220bff700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f31fc0a11d0 0x7f31fc0a3680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3214099e50 tx=0x7f3214009640 comp rx=0 tx=0).stop 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.858+0000 7f3220bff700 1 -- 172.21.15.8:0/422198302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f321c101e70 msgr2=0x7f321c1146e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.858+0000 7f3220bff700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f321c101e70 0x7f321c1146e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f321400e940 tx=0x7f321400ec50 comp rx=0 tx=0).stop 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.859+0000 7f3220bff700 1 -- 172.21.15.8:0/422198302 shutdown_connections 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.859+0000 7f3220bff700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f31fc0a11d0 0x7f31fc0a3680 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.859+0000 7f3220bff700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f321c100230 0x7f321c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.859+0000 7f3220bff700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f321c101e70 0x7f321c1146e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.859+0000 7f3220bff700 1 --2- 172.21.15.8:0/422198302 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f321c1014c0 0x7f321c10f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.859+0000 7f3220bff700 1 -- 172.21.15.8:0/422198302 >> 172.21.15.8:0/422198302 conn(0x7f321c0fb810 msgr2=0x7f321c1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.859+0000 7f3220bff700 1 -- 172.21.15.8:0/422198302 shutdown_connections 2023-11-27T15:29:48.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:29:48.859+0000 7f3220bff700 1 -- 172.21.15.8:0/422198302 wait complete. 2023-11-27T15:29:48.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:48 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.60:6838/871970532,v1:172.21.15.60:6839/871970532] up:boot 2023-11-27T15:29:48.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:48 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:29:48.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:48 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:29:48.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:48 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:29:48.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:48 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.60:6838/871970532,v1:172.21.15.60:6839/871970532] up:boot 2023-11-27T15:29:48.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:48 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:29:48.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:48 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:29:48.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:48 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:29:49.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:49 smithi060 ceph-mon[131518]: pgmap v478: 129 pgs: 129 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 408 op/s 2023-11-27T15:29:49.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:49 smithi008 ceph-mon[128788]: pgmap v478: 129 pgs: 129 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 408 op/s 2023-11-27T15:29:49.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:49 smithi029 ceph-mon[131166]: pgmap v478: 129 pgs: 129 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 408 op/s 2023-11-27T15:29:51.074 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[160576]: *** Caught signal (Segmentation fault) ** 2023-11-27T15:29:51.074 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[160576]: in thread 7fd1fdc96700 thread_name:MR_Finisher 2023-11-27T15:29:51.531 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:51 smithi060 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i.service: Main process exited, code=exited, status=139/n/a 2023-11-27T15:29:51.531 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:29:51 smithi060 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i.service: Failed with result 'exit-code'. 2023-11-27T15:29:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:51 smithi060 ceph-mon[131518]: pgmap v479: 129 pgs: 129 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2023-11-27T15:29:51.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:51 smithi008 ceph-mon[128788]: pgmap v479: 129 pgs: 129 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2023-11-27T15:29:51.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:51 smithi029 ceph-mon[131166]: pgmap v479: 129 pgs: 129 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 416 op/s 2023-11-27T15:29:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:53 smithi060 ceph-mon[131518]: pgmap v480: 129 pgs: 129 active+clean; 201 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 544 op/s 2023-11-27T15:29:53.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:53 smithi008 ceph-mon[128788]: pgmap v480: 129 pgs: 129 active+clean; 201 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 544 op/s 2023-11-27T15:29:53.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:53 smithi029 ceph-mon[131166]: pgmap v480: 129 pgs: 129 active+clean; 201 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 544 op/s 2023-11-27T15:29:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:55 smithi060 ceph-mon[131518]: pgmap v481: 129 pgs: 129 active+clean; 203 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 545 op/s 2023-11-27T15:29:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:55 smithi008 ceph-mon[128788]: pgmap v481: 129 pgs: 129 active+clean; 203 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 545 op/s 2023-11-27T15:29:55.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:55 smithi029 ceph-mon[131166]: pgmap v481: 129 pgs: 129 active+clean; 203 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 545 op/s 2023-11-27T15:29:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:57 smithi060 ceph-mon[131518]: pgmap v482: 129 pgs: 129 active+clean; 206 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 516 op/s 2023-11-27T15:29:57.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:57 smithi008 ceph-mon[128788]: pgmap v482: 129 pgs: 129 active+clean; 206 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 516 op/s 2023-11-27T15:29:57.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:57 smithi029 ceph-mon[131166]: pgmap v482: 129 pgs: 129 active+clean; 206 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 516 op/s 2023-11-27T15:29:59.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:29:59 smithi008 ceph-mon[128788]: pgmap v483: 129 pgs: 129 active+clean; 214 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 494 op/s 2023-11-27T15:29:59.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:29:59 smithi029 ceph-mon[131166]: pgmap v483: 129 pgs: 129 active+clean; 214 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 494 op/s 2023-11-27T15:29:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:29:59 smithi060 ceph-mon[131518]: pgmap v483: 129 pgs: 129 active+clean; 214 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 494 op/s 2023-11-27T15:30:00.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:00 smithi008 ceph-mon[128788]: overall HEALTH_OK 2023-11-27T15:30:00.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:00 smithi029 ceph-mon[131166]: overall HEALTH_OK 2023-11-27T15:30:00.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:00 smithi060 ceph-mon[131518]: overall HEALTH_OK 2023-11-27T15:30:01.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:01 smithi029 ceph-mon[131166]: pgmap v484: 129 pgs: 129 active+clean; 216 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 463 op/s 2023-11-27T15:30:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:01 smithi060 ceph-mon[131518]: pgmap v484: 129 pgs: 129 active+clean; 216 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 463 op/s 2023-11-27T15:30:01.824 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:30:01 smithi060 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i.service: Service RestartSec=10s expired, scheduling restart. 2023-11-27T15:30:01.824 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:30:01 smithi060 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i.service: Scheduled restart job, restart counter is at 2. 2023-11-27T15:30:01.825 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:30:01 smithi060 systemd[1]: Stopped Ceph mds.i for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:30:01.825 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:30:01 smithi060 systemd[1]: Starting Ceph mds.i for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:30:01.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:01 smithi008 ceph-mon[128788]: pgmap v484: 129 pgs: 129 active+clean; 216 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 463 op/s 2023-11-27T15:30:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.046+0000 7f403fdff700 1 -- 172.21.15.8:0/362436821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4038100250 msgr2=0x7f4038100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.046+0000 7f403fdff700 1 --2- 172.21.15.8:0/362436821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4038100250 0x7f4038100660 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4034009990 tx=0x7f4034009ca0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.046+0000 7f403fdff700 1 -- 172.21.15.8:0/362436821 shutdown_connections 2023-11-27T15:30:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.046+0000 7f403fdff700 1 --2- 172.21.15.8:0/362436821 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4038100250 0x7f4038100660 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.046+0000 7f403fdff700 1 --2- 172.21.15.8:0/362436821 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4038101dd0 0x7f4038108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.046+0000 7f403fdff700 1 --2- 172.21.15.8:0/362436821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4038101440 0x7f4038101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.046+0000 7f403fdff700 1 -- 172.21.15.8:0/362436821 >> 172.21.15.8:0/362436821 conn(0x7f40380fb850 msgr2=0x7f40380fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.046+0000 7f403fdff700 1 -- 172.21.15.8:0/362436821 shutdown_connections 2023-11-27T15:30:02.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.046+0000 7f403fdff700 1 -- 172.21.15.8:0/362436821 wait complete. 2023-11-27T15:30:02.054 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.047+0000 7f403fdff700 1 Processor -- start 2023-11-27T15:30:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403fdff700 1 -- start start 2023-11-27T15:30:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403fdff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4038100250 0x7f40380748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403fdff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4038101440 0x7f4038072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403fdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4038101dd0 0x7f4038073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403fdff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038073bc0 con 0x7f4038100250 2023-11-27T15:30:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403fdff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038073d30 con 0x7f4038101440 2023-11-27T15:30:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403fdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40381a0020 con 0x7f4038101dd0 2023-11-27T15:30:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403d39a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4038101440 0x7f4038072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403d39a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4038101440 0x7f4038072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50962/0 (socket says 172.21.15.8:50962) 2023-11-27T15:30:02.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403d39a700 1 -- 172.21.15.8:0/705607452 learned_addr learned my addr 172.21.15.8:0/705607452 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403e39c700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4038101dd0 0x7f4038073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.048+0000 7f403db9b700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4038100250 0x7f40380748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f403d39a700 1 -- 172.21.15.8:0/705607452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4038101dd0 msgr2=0x7f4038073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f403d39a700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4038101dd0 0x7f4038073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f403d39a700 1 -- 172.21.15.8:0/705607452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4038100250 msgr2=0x7f40380748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f403d39a700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4038100250 0x7f40380748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f403d39a700 1 -- 172.21.15.8:0/705607452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40340095f0 con 0x7f4038101440 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f403e39c700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4038101dd0 0x7f4038073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f403d39a700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4038101440 0x7f4038072ee0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f402800e9a0 tx=0x7f402800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f402effd700 1 -- 172.21.15.8:0/705607452 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028009790 con 0x7f4038101440 2023-11-27T15:30:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f402effd700 1 -- 172.21.15.8:0/705607452 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4028010430 con 0x7f4038101440 2023-11-27T15:30:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f402effd700 1 -- 172.21.15.8:0/705607452 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028018680 con 0x7f4038101440 2023-11-27T15:30:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.049+0000 7f403fdff700 1 -- 172.21.15.8:0/705607452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403810bbb0 con 0x7f4038101440 2023-11-27T15:30:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.050+0000 7f403fdff700 1 -- 172.21.15.8:0/705607452 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403810ba20 con 0x7f4038101440 2023-11-27T15:30:02.059 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.054+0000 7f402effd700 1 -- 172.21.15.8:0/705607452 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f40280105a0 con 0x7f4038101440 2023-11-27T15:30:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.055+0000 7f4053647b80 1 Processor -- start 2023-11-27T15:30:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.055+0000 7f4053647b80 1 -- start start 2023-11-27T15:30:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f4053647b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee60806b0 0x556ee6224870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f4053647b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee62255d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f4053647b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee6225b10 0x556ee6225f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f4053647b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee612a000 con 0x556ee6225b10 2023-11-27T15:30:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f4053647b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee617c840 con 0x556ee6224db0 2023-11-27T15:30:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f4053647b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6175cd0 con 0x556ee60806b0 2023-11-27T15:30:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f403db9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee62255d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f403e39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee60806b0 0x556ee6224870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f403e39c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee60806b0 0x556ee6224870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:46388/0 (socket says 172.21.15.8:46388) 2023-11-27T15:30:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f403e39c700 1 -- 172.21.15.8:0/2275262149 learned_addr learned my addr 172.21.15.8:0/2275262149 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.056+0000 7f403d39a700 1 --2- 172.21.15.8:0/2275262149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee6225b10 0x556ee6225f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.057+0000 7f402effd700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f40240a1150 0x7f40240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.057+0000 7f403e39c700 1 -- 172.21.15.8:0/2275262149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 msgr2=0x556ee62255d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.057+0000 7f403e39c700 1 --2- 172.21.15.8:0/2275262149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee62255d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.057+0000 7f403e39c700 1 -- 172.21.15.8:0/2275262149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee6225b10 msgr2=0x556ee6225f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.057+0000 7f403e39c700 1 --2- 172.21.15.8:0/2275262149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee6225b10 0x556ee6225f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.057+0000 7f403e39c700 1 -- 172.21.15.8:0/2275262149 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ee5ff75d0 con 0x556ee60806b0 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.057+0000 7f402effd700 1 -- 172.21.15.8:0/705607452 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4028014070 con 0x7f4038101440 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.057+0000 7f403e39c700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f40240a1150 0x7f40240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.057+0000 7f403e39c700 1 --2- 172.21.15.8:0/2275262149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee60806b0 0x556ee6224870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4030003f70 tx=0x7f4030004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb78d68686b8aa76 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.058+0000 7f4023fff700 1 -- 172.21.15.8:0/2275262149 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40300090b0 con 0x556ee60806b0 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.058+0000 7f4023fff700 1 -- 172.21.15.8:0/2275262149 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f403000f040 con 0x556ee60806b0 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.058+0000 7f4023fff700 1 -- 172.21.15.8:0/2275262149 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030018800 con 0x556ee60806b0 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.058+0000 7f4053647b80 1 -- 172.21.15.8:0/2275262149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee60806b0 msgr2=0x556ee6224870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.058+0000 7f4053647b80 1 --2- 172.21.15.8:0/2275262149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee60806b0 0x556ee6224870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4030003f70 tx=0x7f4030004320 comp rx=0 tx=0).stop 2023-11-27T15:30:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.058+0000 7f403e39c700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f40240a1150 0x7f40240a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4030022000 tx=0x7f403001f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.059+0000 7f4053647b80 1 -- 172.21.15.8:0/2275262149 shutdown_connections 2023-11-27T15:30:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.059+0000 7f4053647b80 1 --2- 172.21.15.8:0/2275262149 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee60806b0 0x556ee6224870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.059+0000 7f4053647b80 1 --2- 172.21.15.8:0/2275262149 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee6225b10 0x556ee6225f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.059+0000 7f4053647b80 1 --2- 172.21.15.8:0/2275262149 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee62255d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.059+0000 7f4053647b80 1 -- 172.21.15.8:0/2275262149 >> 172.21.15.8:0/2275262149 conn(0x556ee6219f50 msgr2=0x556ee621f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.059+0000 7f4053647b80 1 -- 172.21.15.8:0/2275262149 shutdown_connections 2023-11-27T15:30:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.059+0000 7f4053647b80 1 -- 172.21.15.8:0/2275262149 wait complete. 2023-11-27T15:30:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.059+0000 7f4053647b80 1 -- start start 2023-11-27T15:30:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.060+0000 7f4053647b80 1 Processor -- start 2023-11-27T15:30:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.060+0000 7f4053647b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee60806b0 0x556ee622dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.060+0000 7f4053647b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6224db0 0x556ee622e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.060+0000 7f4053647b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee6225b10 0x556ee6235200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.060+0000 7f4053647b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6117880 con 0x556ee6225b10 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.060+0000 7f4053647b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6117520 con 0x556ee60806b0 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.060+0000 7f4053647b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee5ff98c0 con 0x556ee6224db0 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.060+0000 7f4053647b80 10 client.0 _ll_register_callbacks cb 0x556ee6135720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403db9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee60806b0 0x556ee622dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403db9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee60806b0 0x556ee622dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50974/0 (socket says 172.21.15.8:50974) 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403db9b700 1 -- 172.21.15.8:0/1898528971 learned_addr learned my addr 172.21.15.8:0/1898528971 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403e39c700 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee6225b10 0x556ee6235200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403d39a700 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6224db0 0x556ee622e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403db9b700 1 -- 172.21.15.8:0/1898528971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6224db0 msgr2=0x556ee622e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403db9b700 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6224db0 0x556ee622e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403db9b700 1 -- 172.21.15.8:0/1898528971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee6225b10 msgr2=0x556ee6235200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403db9b700 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee6225b10 0x556ee6235200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403db9b700 1 -- 172.21.15.8:0/1898528971 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4030003b70 con 0x556ee60806b0 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403d39a700 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6224db0 0x556ee622e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.061+0000 7f403db9b700 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee60806b0 0x556ee622dc80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f40381012a0 tx=0x7f4034005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18021ca8054a73a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.062+0000 7f40217fa700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.062+0000 7f40217fa700 1 -- 172.21.15.8:0/1898528971 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403401d030 con 0x556ee60806b0 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.062+0000 7f4053647b80 1 -- 172.21.15.8:0/1898528971 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ee6217710 con 0x556ee60806b0 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.062+0000 7f40217fa700 1 -- 172.21.15.8:0/1898528971 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4034004700 con 0x556ee60806b0 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.062+0000 7f40217fa700 1 -- 172.21.15.8:0/1898528971 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40340056f0 con 0x556ee60806b0 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.062+0000 7f4053647b80 10 client.25670 fetch_fsmap learned FSMap version 32 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.062+0000 7f4053647b80 1 -- 172.21.15.8:0/1898528971 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556ee613c130 con 0x556ee60806b0 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.063+0000 7f40217fa700 1 -- 172.21.15.8:0/1898528971 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556ee6217710 con 0x556ee60806b0 2023-11-27T15:30:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.063+0000 7f40217fa700 1 -- 172.21.15.8:0/1898528971 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 32) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f403400f9c0 con 0x556ee60806b0 2023-11-27T15:30:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.063+0000 7f4053647b80 10 client.25670 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:30:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.063+0000 7f4053647b80 10 client.25670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:30:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.063+0000 7f4053647b80 20 client.25670 populate_metadata read hostname 'smithi008' 2023-11-27T15:30:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.063+0000 7f4053647b80 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x556ee6239ae0 0x556ee623bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.063+0000 7f4053647b80 4 client.25670 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:30:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.063+0000 7f4053647b80 1 -- 172.21.15.8:0/1898528971 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ee6138040 con 0x556ee6239ae0 2023-11-27T15:30:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.063+0000 7f403d39a700 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x556ee6239ae0 0x556ee623bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.064+0000 7f403d39a700 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x556ee6239ae0 0x556ee623bef0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f00402c1727c2c server_cookie=fc9ffc60e4686c4e in_seq=0 out_seq=0 2023-11-27T15:30:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.064+0000 7f40217fa700 10 client.25670 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.064+0000 7f40217fa700 1 -- 172.21.15.8:0/1898528971 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ee6138040 con 0x556ee6239ae0 2023-11-27T15:30:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.064+0000 7f40217fa700 10 client.25670 handle_command_reply: tid=0 2023-11-27T15:30:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 client.25670 shutdown 2023-11-27T15:30:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 -- 172.21.15.8:0/1898528971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee60806b0 msgr2=0x556ee622dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee60806b0 0x556ee622dc80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f40381012a0 tx=0x7f4034005f00 comp rx=0 tx=0).stop 2023-11-27T15:30:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 -- 172.21.15.8:0/1898528971 shutdown_connections 2023-11-27T15:30:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee6225b10 0x556ee6235200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee60806b0 0x556ee622dc80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x556ee6239ae0 0x556ee623bef0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 --2- 172.21.15.8:0/1898528971 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6224db0 0x556ee622e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 -- 172.21.15.8:0/1898528971 >> 172.21.15.8:0/1898528971 conn(0x556ee6219f50 msgr2=0x556ee621e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f40217fa700 0 client.25670 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 -- 172.21.15.8:0/1898528971 shutdown_connections 2023-11-27T15:30:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.065+0000 7f4053647b80 1 -- 172.21.15.8:0/1898528971 wait complete. 2023-11-27T15:30:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.066+0000 7f4053647b80 20 client.25670 trim_cache size 0 max 16384 2023-11-27T15:30:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f4053647b80 1 Processor -- start 2023-11-27T15:30:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f4053647b80 1 -- start start 2023-11-27T15:30:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f4053647b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 0x556ee6225ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f4053647b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6235120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f4053647b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6225b10 0x556ee62289c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f4053647b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6225420 con 0x556ee60806b0 2023-11-27T15:30:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f4053647b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6220930 con 0x556ee6224db0 2023-11-27T15:30:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f4053647b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6226420 con 0x556ee6225b10 2023-11-27T15:30:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f403d39a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6235120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f403d39a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6235120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50978/0 (socket says 172.21.15.8:50978) 2023-11-27T15:30:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.085+0000 7f403d39a700 1 -- 172.21.15.8:0/2781910378 learned_addr learned my addr 172.21.15.8:0/2781910378 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.086+0000 7f403e39c700 1 --2- 172.21.15.8:0/2781910378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6225b10 0x556ee62289c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.086+0000 7f403db9b700 1 --2- 172.21.15.8:0/2781910378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 0x556ee6225ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.086+0000 7f403d39a700 1 -- 172.21.15.8:0/2781910378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6225b10 msgr2=0x556ee62289c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.086+0000 7f403d39a700 1 --2- 172.21.15.8:0/2781910378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6225b10 0x556ee62289c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.086+0000 7f403d39a700 1 -- 172.21.15.8:0/2781910378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 msgr2=0x556ee6225ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.086+0000 7f403d39a700 1 --2- 172.21.15.8:0/2781910378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 0x556ee6225ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.086+0000 7f403d39a700 1 -- 172.21.15.8:0/2781910378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ee613be80 con 0x556ee6224db0 2023-11-27T15:30:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.086+0000 7f403d39a700 1 --2- 172.21.15.8:0/2781910378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6235120 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4028099e70 tx=0x7f4028010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5720f9ed8f6e271f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.086+0000 7f402cbff700 1 -- 172.21.15.8:0/2781910378 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402809df10 con 0x556ee6224db0 2023-11-27T15:30:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f402cbff700 1 -- 172.21.15.8:0/2781910378 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40280a2330 con 0x556ee6224db0 2023-11-27T15:30:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f402cbff700 1 -- 172.21.15.8:0/2781910378 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402809a3a0 con 0x556ee6224db0 2023-11-27T15:30:02.095 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f4053647b80 1 -- 172.21.15.8:0/2781910378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 msgr2=0x556ee6235120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f4053647b80 1 --2- 172.21.15.8:0/2781910378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6235120 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4028099e70 tx=0x7f4028010c80 comp rx=0 tx=0).stop 2023-11-27T15:30:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f4053647b80 1 -- 172.21.15.8:0/2781910378 shutdown_connections 2023-11-27T15:30:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f4053647b80 1 --2- 172.21.15.8:0/2781910378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6225b10 0x556ee62289c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f4053647b80 1 --2- 172.21.15.8:0/2781910378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 0x556ee6225ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f4053647b80 1 --2- 172.21.15.8:0/2781910378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6235120 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f4053647b80 1 -- 172.21.15.8:0/2781910378 >> 172.21.15.8:0/2781910378 conn(0x556ee6219f50 msgr2=0x556ee6240830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f4053647b80 1 -- 172.21.15.8:0/2781910378 shutdown_connections 2023-11-27T15:30:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.087+0000 7f4053647b80 1 -- 172.21.15.8:0/2781910378 wait complete. 2023-11-27T15:30:02.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.088+0000 7f4053647b80 1 -- start start 2023-11-27T15:30:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.088+0000 7f4053647b80 1 Processor -- start 2023-11-27T15:30:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.088+0000 7f4053647b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 0x556ee6231350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.088+0000 7f4053647b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6231890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.088+0000 7f4053647b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6225b10 0x556ee6234460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.088+0000 7f4053647b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee62349a0 con 0x556ee60806b0 2023-11-27T15:30:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.088+0000 7f4053647b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6234b10 con 0x556ee6224db0 2023-11-27T15:30:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.088+0000 7f4053647b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ee6234cd0 con 0x556ee6225b10 2023-11-27T15:30:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f4053647b80 10 client.0 _ll_register_callbacks cb 0x7f4034009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:30:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403d39a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6231890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403db9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 0x556ee6231350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403d39a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6231890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50988/0 (socket says 172.21.15.8:50988) 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403d39a700 1 -- 172.21.15.8:0/2772460712 learned_addr learned my addr 172.21.15.8:0/2772460712 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403e39c700 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6225b10 0x556ee6234460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403db9b700 1 -- 172.21.15.8:0/2772460712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6225b10 msgr2=0x556ee6234460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403db9b700 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6225b10 0x556ee6234460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403db9b700 1 -- 172.21.15.8:0/2772460712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 msgr2=0x556ee6231890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403db9b700 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6231890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403db9b700 1 -- 172.21.15.8:0/2772460712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403810ba20 con 0x556ee60806b0 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403d39a700 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6231890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f403db9b700 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 0x556ee6231350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f403400c010 tx=0x7f40340178e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bed7077b3305966 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.089+0000 7f40217fa700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:30:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.090+0000 7f40217fa700 1 -- 172.21.15.8:0/2772460712 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403401d040 con 0x556ee60806b0 2023-11-27T15:30:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.090+0000 7f40217fa700 1 -- 172.21.15.8:0/2772460712 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f4034000e40 con 0x556ee60806b0 2023-11-27T15:30:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.090+0000 7f4053647b80 1 -- 172.21.15.8:0/2772460712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f403000f040 con 0x556ee60806b0 2023-11-27T15:30:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.090+0000 7f40217fa700 1 -- 172.21.15.8:0/2772460712 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f403400fea0 con 0x556ee60806b0 2023-11-27T15:30:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.090+0000 7f4053647b80 10 client.15636 fetch_fsmap learned FSMap version 32 2023-11-27T15:30:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.090+0000 7f4053647b80 1 -- 172.21.15.8:0/2772460712 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556ee5f8cc00 con 0x556ee60806b0 2023-11-27T15:30:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.091+0000 7f40217fa700 1 -- 172.21.15.8:0/2772460712 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f403000f040 con 0x556ee60806b0 2023-11-27T15:30:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.091+0000 7f40217fa700 1 -- 172.21.15.8:0/2772460712 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 32) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f40340099c0 con 0x556ee60806b0 2023-11-27T15:30:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.091+0000 7f4053647b80 10 client.15636 fetch_fsmap finished waiting for FSMap version 32 2023-11-27T15:30:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.091+0000 7f4053647b80 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:30:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.091+0000 7f4053647b80 20 client.15636 populate_metadata read hostname 'smithi008' 2023-11-27T15:30:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.091+0000 7f4053647b80 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x556ee628e670 0x556ee6290a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.091+0000 7f4053647b80 4 client.15636 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:30:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.091+0000 7f4053647b80 1 -- 172.21.15.8:0/2772460712 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ee6144600 con 0x556ee628e670 2023-11-27T15:30:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.091+0000 7f403d39a700 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x556ee628e670 0x556ee6290a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.092+0000 7f403d39a700 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x556ee628e670 0x556ee6290a80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee4f6d798814a60 server_cookie=7d59d00b316dcea6 in_seq=0 out_seq=0 2023-11-27T15:30:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.092+0000 7f40217fa700 10 client.15636 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.549+0000 7f40217fa700 1 -- 172.21.15.8:0/2772460712 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ee6144600 con 0x556ee628e670 2023-11-27T15:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.549+0000 7f40217fa700 10 client.15636 handle_command_reply: tid=0 2023-11-27T15:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.549+0000 7f4053647b80 1 client.15636 shutdown 2023-11-27T15:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.550+0000 7f4053647b80 1 -- 172.21.15.8:0/2772460712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 msgr2=0x556ee6231350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.550+0000 7f4053647b80 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 0x556ee6231350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f403400c010 tx=0x7f40340178e0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.550+0000 7f4053647b80 1 -- 172.21.15.8:0/2772460712 shutdown_connections 2023-11-27T15:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.550+0000 7f4053647b80 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x556ee628e670 0x556ee6290a80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.550+0000 7f4053647b80 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x556ee6225b10 0x556ee6234460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.550+0000 7f4053647b80 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x556ee60806b0 0x556ee6231350 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.550+0000 7f4053647b80 1 --2- 172.21.15.8:0/2772460712 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x556ee6224db0 0x556ee6231890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.550+0000 7f4053647b80 1 -- 172.21.15.8:0/2772460712 >> 172.21.15.8:0/2772460712 conn(0x556ee6219f50 msgr2=0x556ee62289c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.550+0000 7f40217fa700 0 client.15636 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.550+0000 7f4053647b80 1 -- 172.21.15.8:0/2772460712 shutdown_connections 2023-11-27T15:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.551+0000 7f4053647b80 1 -- 172.21.15.8:0/2772460712 wait complete. 2023-11-27T15:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.552+0000 7f4053647b80 20 client.15636 trim_cache size 0 max 16384 2023-11-27T15:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:30:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.552+0000 7f403fdff700 1 -- 172.21.15.8:0/705607452 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f40240a1150 msgr2=0x7f40240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.552+0000 7f403fdff700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f40240a1150 0x7f40240a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4030022000 tx=0x7f403001f000 comp rx=0 tx=0).stop 2023-11-27T15:30:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.553+0000 7f403fdff700 1 -- 172.21.15.8:0/705607452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4038101440 msgr2=0x7f4038072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.553+0000 7f403fdff700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4038101440 0x7f4038072ee0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f402800e9a0 tx=0x7f402800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.553+0000 7f403fdff700 1 -- 172.21.15.8:0/705607452 shutdown_connections 2023-11-27T15:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.553+0000 7f403fdff700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f40240a1150 0x7f40240a3600 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.553+0000 7f403fdff700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4038101dd0 0x7f4038073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.553+0000 7f403fdff700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4038100250 0x7f40380748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.553+0000 7f403fdff700 1 --2- 172.21.15.8:0/705607452 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4038101440 0x7f4038072ee0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.553+0000 7f403fdff700 1 -- 172.21.15.8:0/705607452 >> 172.21.15.8:0/705607452 conn(0x7f40380fb850 msgr2=0x7f40380fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.553+0000 7f403fdff700 1 -- 172.21.15.8:0/705607452 shutdown_connections 2023-11-27T15:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:02.553+0000 7f403fdff700 1 -- 172.21.15.8:0/705607452 wait complete. 2023-11-27T15:30:03.324 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:30:02 smithi060 podman[160765]: 2023-11-27T15:30:03.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:03 smithi060 ceph-mon[131518]: pgmap v485: 129 pgs: 129 active+clean; 222 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 468 op/s 2023-11-27T15:30:03.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:03 smithi060 ceph-mon[131518]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2023-11-27T15:30:03.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:03 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 5 up:standby 2023-11-27T15:30:03.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:03 smithi008 ceph-mon[128788]: pgmap v485: 129 pgs: 129 active+clean; 222 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 468 op/s 2023-11-27T15:30:03.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:03 smithi008 ceph-mon[128788]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2023-11-27T15:30:03.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:03 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 5 up:standby 2023-11-27T15:30:03.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:03 smithi029 ceph-mon[131166]: pgmap v485: 129 pgs: 129 active+clean; 222 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 468 op/s 2023-11-27T15:30:03.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:03 smithi029 ceph-mon[131166]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2023-11-27T15:30:03.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:03 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 5 up:standby 2023-11-27T15:30:04.074 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:30:03 smithi060 bash[160765]: 5220d23c8168ea6f1ab6904b92f625d400497a2c7efffe0d66ee714737ff7adf 2023-11-27T15:30:04.074 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:30:03 smithi060 ceph-mds[160808]: -- 172.21.15.60:0/3140635052 <== mon.1 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5564ff707520 con 0x5564ff6dcc00 2023-11-27T15:30:04.074 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:30:03 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[160783]: starting mds.i at 2023-11-27T15:30:04.074 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:30:03 smithi060 systemd[1]: Started Ceph mds.i for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:30:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:05 smithi060 ceph-mon[131518]: pgmap v486: 129 pgs: 129 active+clean; 226 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 379 op/s 2023-11-27T15:30:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:05 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.60:6838/2258758502,v1:172.21.15.60:6839/2258758502] up:boot 2023-11-27T15:30:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:05 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:30:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:05 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:30:05.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:05 smithi008 ceph-mon[128788]: pgmap v486: 129 pgs: 129 active+clean; 226 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 379 op/s 2023-11-27T15:30:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:05 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.60:6838/2258758502,v1:172.21.15.60:6839/2258758502] up:boot 2023-11-27T15:30:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:05 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:30:05.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:05 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:30:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:05 smithi029 ceph-mon[131166]: pgmap v486: 129 pgs: 129 active+clean; 226 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 379 op/s 2023-11-27T15:30:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:05 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.60:6838/2258758502,v1:172.21.15.60:6839/2258758502] up:boot 2023-11-27T15:30:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:05 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:30:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:05 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:30:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:07 smithi060 ceph-mon[131518]: pgmap v487: 129 pgs: 129 active+clean; 230 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 408 op/s 2023-11-27T15:30:07.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:07 smithi008 ceph-mon[128788]: pgmap v487: 129 pgs: 129 active+clean; 230 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 408 op/s 2023-11-27T15:30:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:07 smithi029 ceph-mon[131166]: pgmap v487: 129 pgs: 129 active+clean; 230 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 408 op/s 2023-11-27T15:30:08.783 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:08 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:30:08.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:08 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:30:08.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:08 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:30:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:09 smithi060 ceph-mon[131518]: pgmap v488: 129 pgs: 129 active+clean; 228 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 451 op/s 2023-11-27T15:30:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:30:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:30:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:09 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:30:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:09 smithi008 ceph-mon[128788]: pgmap v488: 129 pgs: 129 active+clean; 228 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 451 op/s 2023-11-27T15:30:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:30:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:30:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:09 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:30:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:09 smithi029 ceph-mon[131166]: pgmap v488: 129 pgs: 129 active+clean; 228 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 451 op/s 2023-11-27T15:30:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:30:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:30:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:09 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:30:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:10 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:30:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:10 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:30:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:10 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:30:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:10 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:30:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:10 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:30:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:10 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:30:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:10 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:30:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:10 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:30:10.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:10 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:30:10.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:10 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:30:10.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:10 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:30:10.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:10 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:30:11.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:11 smithi060 ceph-mon[131518]: pgmap v489: 129 pgs: 129 active+clean; 232 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 422 op/s 2023-11-27T15:30:11.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:11 smithi008 ceph-mon[128788]: pgmap v489: 129 pgs: 129 active+clean; 232 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 422 op/s 2023-11-27T15:30:11.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:11 smithi029 ceph-mon[131166]: pgmap v489: 129 pgs: 129 active+clean; 232 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 422 op/s 2023-11-27T15:30:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:13 smithi060 ceph-mon[131518]: pgmap v490: 129 pgs: 129 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 456 op/s 2023-11-27T15:30:13.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:13 smithi008 ceph-mon[128788]: pgmap v490: 129 pgs: 129 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 456 op/s 2023-11-27T15:30:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:13 smithi029 ceph-mon[131166]: pgmap v490: 129 pgs: 129 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 456 op/s 2023-11-27T15:30:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:15 smithi060 ceph-mon[131518]: pgmap v491: 129 pgs: 129 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 393 op/s 2023-11-27T15:30:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.856+0000 7fe681f9d700 1 -- 172.21.15.8:0/1657630849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6740975b0 msgr2=0x7fe6740979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.856+0000 7fe681f9d700 1 --2- 172.21.15.8:0/1657630849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6740975b0 0x7fe6740979c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe6700098f0 tx=0x7fe670009c00 comp rx=0 tx=0).stop 2023-11-27T15:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.857+0000 7fe681f9d700 1 -- 172.21.15.8:0/1657630849 shutdown_connections 2023-11-27T15:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.857+0000 7fe681f9d700 1 --2- 172.21.15.8:0/1657630849 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe674099130 0x7fe67409f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.857+0000 7fe681f9d700 1 --2- 172.21.15.8:0/1657630849 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6740975b0 0x7fe6740979c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.857+0000 7fe681f9d700 1 --2- 172.21.15.8:0/1657630849 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6740987a0 0x7fe674098bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.857+0000 7fe681f9d700 1 -- 172.21.15.8:0/1657630849 >> 172.21.15.8:0/1657630849 conn(0x7fe674092bb0 msgr2=0x7fe674095000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.858+0000 7fe681f9d700 1 -- 172.21.15.8:0/1657630849 shutdown_connections 2023-11-27T15:30:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.858+0000 7fe681f9d700 1 -- 172.21.15.8:0/1657630849 wait complete. 2023-11-27T15:30:15.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:15 smithi008 ceph-mon[128788]: pgmap v491: 129 pgs: 129 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 393 op/s 2023-11-27T15:30:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.860+0000 7fe681f9d700 1 Processor -- start 2023-11-27T15:30:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.860+0000 7fe681f9d700 1 -- start start 2023-11-27T15:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.861+0000 7fe681f9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6740975b0 0x7fe67413e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.862+0000 7fe681f9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6740987a0 0x7fe67413e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.862+0000 7fe681f9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe674099130 0x7fe6741459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.862+0000 7fe681f9d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674145fb0 con 0x7fe674099130 2023-11-27T15:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.862+0000 7fe681f9d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6741460f0 con 0x7fe6740987a0 2023-11-27T15:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.862+0000 7fe681f9d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674146280 con 0x7fe6740975b0 2023-11-27T15:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.862+0000 7fe680f9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6740975b0 0x7fe67413e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.863+0000 7fe67bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6740987a0 0x7fe67413e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.863+0000 7fe67bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6740987a0 0x7fe67413e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:40952/0 (socket says 172.21.15.8:40952) 2023-11-27T15:30:15.868 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.863+0000 7fe67bfff700 1 -- 172.21.15.8:0/441178063 learned_addr learned my addr 172.21.15.8:0/441178063 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.864+0000 7fe67bfff700 1 -- 172.21.15.8:0/441178063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6740975b0 msgr2=0x7fe67413e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.864+0000 7fe67bfff700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6740975b0 0x7fe67413e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.864+0000 7fe67bfff700 1 -- 172.21.15.8:0/441178063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe674099130 msgr2=0x7fe6741459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.864+0000 7fe67bfff700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe674099130 0x7fe6741459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.864+0000 7fe67bfff700 1 -- 172.21.15.8:0/441178063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe670009520 con 0x7fe6740987a0 2023-11-27T15:30:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.864+0000 7fe680f9b700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6740975b0 0x7fe67413e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:30:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.864+0000 7fe67bfff700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6740987a0 0x7fe67413e970 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe66800d980 tx=0x7fe66800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.865+0000 7fe679ffb700 1 -- 172.21.15.8:0/441178063 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668009930 con 0x7fe6740987a0 2023-11-27T15:30:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.865+0000 7fe679ffb700 1 -- 172.21.15.8:0/441178063 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe668010460 con 0x7fe6740987a0 2023-11-27T15:30:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.865+0000 7fe679ffb700 1 -- 172.21.15.8:0/441178063 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe668009930 con 0x7fe6740987a0 2023-11-27T15:30:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.865+0000 7fe681f9d700 1 -- 172.21.15.8:0/441178063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6740a2fd0 con 0x7fe6740987a0 2023-11-27T15:30:15.870 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.865+0000 7fe681f9d700 1 -- 172.21.15.8:0/441178063 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6740a2e40 con 0x7fe6740987a0 2023-11-27T15:30:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.873+0000 7fe696c18b80 1 Processor -- start 2023-11-27T15:30:15.877 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.874+0000 7fe679ffb700 1 -- 172.21.15.8:0/441178063 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe668009a90 con 0x7fe6740987a0 2023-11-27T15:30:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.874+0000 7fe696c18b80 1 -- start start 2023-11-27T15:30:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.876+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 0x563976e13a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.876+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e13fb0 0x563976e14820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.876+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e14d60 0x563976e15200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.876+0000 7fe696c18b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976d19000 con 0x563976c6f6b0 2023-11-27T15:30:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.876+0000 7fe696c18b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976d6b840 con 0x563976e13fb0 2023-11-27T15:30:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.876+0000 7fe696c18b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976d64cd0 con 0x563976e14d60 2023-11-27T15:30:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.876+0000 7fe67bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e14d60 0x563976e15200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.876+0000 7fe680f9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e13fb0 0x563976e14820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.877+0000 7fe67bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e14d60 0x563976e15200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58126/0 (socket says 172.21.15.8:58126) 2023-11-27T15:30:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.877+0000 7fe67bfff700 1 -- 172.21.15.8:0/1979839859 learned_addr learned my addr 172.21.15.8:0/1979839859 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.877+0000 7fe68179c700 1 --2- 172.21.15.8:0/1979839859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 0x563976e13a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.878+0000 7fe680f9b700 1 -- 172.21.15.8:0/1979839859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e14d60 msgr2=0x563976e15200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.878+0000 7fe680f9b700 1 --2- 172.21.15.8:0/1979839859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e14d60 0x563976e15200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.878+0000 7fe680f9b700 1 -- 172.21.15.8:0/1979839859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 msgr2=0x563976e13a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.879+0000 7fe679ffb700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe66c0a1230 0x7fe66c0a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.879+0000 7fe679ffb700 1 -- 172.21.15.8:0/441178063 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe6680cc690 con 0x7fe6740987a0 2023-11-27T15:30:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.880+0000 7fe680f9b700 1 --2- 172.21.15.8:0/1979839859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 0x563976e13a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.880+0000 7fe680f9b700 1 -- 172.21.15.8:0/1979839859 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563976be65d0 con 0x563976e13fb0 2023-11-27T15:30:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.880+0000 7fe67bfff700 1 --2- 172.21.15.8:0/1979839859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e14d60 0x563976e15200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:30:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.880+0000 7fe68179c700 1 --2- 172.21.15.8:0/1979839859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 0x563976e13a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:30:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.880+0000 7fe680f9b700 1 --2- 172.21.15.8:0/1979839859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e13fb0 0x563976e14820 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe670000c00 tx=0x7fe670005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695328d0ffee73b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.881+0000 7fe67bfff700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe66c0a1230 0x7fe66c0a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.881+0000 7fe666ffd700 1 -- 172.21.15.8:0/1979839859 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67001c050 con 0x563976e13fb0 2023-11-27T15:30:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.881+0000 7fe666ffd700 1 -- 172.21.15.8:0/1979839859 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe67001d040 con 0x563976e13fb0 2023-11-27T15:30:15.885 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.881+0000 7fe666ffd700 1 -- 172.21.15.8:0/1979839859 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700055a0 con 0x563976e13fb0 2023-11-27T15:30:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.884+0000 7fe67bfff700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe66c0a1230 0x7fe66c0a36e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe668096180 tx=0x7fe6680d2110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.884+0000 7fe696c18b80 1 -- 172.21.15.8:0/1979839859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e13fb0 msgr2=0x563976e14820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.885+0000 7fe696c18b80 1 --2- 172.21.15.8:0/1979839859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e13fb0 0x563976e14820 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe670000c00 tx=0x7fe670005bc0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.885+0000 7fe696c18b80 1 -- 172.21.15.8:0/1979839859 shutdown_connections 2023-11-27T15:30:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.885+0000 7fe696c18b80 1 --2- 172.21.15.8:0/1979839859 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e14d60 0x563976e15200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.885+0000 7fe696c18b80 1 --2- 172.21.15.8:0/1979839859 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 0x563976e13a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.885+0000 7fe696c18b80 1 --2- 172.21.15.8:0/1979839859 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e13fb0 0x563976e14820 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.885+0000 7fe696c18b80 1 -- 172.21.15.8:0/1979839859 >> 172.21.15.8:0/1979839859 conn(0x563976e091a0 msgr2=0x563976e0e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.887+0000 7fe696c18b80 1 -- 172.21.15.8:0/1979839859 shutdown_connections 2023-11-27T15:30:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.887+0000 7fe696c18b80 1 -- 172.21.15.8:0/1979839859 wait complete. 2023-11-27T15:30:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.887+0000 7fe696c18b80 1 -- start start 2023-11-27T15:30:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.888+0000 7fe696c18b80 1 Processor -- start 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.890+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 0x563976e1ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.890+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e13fb0 0x563976e1d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.890+0000 7fe68179c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 0x563976e1ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.890+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e24430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.890+0000 7fe68179c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 0x563976e1ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36494/0 (socket says 172.21.15.8:36494) 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.891+0000 7fe68179c700 1 -- 172.21.15.8:0/2095492318 learned_addr learned my addr 172.21.15.8:0/2095492318 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.891+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976d06880 con 0x563976c6f6b0 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.891+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976d06520 con 0x563976e14d60 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.891+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976be88c0 con 0x563976e13fb0 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.891+0000 7fe696c18b80 10 client.0 _ll_register_callbacks cb 0x563976d24720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:30:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.891+0000 7fe680f9b700 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e13fb0 0x563976e1d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.892+0000 7fe68179c700 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e24430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.892+0000 7fe680f9b700 1 -- 172.21.15.8:0/2095492318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 msgr2=0x563976e24430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.892+0000 7fe680f9b700 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e24430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.892+0000 7fe680f9b700 1 -- 172.21.15.8:0/2095492318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 msgr2=0x563976e1ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.892+0000 7fe680f9b700 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 0x563976e1ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.892+0000 7fe680f9b700 1 -- 172.21.15.8:0/2095492318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6740a0320 con 0x563976e13fb0 2023-11-27T15:30:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.893+0000 7fe68179c700 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e24430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:30:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:15 smithi029 ceph-mon[131166]: pgmap v491: 129 pgs: 129 active+clean; 238 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 393 op/s 2023-11-27T15:30:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.895+0000 7fe680f9b700 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e13fb0 0x563976e1d3f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6700098f0 tx=0x7fe670005600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c0d511eac823ff server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.895+0000 7fe65bfff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:30:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.895+0000 7fe65bfff700 1 -- 172.21.15.8:0/2095492318 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67000ba80 con 0x563976e13fb0 2023-11-27T15:30:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.896+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563976e06960 con 0x563976e13fb0 2023-11-27T15:30:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.897+0000 7fe65bfff700 1 -- 172.21.15.8:0/2095492318 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe67000bbe0 con 0x563976e13fb0 2023-11-27T15:30:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.897+0000 7fe65bfff700 1 -- 172.21.15.8:0/2095492318 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe67001c020 con 0x563976e13fb0 2023-11-27T15:30:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.897+0000 7fe696c18b80 10 client.25708 fetch_fsmap learned FSMap version 34 2023-11-27T15:30:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.897+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563976d2b130 con 0x563976e13fb0 2023-11-27T15:30:15.901 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.897+0000 7fe65bfff700 1 -- 172.21.15.8:0/2095492318 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563976e06960 con 0x563976e13fb0 2023-11-27T15:30:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.898+0000 7fe65bfff700 1 -- 172.21.15.8:0/2095492318 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe670015020 con 0x563976e13fb0 2023-11-27T15:30:15.902 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.899+0000 7fe696c18b80 10 client.25708 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:30:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.899+0000 7fe696c18b80 10 client.25708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:30:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.899+0000 7fe696c18b80 20 client.25708 populate_metadata read hostname 'smithi008' 2023-11-27T15:30:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.900+0000 7fe696c18b80 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x563976e28680 0x563976e2aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.900+0000 7fe696c18b80 4 client.25708 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:30:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.900+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563976d27040 con 0x563976e28680 2023-11-27T15:30:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.900+0000 7fe68179c700 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x563976e28680 0x563976e2aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.902+0000 7fe68179c700 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x563976e28680 0x563976e2aa90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a82e271fbb5f274e server_cookie=28cc0fba370966e9 in_seq=0 out_seq=0 2023-11-27T15:30:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.903+0000 7fe65bfff700 10 client.25708 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.904+0000 7fe65bfff700 1 -- 172.21.15.8:0/2095492318 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563976d27040 con 0x563976e28680 2023-11-27T15:30:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.904+0000 7fe65bfff700 10 client.25708 handle_command_reply: tid=0 2023-11-27T15:30:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.904+0000 7fe696c18b80 1 client.25708 shutdown 2023-11-27T15:30:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.905+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e13fb0 msgr2=0x563976e1d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.905+0000 7fe696c18b80 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e13fb0 0x563976e1d3f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6700098f0 tx=0x7fe670005600 comp rx=0 tx=0).stop 2023-11-27T15:30:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.905+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 shutdown_connections 2023-11-27T15:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.905+0000 7fe696c18b80 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e24430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.905+0000 7fe696c18b80 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976c6f6b0 0x563976e1ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.905+0000 7fe696c18b80 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x563976e28680 0x563976e2aa90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.905+0000 7fe696c18b80 1 --2- 172.21.15.8:0/2095492318 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976e13fb0 0x563976e1d3f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.905+0000 7fe65bfff700 0 client.25708 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.905+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 >> 172.21.15.8:0/2095492318 conn(0x563976e091a0 msgr2=0x563976e0da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.906+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 shutdown_connections 2023-11-27T15:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.906+0000 7fe696c18b80 1 -- 172.21.15.8:0/2095492318 wait complete. 2023-11-27T15:30:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.906+0000 7fe696c18b80 20 client.25708 trim_cache size 0 max 16384 2023-11-27T15:30:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.941+0000 7fe696c18b80 1 Processor -- start 2023-11-27T15:30:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.942+0000 7fe696c18b80 1 -- start start 2023-11-27T15:30:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.942+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e17c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.942+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 0x563976e18150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.942+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.942+0000 7fe696c18b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976e1ca20 con 0x563976e13fb0 2023-11-27T15:30:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.942+0000 7fe696c18b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976e0fb10 con 0x563976e14d60 2023-11-27T15:30:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.942+0000 7fe696c18b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976e0f5e0 con 0x563976c6f6b0 2023-11-27T15:30:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.943+0000 7fe68179c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e17c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.944+0000 7fe68179c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e17c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58148/0 (socket says 172.21.15.8:58148) 2023-11-27T15:30:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.944+0000 7fe68179c700 1 -- 172.21.15.8:0/3266607207 learned_addr learned my addr 172.21.15.8:0/3266607207 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.944+0000 7fe68179c700 1 --2- 172.21.15.8:0/3266607207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.944+0000 7fe680f9b700 1 --2- 172.21.15.8:0/3266607207 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 0x563976e18150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.946+0000 7fe68179c700 1 -- 172.21.15.8:0/3266607207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 msgr2=0x563976e243f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.946+0000 7fe68179c700 1 --2- 172.21.15.8:0/3266607207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e243f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.946+0000 7fe68179c700 1 -- 172.21.15.8:0/3266607207 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 msgr2=0x563976e18150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.946+0000 7fe68179c700 1 --2- 172.21.15.8:0/3266607207 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 0x563976e18150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.946+0000 7fe68179c700 1 -- 172.21.15.8:0/3266607207 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563976d2ae80 con 0x563976c6f6b0 2023-11-27T15:30:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.948+0000 7fe68179c700 1 --2- 172.21.15.8:0/3266607207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e17c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe67c064410 tx=0x7fe67c04eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8334caca710a4e84 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.948+0000 7fe6677fe700 1 -- 172.21.15.8:0/3266607207 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c06f030 con 0x563976c6f6b0 2023-11-27T15:30:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.949+0000 7fe6677fe700 1 -- 172.21.15.8:0/3266607207 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563976d27040 con 0x563976c6f6b0 2023-11-27T15:30:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.949+0000 7fe6677fe700 1 -- 172.21.15.8:0/3266607207 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c073ea0 con 0x563976c6f6b0 2023-11-27T15:30:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.950+0000 7fe696c18b80 1 -- 172.21.15.8:0/3266607207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 msgr2=0x563976e17c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.950+0000 7fe696c18b80 1 --2- 172.21.15.8:0/3266607207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e17c10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe67c064410 tx=0x7fe67c04eb70 comp rx=0 tx=0).stop 2023-11-27T15:30:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.951+0000 7fe696c18b80 1 -- 172.21.15.8:0/3266607207 shutdown_connections 2023-11-27T15:30:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.951+0000 7fe696c18b80 1 --2- 172.21.15.8:0/3266607207 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e17c10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.951+0000 7fe696c18b80 1 --2- 172.21.15.8:0/3266607207 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.951+0000 7fe696c18b80 1 --2- 172.21.15.8:0/3266607207 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 0x563976e18150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.951+0000 7fe696c18b80 1 -- 172.21.15.8:0/3266607207 >> 172.21.15.8:0/3266607207 conn(0x563976e091a0 msgr2=0x563976e2f410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.951+0000 7fe696c18b80 1 -- 172.21.15.8:0/3266607207 shutdown_connections 2023-11-27T15:30:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.951+0000 7fe696c18b80 1 -- 172.21.15.8:0/3266607207 wait complete. 2023-11-27T15:30:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.952+0000 7fe696c18b80 1 -- start start 2023-11-27T15:30:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.952+0000 7fe696c18b80 1 Processor -- start 2023-11-27T15:30:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.955+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e20370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.955+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 0x563976e208b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.955+0000 7fe68179c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e20370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.956+0000 7fe68179c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e20370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58164/0 (socket says 172.21.15.8:58164) 2023-11-27T15:30:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.956+0000 7fe696c18b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.956+0000 7fe696c18b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976e23e10 con 0x563976e13fb0 2023-11-27T15:30:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.956+0000 7fe696c18b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976e23f80 con 0x563976e14d60 2023-11-27T15:30:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.956+0000 7fe696c18b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x563976e24140 con 0x563976c6f6b0 2023-11-27T15:30:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.956+0000 7fe696c18b80 10 client.0 _ll_register_callbacks cb 0x7fe6680d2060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:30:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.956+0000 7fe68179c700 1 -- 172.21.15.8:0/3952691364 learned_addr learned my addr 172.21.15.8:0/3952691364 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.957+0000 7fe68179c700 1 -- 172.21.15.8:0/3952691364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 msgr2=0x563976e238d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.957+0000 7fe680f9b700 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 0x563976e208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.957+0000 7fe68179c700 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.957+0000 7fe68179c700 1 -- 172.21.15.8:0/3952691364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 msgr2=0x563976e208b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.957+0000 7fe68179c700 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 0x563976e208b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.957+0000 7fe68179c700 1 -- 172.21.15.8:0/3952691364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c086040 con 0x563976c6f6b0 2023-11-27T15:30:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.957+0000 7fe680f9b700 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 0x563976e208b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:30:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.958+0000 7fe68179c700 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e20370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe67c02d210 tx=0x7fe67c072400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7d02a4de7d822f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.958+0000 7fe65bfff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:30:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.958+0000 7fe65bfff700 1 -- 172.21.15.8:0/3952691364 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c073ea0 con 0x563976c6f6b0 2023-11-27T15:30:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.958+0000 7fe65bfff700 1 -- 172.21.15.8:0/3952691364 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fe67c07a070 con 0x563976c6f6b0 2023-11-27T15:30:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.960+0000 7fe696c18b80 1 -- 172.21.15.8:0/3952691364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe67001d040 con 0x563976c6f6b0 2023-11-27T15:30:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.962+0000 7fe65bfff700 1 -- 172.21.15.8:0/3952691364 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe67c06f050 con 0x563976c6f6b0 2023-11-27T15:30:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.962+0000 7fe696c18b80 10 client.25714 fetch_fsmap learned FSMap version 34 2023-11-27T15:30:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.962+0000 7fe696c18b80 1 -- 172.21.15.8:0/3952691364 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563976b7bc00 con 0x563976c6f6b0 2023-11-27T15:30:15.966 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.963+0000 7fe65bfff700 1 -- 172.21.15.8:0/3952691364 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe67001d040 con 0x563976c6f6b0 2023-11-27T15:30:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.963+0000 7fe65bfff700 1 -- 172.21.15.8:0/3952691364 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe67c095680 con 0x563976c6f6b0 2023-11-27T15:30:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.963+0000 7fe696c18b80 10 client.25714 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:30:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.963+0000 7fe696c18b80 10 client.25714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:30:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.963+0000 7fe696c18b80 20 client.25714 populate_metadata read hostname 'smithi008' 2023-11-27T15:30:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.964+0000 7fe696c18b80 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x563976e7d350 0x563976e7f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.964+0000 7fe696c18b80 4 client.25714 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:30:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.964+0000 7fe696c18b80 1 -- 172.21.15.8:0/3952691364 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563976d33600 con 0x563976e7d350 2023-11-27T15:30:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.965+0000 7fe680f9b700 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x563976e7d350 0x563976e7f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.966+0000 7fe680f9b700 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x563976e7d350 0x563976e7f760 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a553b77fa135e6a server_cookie=5b65eae992ad09fe in_seq=0 out_seq=0 2023-11-27T15:30:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:15.966+0000 7fe65bfff700 10 client.25714 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.532+0000 7fe65bfff700 1 -- 172.21.15.8:0/3952691364 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563976d33600 con 0x563976e7d350 2023-11-27T15:30:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.532+0000 7fe65bfff700 10 client.25714 handle_command_reply: tid=0 2023-11-27T15:30:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.532+0000 7fe696c18b80 1 client.25714 shutdown 2023-11-27T15:30:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.533+0000 7fe696c18b80 1 -- 172.21.15.8:0/3952691364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 msgr2=0x563976e20370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.533+0000 7fe696c18b80 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e20370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe67c02d210 tx=0x7fe67c072400 comp rx=0 tx=0).stop 2023-11-27T15:30:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.533+0000 7fe696c18b80 1 -- 172.21.15.8:0/3952691364 shutdown_connections 2023-11-27T15:30:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.533+0000 7fe696c18b80 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x563976e7d350 0x563976e7f760 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.533+0000 7fe696c18b80 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x563976c6f6b0 0x563976e20370 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.533+0000 7fe696c18b80 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x563976e14d60 0x563976e238d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.533+0000 7fe696c18b80 1 --2- 172.21.15.8:0/3952691364 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x563976e13fb0 0x563976e208b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.533+0000 7fe65bfff700 0 client.25714 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.533+0000 7fe696c18b80 1 -- 172.21.15.8:0/3952691364 >> 172.21.15.8:0/3952691364 conn(0x563976e091a0 msgr2=0x563976e17c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.533+0000 7fe696c18b80 1 -- 172.21.15.8:0/3952691364 shutdown_connections 2023-11-27T15:30:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.534+0000 7fe696c18b80 1 -- 172.21.15.8:0/3952691364 wait complete. 2023-11-27T15:30:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.534+0000 7fe696c18b80 20 client.25714 trim_cache size 0 max 16384 2023-11-27T15:30:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:30:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:30:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:30:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:30:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.535+0000 7fe681f9d700 1 -- 172.21.15.8:0/441178063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe66c0a1230 msgr2=0x7fe66c0a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.535+0000 7fe681f9d700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe66c0a1230 0x7fe66c0a36e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe668096180 tx=0x7fe6680d2110 comp rx=0 tx=0).stop 2023-11-27T15:30:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.535+0000 7fe681f9d700 1 -- 172.21.15.8:0/441178063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6740987a0 msgr2=0x7fe67413e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.535+0000 7fe681f9d700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6740987a0 0x7fe67413e970 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe66800d980 tx=0x7fe66800dc90 comp rx=0 tx=0).stop 2023-11-27T15:30:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.536+0000 7fe681f9d700 1 -- 172.21.15.8:0/441178063 shutdown_connections 2023-11-27T15:30:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.536+0000 7fe681f9d700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fe66c0a1230 0x7fe66c0a36e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.536+0000 7fe681f9d700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fe6740975b0 0x7fe67413e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.536+0000 7fe681f9d700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe674099130 0x7fe6741459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.536+0000 7fe681f9d700 1 --2- 172.21.15.8:0/441178063 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fe6740987a0 0x7fe67413e970 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.536+0000 7fe681f9d700 1 -- 172.21.15.8:0/441178063 >> 172.21.15.8:0/441178063 conn(0x7fe674092bb0 msgr2=0x7fe67409bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.537+0000 7fe681f9d700 1 -- 172.21.15.8:0/441178063 shutdown_connections 2023-11-27T15:30:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:16.537+0000 7fe681f9d700 1 -- 172.21.15.8:0/441178063 wait complete. 2023-11-27T15:30:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:17 smithi060 ceph-mon[131518]: pgmap v492: 129 pgs: 129 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 338 op/s 2023-11-27T15:30:17.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:17 smithi008 ceph-mon[128788]: pgmap v492: 129 pgs: 129 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 338 op/s 2023-11-27T15:30:17.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:17 smithi029 ceph-mon[131166]: pgmap v492: 129 pgs: 129 active+clean; 246 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 338 op/s 2023-11-27T15:30:19.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:19 smithi029 ceph-mon[131166]: pgmap v493: 129 pgs: 129 active+clean; 248 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 320 op/s 2023-11-27T15:30:19.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:19 smithi060 ceph-mon[131518]: pgmap v493: 129 pgs: 129 active+clean; 248 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 320 op/s 2023-11-27T15:30:19.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:19 smithi008 ceph-mon[128788]: pgmap v493: 129 pgs: 129 active+clean; 248 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 320 op/s 2023-11-27T15:30:21.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:21 smithi060 ceph-mon[131518]: pgmap v494: 129 pgs: 129 active+clean; 245 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 311 op/s 2023-11-27T15:30:21.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:21 smithi008 ceph-mon[128788]: pgmap v494: 129 pgs: 129 active+clean; 245 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 311 op/s 2023-11-27T15:30:21.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:21 smithi029 ceph-mon[131166]: pgmap v494: 129 pgs: 129 active+clean; 245 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 311 op/s 2023-11-27T15:30:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:23 smithi060 ceph-mon[131518]: pgmap v495: 129 pgs: 129 active+clean; 249 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2023-11-27T15:30:23.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:23 smithi008 ceph-mon[128788]: pgmap v495: 129 pgs: 129 active+clean; 249 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2023-11-27T15:30:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:23 smithi029 ceph-mon[131166]: pgmap v495: 129 pgs: 129 active+clean; 249 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2023-11-27T15:30:25.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:25 smithi060 ceph-mon[131518]: pgmap v496: 129 pgs: 129 active+clean; 253 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 335 op/s 2023-11-27T15:30:25.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:25 smithi008 ceph-mon[128788]: pgmap v496: 129 pgs: 129 active+clean; 253 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 335 op/s 2023-11-27T15:30:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:25 smithi029 ceph-mon[131166]: pgmap v496: 129 pgs: 129 active+clean; 253 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 335 op/s 2023-11-27T15:30:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:27 smithi060 ceph-mon[131518]: pgmap v497: 129 pgs: 129 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 369 op/s 2023-11-27T15:30:27.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:27 smithi008 ceph-mon[128788]: pgmap v497: 129 pgs: 129 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 369 op/s 2023-11-27T15:30:27.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:27 smithi029 ceph-mon[131166]: pgmap v497: 129 pgs: 129 active+clean; 258 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 369 op/s 2023-11-27T15:30:29.648 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:29 smithi029 ceph-mon[131166]: pgmap v498: 129 pgs: 129 active+clean; 263 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 385 op/s 2023-11-27T15:30:29.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:29 smithi060 ceph-mon[131518]: pgmap v498: 129 pgs: 129 active+clean; 263 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 385 op/s 2023-11-27T15:30:29.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:29 smithi008 ceph-mon[128788]: pgmap v498: 129 pgs: 129 active+clean; 263 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 385 op/s 2023-11-27T15:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.878+0000 7f6bacdff700 1 -- 172.21.15.8:0/1775906137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba8074a90 msgr2=0x7f6ba8072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.878+0000 7f6bacdff700 1 --2- 172.21.15.8:0/1775906137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba8074a90 0x7f6ba8072f80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009990 tx=0x7f6ba0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.878+0000 7f6bacdff700 1 -- 172.21.15.8:0/1775906137 shutdown_connections 2023-11-27T15:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.878+0000 7f6bacdff700 1 --2- 172.21.15.8:0/1775906137 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba8074a90 0x7f6ba8072f80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.878+0000 7f6bacdff700 1 --2- 172.21.15.8:0/1775906137 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba8103ca0 0x7f6ba810e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.878+0000 7f6bacdff700 1 --2- 172.21.15.8:0/1775906137 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ba80734c0 0x7f6ba8073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.878+0000 7f6bacdff700 1 -- 172.21.15.8:0/1775906137 >> 172.21.15.8:0/1775906137 conn(0x7f6ba80fb810 msgr2=0x7f6ba80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.878+0000 7f6bacdff700 1 -- 172.21.15.8:0/1775906137 shutdown_connections 2023-11-27T15:30:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.878+0000 7f6bacdff700 1 -- 172.21.15.8:0/1775906137 wait complete. 2023-11-27T15:30:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.880+0000 7f6bacdff700 1 Processor -- start 2023-11-27T15:30:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.880+0000 7f6bacdff700 1 -- start start 2023-11-27T15:30:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.881+0000 7f6bacdff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80734c0 0x7f6ba8196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.881+0000 7f6bacdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba8074a90 0x7f6ba8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.881+0000 7f6bacdff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ba8103ca0 0x7f6ba819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.881+0000 7f6bacdff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba819b9d0 con 0x7f6ba8103ca0 2023-11-27T15:30:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.881+0000 7f6bacdff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba819bb40 con 0x7f6ba80734c0 2023-11-27T15:30:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.881+0000 7f6bacdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba819bd00 con 0x7f6ba8074a90 2023-11-27T15:30:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.881+0000 7f6ba5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba8074a90 0x7f6ba8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.881+0000 7f6ba659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80734c0 0x7f6ba8196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.881+0000 7f6ba5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba8074a90 0x7f6ba8198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:54080/0 (socket says 172.21.15.8:54080) 2023-11-27T15:30:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.881+0000 7f6ba5d9b700 1 -- 172.21.15.8:0/3689166852 learned_addr learned my addr 172.21.15.8:0/3689166852 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.882+0000 7f6ba659c700 1 -- 172.21.15.8:0/3689166852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba8074a90 msgr2=0x7f6ba8198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.882+0000 7f6ba659c700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba8074a90 0x7f6ba8198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.882+0000 7f6ba659c700 1 -- 172.21.15.8:0/3689166852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ba8103ca0 msgr2=0x7f6ba819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.882+0000 7f6ba659c700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ba8103ca0 0x7f6ba819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.882+0000 7f6ba659c700 1 -- 172.21.15.8:0/3689166852 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba00095f0 con 0x7f6ba80734c0 2023-11-27T15:30:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.882+0000 7f6ba659c700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80734c0 0x7f6ba8196740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00095c0 tx=0x7f6ba000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.883+0000 7f6b9f7fe700 1 -- 172.21.15.8:0/3689166852 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba001d070 con 0x7f6ba80734c0 2023-11-27T15:30:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.884+0000 7f6b9f7fe700 1 -- 172.21.15.8:0/3689166852 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ba0004830 con 0x7f6ba80734c0 2023-11-27T15:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.884+0000 7f6b9f7fe700 1 -- 172.21.15.8:0/3689166852 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00053a0 con 0x7f6ba80734c0 2023-11-27T15:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.884+0000 7f6bacdff700 1 -- 172.21.15.8:0/3689166852 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba8106a90 con 0x7f6ba80734c0 2023-11-27T15:30:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.884+0000 7f6bacdff700 1 -- 172.21.15.8:0/3689166852 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba8106900 con 0x7f6ba80734c0 2023-11-27T15:30:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.892+0000 7f6bc0707b80 1 Processor -- start 2023-11-27T15:30:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.893+0000 7f6b9f7fe700 1 -- 172.21.15.8:0/3689166852 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6ba0005520 con 0x7f6ba80734c0 2023-11-27T15:30:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.895+0000 7f6b9f7fe700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6b940b21e0 0x7f6b940b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.895+0000 7f6b9f7fe700 1 -- 172.21.15.8:0/3689166852 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6ba00cdcb0 con 0x7f6ba80734c0 2023-11-27T15:30:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.896+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6b940b21e0 0x7f6b940b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.897+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6b940b21e0 0x7f6b940b4690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b900060b0 tx=0x7f6b90006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.898+0000 7f6bc0707b80 1 -- start start 2023-11-27T15:30:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.898+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbe9e0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.898+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e0db0 0x561cbe9e15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.898+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbe9e1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.898+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbe8e6000 con 0x561cbe83c6b0 2023-11-27T15:30:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.898+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbe938840 con 0x561cbe9e0db0 2023-11-27T15:30:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.898+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbe931cd0 con 0x561cbe9e1b10 2023-11-27T15:30:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e0db0 0x561cbe9e15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbe9e0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba5d9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e0db0 0x561cbe9e15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:42236/0 (socket says 172.21.15.8:42236) 2023-11-27T15:30:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba5d9b700 1 -- 172.21.15.8:0/3024291225 learned_addr learned my addr 172.21.15.8:0/3024291225 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/3024291225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbe9e1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba5d9b700 1 -- 172.21.15.8:0/3024291225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 msgr2=0x561cbe9e1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba5d9b700 1 --2- 172.21.15.8:0/3024291225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbe9e1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba5d9b700 1 -- 172.21.15.8:0/3024291225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 msgr2=0x561cbe9e0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba5d9b700 1 --2- 172.21.15.8:0/3024291225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbe9e0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba5d9b700 1 -- 172.21.15.8:0/3024291225 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cbe7b35d0 con 0x561cbe9e0db0 2023-11-27T15:30:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.899+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/3024291225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbe9e1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:30:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.903+0000 7f6ba5d9b700 1 --2- 172.21.15.8:0/3024291225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e0db0 0x561cbe9e15d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009b80 tx=0x7f6b98009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95c1de4c7ecdccc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.903+0000 7f6b9cff9700 1 -- 172.21.15.8:0/3024291225 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98009650 con 0x561cbe9e0db0 2023-11-27T15:30:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.903+0000 7f6b9cff9700 1 -- 172.21.15.8:0/3024291225 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b9800ee50 con 0x561cbe9e0db0 2023-11-27T15:30:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.903+0000 7f6b9cff9700 1 -- 172.21.15.8:0/3024291225 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980123d0 con 0x561cbe9e0db0 2023-11-27T15:30:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.906+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3024291225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e0db0 msgr2=0x561cbe9e15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.906+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3024291225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e0db0 0x561cbe9e15d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009b80 tx=0x7f6b98009e90 comp rx=0 tx=0).stop 2023-11-27T15:30:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.906+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3024291225 shutdown_connections 2023-11-27T15:30:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.906+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3024291225 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbe9e1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.906+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3024291225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbe9e0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.906+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3024291225 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e0db0 0x561cbe9e15d0 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009b80 tx=0x7f6b98009e90 comp rx=0 tx=0).stop 2023-11-27T15:30:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.906+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3024291225 >> 172.21.15.8:0/3024291225 conn(0x561cbe9d5f50 msgr2=0x561cbe9db100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.907+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3024291225 shutdown_connections 2023-11-27T15:30:29.912 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.907+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3024291225 wait complete. 2023-11-27T15:30:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.908+0000 7f6bc0707b80 1 -- start start 2023-11-27T15:30:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.908+0000 7f6bc0707b80 1 Processor -- start 2023-11-27T15:30:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.909+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe83c6b0 0x561cbe9e9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.909+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbe9ea450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.909+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe9ef480 0x561cbe9f1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.909+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbe8d3880 con 0x561cbe9ef480 2023-11-27T15:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.909+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbe8d3520 con 0x561cbe83c6b0 2023-11-27T15:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.909+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbe7b58c0 con 0x561cbe9e1b10 2023-11-27T15:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.910+0000 7f6bc0707b80 10 client.0 _ll_register_callbacks cb 0x561cbe8f1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.910+0000 7f6ba659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe83c6b0 0x561cbe9e9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.910+0000 7f6ba659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe83c6b0 0x561cbe9e9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:42242/0 (socket says 172.21.15.8:42242) 2023-11-27T15:30:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.910+0000 7f6ba659c700 1 -- 172.21.15.8:0/3167137609 learned_addr learned my addr 172.21.15.8:0/3167137609 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.910+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe9ef480 0x561cbe9f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.910+0000 7f6ba659c700 1 -- 172.21.15.8:0/3167137609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 msgr2=0x561cbe9ea450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:30:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.910+0000 7f6ba659c700 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbe9ea450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.910+0000 7f6ba659c700 1 -- 172.21.15.8:0/3167137609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe9ef480 msgr2=0x561cbe9f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.910+0000 7f6ba659c700 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe9ef480 0x561cbe9f1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.910+0000 7f6ba659c700 1 -- 172.21.15.8:0/3167137609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b980097b0 con 0x561cbe83c6b0 2023-11-27T15:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.911+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3167137609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cbe9d3710 con 0x561cbe83c6b0 2023-11-27T15:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.911+0000 7f6ba659c700 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe83c6b0 0x561cbe9e9f10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000b5c0 tx=0x7f6ba0004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e768faba9ca9a8ea server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.911+0000 7f6b8dffb700 10 client.25706 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.911+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3167137609 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba009b280 con 0x561cbe83c6b0 2023-11-27T15:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.911+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3167137609 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6ba00a31e0 con 0x561cbe83c6b0 2023-11-27T15:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.911+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3167137609 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba009a310 con 0x561cbe83c6b0 2023-11-27T15:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.912+0000 7f6bc0707b80 10 client.25706 fetch_fsmap learned FSMap version 34 2023-11-27T15:30:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.912+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3167137609 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({fsmap=34}) v3 -- 0x561cbe8f8130 con 0x561cbe83c6b0 2023-11-27T15:30:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.912+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3167137609 <== mon.2 v2:172.21.15.29:3300/0 4 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6ba00a2d50 con 0x561cbe83c6b0 2023-11-27T15:30:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.912+0000 7f6bc0707b80 10 client.25706 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:30:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.912+0000 7f6bc0707b80 10 client.25706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:30:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.912+0000 7f6bc0707b80 20 client.25706 populate_metadata read hostname 'smithi008' 2023-11-27T15:30:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.912+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x561cbea06c30 0x561cbea09040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.913+0000 7f6bc0707b80 4 client.25706 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.913+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3167137609 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cbe8f4040 con 0x561cbea06c30 2023-11-27T15:30:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.914+0000 7f6ba5d9b700 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x561cbea06c30 0x561cbea09040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.917+0000 7f6ba5d9b700 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x561cbea06c30 0x561cbea09040 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f0b7df0bac5d0b2 server_cookie=66fe495be50cfa7c in_seq=0 out_seq=0 2023-11-27T15:30:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.917+0000 7f6b8dffb700 10 client.25706 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.936+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3167137609 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561cbe8f4040 con 0x561cbea06c30 2023-11-27T15:30:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.936+0000 7f6b8dffb700 10 client.25706 handle_command_reply: tid=0 2023-11-27T15:30:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.937+0000 7f6bc0707b80 1 client.25706 shutdown 2023-11-27T15:30:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.939+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3167137609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe83c6b0 msgr2=0x561cbe9e9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.939+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe83c6b0 0x561cbe9e9f10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000b5c0 tx=0x7f6ba0004f60 comp rx=0 tx=0).stop 2023-11-27T15:30:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.940+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3167137609 shutdown_connections 2023-11-27T15:30:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.940+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe9ef480 0x561cbe9f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.940+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe83c6b0 0x561cbe9e9f10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.940+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x561cbea06c30 0x561cbea09040 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.940+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3167137609 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbe9ea450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.940+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3167137609 >> 172.21.15.8:0/3167137609 conn(0x561cbe9d5f50 msgr2=0x561cbe9daee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.940+0000 7f6b8dffb700 0 client.25706 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.940+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3167137609 shutdown_connections 2023-11-27T15:30:29.944 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.941+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3167137609 wait complete. 2023-11-27T15:30:29.945 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.942+0000 7f6bc0707b80 20 client.25706 trim_cache size 0 max 16384 2023-11-27T15:30:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.983+0000 7f6bc0707b80 1 Processor -- start 2023-11-27T15:30:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.983+0000 7f6bc0707b80 1 -- start start 2023-11-27T15:30:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.983+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbe9e8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.984+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e1b10 0x561cbea168a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.984+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9ef480 0x561cbe9e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.984+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbe9db6b0 con 0x561cbe83c6b0 2023-11-27T15:30:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.984+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbea067f0 con 0x561cbe9e1b10 2023-11-27T15:30:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.984+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbe8f4070 con 0x561cbe9ef480 2023-11-27T15:30:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.984+0000 7f6ba6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9ef480 0x561cbe9e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.984+0000 7f6ba6d9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9ef480 0x561cbe9e89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:39050/0 (socket says 172.21.15.8:39050) 2023-11-27T15:30:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.984+0000 7f6ba6d9d700 1 -- 172.21.15.8:0/2470405777 learned_addr learned my addr 172.21.15.8:0/2470405777 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.985+0000 7f6ba5d9b700 1 --2- 172.21.15.8:0/2470405777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e1b10 0x561cbea168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.985+0000 7f6ba659c700 1 --2- 172.21.15.8:0/2470405777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbe9e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.985+0000 7f6ba6d9d700 1 -- 172.21.15.8:0/2470405777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e1b10 msgr2=0x561cbea168a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.985+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/2470405777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e1b10 0x561cbea168a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.985+0000 7f6ba6d9d700 1 -- 172.21.15.8:0/2470405777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 msgr2=0x561cbe9e8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.986+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/2470405777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbe9e8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.986+0000 7f6ba6d9d700 1 -- 172.21.15.8:0/2470405777 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cbe8f7e80 con 0x561cbe9ef480 2023-11-27T15:30:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.986+0000 7f6ba5d9b700 1 --2- 172.21.15.8:0/2470405777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e1b10 0x561cbea168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:30:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.986+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/2470405777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9ef480 0x561cbe9e89d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b90015410 tx=0x7f6b90010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f02cc2fc550430 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.987+0000 7f6b9d7fa700 1 -- 172.21.15.8:0/2470405777 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90015c60 con 0x561cbe9ef480 2023-11-27T15:30:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.987+0000 7f6b9d7fa700 1 -- 172.21.15.8:0/2470405777 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b9001d050 con 0x561cbe9ef480 2023-11-27T15:30:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.987+0000 7f6b9d7fa700 1 -- 172.21.15.8:0/2470405777 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b900145c0 con 0x561cbe9ef480 2023-11-27T15:30:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.988+0000 7f6bc0707b80 1 -- 172.21.15.8:0/2470405777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9ef480 msgr2=0x561cbe9e89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.988+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/2470405777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9ef480 0x561cbe9e89d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b90015410 tx=0x7f6b90010000 comp rx=0 tx=0).stop 2023-11-27T15:30:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.988+0000 7f6bc0707b80 1 -- 172.21.15.8:0/2470405777 shutdown_connections 2023-11-27T15:30:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.988+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/2470405777 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9ef480 0x561cbe9e89d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.988+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/2470405777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbe9e8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.988+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/2470405777 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9e1b10 0x561cbea168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.988+0000 7f6bc0707b80 1 -- 172.21.15.8:0/2470405777 >> 172.21.15.8:0/2470405777 conn(0x561cbe9d5f50 msgr2=0x561cbea0d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.988+0000 7f6bc0707b80 1 -- 172.21.15.8:0/2470405777 shutdown_connections 2023-11-27T15:30:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.989+0000 7f6bc0707b80 1 -- 172.21.15.8:0/2470405777 wait complete. 2023-11-27T15:30:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.989+0000 7f6bc0707b80 1 -- start start 2023-11-27T15:30:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.989+0000 7f6bc0707b80 1 Processor -- start 2023-11-27T15:30:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.990+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbea13d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.990+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbea142c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.990+0000 7f6bc0707b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9ef480 0x561cbea152d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.991+0000 7f6ba5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbea142c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.991+0000 7f6ba5d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbea142c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:39056/0 (socket says 172.21.15.8:39056) 2023-11-27T15:30:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.991+0000 7f6ba6d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9ef480 0x561cbea152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.991+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbea15810 con 0x561cbe83c6b0 2023-11-27T15:30:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.991+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbea15980 con 0x561cbe9ef480 2023-11-27T15:30:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.991+0000 7f6bc0707b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cbea15b40 con 0x561cbe9e1b10 2023-11-27T15:30:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.992+0000 7f6bc0707b80 10 client.0 _ll_register_callbacks cb 0x7f6ba009a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:30:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.991+0000 7f6ba6d9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9ef480 0x561cbea152d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:55432/0 (socket says 172.21.15.8:55432) 2023-11-27T15:30:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.992+0000 7f6ba6d9d700 1 -- 172.21.15.8:0/3374439336 learned_addr learned my addr 172.21.15.8:0/3374439336 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.992+0000 7f6ba659c700 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbea13d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.992+0000 7f6ba6d9d700 1 -- 172.21.15.8:0/3374439336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 msgr2=0x561cbea142c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.992+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbea142c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.992+0000 7f6ba6d9d700 1 -- 172.21.15.8:0/3374439336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 msgr2=0x561cbea13d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.992+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbea13d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.993+0000 7f6ba6d9d700 1 -- 172.21.15.8:0/3374439336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cbe7b35d0 con 0x561cbe9ef480 2023-11-27T15:30:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.993+0000 7f6b8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:30:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.993+0000 7f6ba6d9d700 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9ef480 0x561cbea152d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b90022640 tx=0x7f6b90022720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8709426d45ad5090 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.995+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3374439336 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9002b820 con 0x561cbe9ef480 2023-11-27T15:30:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.995+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3374439336 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6b9002b980 con 0x561cbe9ef480 2023-11-27T15:30:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.995+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3374439336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b9800ee50 con 0x561cbe9ef480 2023-11-27T15:30:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.997+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3374439336 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b90022bd0 con 0x561cbe9ef480 2023-11-27T15:30:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.999+0000 7f6bc0707b80 10 client.25712 fetch_fsmap learned FSMap version 34 2023-11-27T15:30:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.999+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3374439336 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f6b90007ce0 con 0x561cbe9ef480 2023-11-27T15:30:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:29.999+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3374439336 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b9800ee50 con 0x561cbe9ef480 2023-11-27T15:30:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.000+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3374439336 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 34) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6b90034990 con 0x561cbe9ef480 2023-11-27T15:30:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.000+0000 7f6bc0707b80 10 client.25712 fetch_fsmap finished waiting for FSMap version 34 2023-11-27T15:30:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.000+0000 7f6bc0707b80 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:30:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.000+0000 7f6bc0707b80 20 client.25712 populate_metadata read hostname 'smithi008' 2023-11-27T15:30:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.000+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x561cbea5b780 0x561cbea5db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.000+0000 7f6bc0707b80 4 client.25712 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:30:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.000+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3374439336 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561cbe900600 con 0x561cbea5b780 2023-11-27T15:30:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.001+0000 7f6ba659c700 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x561cbea5b780 0x561cbea5db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.006+0000 7f6ba659c700 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x561cbea5b780 0x561cbea5db90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9059603378a596 server_cookie=9737b0be358cf939 in_seq=0 out_seq=0 2023-11-27T15:30:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.006+0000 7f6b8dffb700 10 client.25712 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.542+0000 7f6b8dffb700 1 -- 172.21.15.8:0/3374439336 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561cbe900600 con 0x561cbea5b780 2023-11-27T15:30:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.542+0000 7f6b8dffb700 10 client.25712 handle_command_reply: tid=0 2023-11-27T15:30:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.542+0000 7f6bc0707b80 1 client.25712 shutdown 2023-11-27T15:30:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3374439336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9ef480 msgr2=0x561cbea152d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9ef480 0x561cbea152d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b90022640 tx=0x7f6b90022720 comp rx=0 tx=0).stop 2023-11-27T15:30:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3374439336 shutdown_connections 2023-11-27T15:30:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x561cbe9ef480 0x561cbea152d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x561cbe83c6b0 0x561cbea13d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x561cbea5b780 0x561cbea5db90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 1 --2- 172.21.15.8:0/3374439336 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x561cbe9e1b10 0x561cbea142c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3374439336 >> 172.21.15.8:0/3374439336 conn(0x561cbe9d5f50 msgr2=0x561cbe9e8160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6b8dffb700 0 client.25712 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3374439336 shutdown_connections 2023-11-27T15:30:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 1 -- 172.21.15.8:0/3374439336 wait complete. 2023-11-27T15:30:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.543+0000 7f6bc0707b80 20 client.25712 trim_cache size 0 max 16384 2023-11-27T15:30:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:30:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:30:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:30:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:30:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.544+0000 7f6bacdff700 1 -- 172.21.15.8:0/3689166852 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6b940b21e0 msgr2=0x7f6b940b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.544+0000 7f6bacdff700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6b940b21e0 0x7f6b940b4690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b900060b0 tx=0x7f6b90006040 comp rx=0 tx=0).stop 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.544+0000 7f6bacdff700 1 -- 172.21.15.8:0/3689166852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80734c0 msgr2=0x7f6ba8196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.544+0000 7f6bacdff700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80734c0 0x7f6ba8196740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00095c0 tx=0x7f6ba000f690 comp rx=0 tx=0).stop 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.545+0000 7f6bacdff700 1 -- 172.21.15.8:0/3689166852 shutdown_connections 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.545+0000 7f6bacdff700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ba8103ca0 0x7f6ba819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.545+0000 7f6bacdff700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6ba80734c0 0x7f6ba8196740 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.545+0000 7f6bacdff700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6b940b21e0 0x7f6b940b4690 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.545+0000 7f6bacdff700 1 --2- 172.21.15.8:0/3689166852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6ba8074a90 0x7f6ba8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.545+0000 7f6bacdff700 1 -- 172.21.15.8:0/3689166852 >> 172.21.15.8:0/3689166852 conn(0x7f6ba80fb810 msgr2=0x7f6ba80fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.545+0000 7f6bacdff700 1 -- 172.21.15.8:0/3689166852 shutdown_connections 2023-11-27T15:30:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:30.545+0000 7f6bacdff700 1 -- 172.21.15.8:0/3689166852 wait complete. 2023-11-27T15:30:31.397 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:30:30 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-e[155797]: ignoring --setuser ceph since I am not root 2023-11-27T15:30:31.397 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:30:30 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-e[155797]: ignoring --setgroup ceph since I am not root 2023-11-27T15:30:31.397 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:30:30 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-e[155797]: starting mds.e at 2023-11-27T15:30:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:31 smithi060 ceph-mon[131518]: pgmap v499: 129 pgs: 129 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 345 op/s 2023-11-27T15:30:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:31 smithi060 ceph-mon[131518]: daemon mds.e restarted 2023-11-27T15:30:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:31 smithi060 ceph-mon[131518]: osdmap e94: 12 total, 12 up, 12 in 2023-11-27T15:30:31.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:31 smithi008 ceph-mon[128788]: pgmap v499: 129 pgs: 129 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 345 op/s 2023-11-27T15:30:31.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:31 smithi008 ceph-mon[128788]: daemon mds.e restarted 2023-11-27T15:30:31.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:31 smithi008 ceph-mon[128788]: osdmap e94: 12 total, 12 up, 12 in 2023-11-27T15:30:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:31 smithi029 ceph-mon[131166]: pgmap v499: 129 pgs: 129 active+clean; 265 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 345 op/s 2023-11-27T15:30:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:31 smithi029 ceph-mon[131166]: daemon mds.e restarted 2023-11-27T15:30:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:31 smithi029 ceph-mon[131166]: osdmap e94: 12 total, 12 up, 12 in 2023-11-27T15:30:32.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:32 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.29:6834/4221144355,v1:172.21.15.29:6835/4221144355] up:boot 2023-11-27T15:30:32.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:32 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:30:32.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:32 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:30:32.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:32 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:30:32.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:32 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.29:6834/4221144355,v1:172.21.15.29:6835/4221144355] up:boot 2023-11-27T15:30:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:32 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:30:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:32 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:30:32.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:32 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:30:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:32 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.29:6834/4221144355,v1:172.21.15.29:6835/4221144355] up:boot 2023-11-27T15:30:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:32 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:30:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:32 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:30:32.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:32 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:30:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:33 smithi060 ceph-mon[131518]: pgmap v501: 129 pgs: 129 active+clean; 267 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 416 op/s 2023-11-27T15:30:33.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:33 smithi008 ceph-mon[128788]: pgmap v501: 129 pgs: 129 active+clean; 267 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 416 op/s 2023-11-27T15:30:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:33 smithi029 ceph-mon[131166]: pgmap v501: 129 pgs: 129 active+clean; 267 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 416 op/s 2023-11-27T15:30:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:35 smithi060 ceph-mon[131518]: pgmap v502: 129 pgs: 129 active+clean; 270 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 452 op/s 2023-11-27T15:30:35.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:35 smithi008 ceph-mon[128788]: pgmap v502: 129 pgs: 129 active+clean; 270 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 452 op/s 2023-11-27T15:30:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:35 smithi029 ceph-mon[131166]: pgmap v502: 129 pgs: 129 active+clean; 270 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 452 op/s 2023-11-27T15:30:37.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:37 smithi060 ceph-mon[131518]: pgmap v503: 129 pgs: 129 active+clean; 267 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 431 op/s 2023-11-27T15:30:37.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:37 smithi008 ceph-mon[128788]: pgmap v503: 129 pgs: 129 active+clean; 267 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 431 op/s 2023-11-27T15:30:37.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:37 smithi029 ceph-mon[131166]: pgmap v503: 129 pgs: 129 active+clean; 267 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 431 op/s 2023-11-27T15:30:39.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:39 smithi029 ceph-mon[131166]: pgmap v504: 129 pgs: 129 active+clean; 275 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 454 op/s 2023-11-27T15:30:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:39 smithi060 ceph-mon[131518]: pgmap v504: 129 pgs: 129 active+clean; 275 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 454 op/s 2023-11-27T15:30:39.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:39 smithi008 ceph-mon[128788]: pgmap v504: 129 pgs: 129 active+clean; 275 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 454 op/s 2023-11-27T15:30:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:41 smithi060 ceph-mon[131518]: pgmap v505: 129 pgs: 129 active+clean; 275 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 466 op/s 2023-11-27T15:30:41.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:41 smithi008 ceph-mon[128788]: pgmap v505: 129 pgs: 129 active+clean; 275 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 466 op/s 2023-11-27T15:30:41.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:41 smithi029 ceph-mon[131166]: pgmap v505: 129 pgs: 129 active+clean; 275 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 466 op/s 2023-11-27T15:30:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.754+0000 7f1499b9d700 1 -- 172.21.15.8:0/1110742731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14941022d0 msgr2=0x7f14941026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.754+0000 7f1499b9d700 1 --2- 172.21.15.8:0/1110742731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14941022d0 0x7f14941026e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1488009990 tx=0x7f1488009ca0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.754+0000 7f1499b9d700 1 -- 172.21.15.8:0/1110742731 shutdown_connections 2023-11-27T15:30:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.754+0000 7f1499b9d700 1 --2- 172.21.15.8:0/1110742731 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1494103e50 0x7f149410a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.754+0000 7f1499b9d700 1 --2- 172.21.15.8:0/1110742731 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14941022d0 0x7f14941026e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.754+0000 7f1499b9d700 1 --2- 172.21.15.8:0/1110742731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14941034c0 0x7f1494103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.754+0000 7f1499b9d700 1 -- 172.21.15.8:0/1110742731 >> 172.21.15.8:0/1110742731 conn(0x7f14940fd8f0 msgr2=0x7f14940ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.754+0000 7f1499b9d700 1 -- 172.21.15.8:0/1110742731 shutdown_connections 2023-11-27T15:30:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.754+0000 7f1499b9d700 1 -- 172.21.15.8:0/1110742731 wait complete. 2023-11-27T15:30:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.755+0000 7f1499b9d700 1 Processor -- start 2023-11-27T15:30:43.760 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.755+0000 7f1499b9d700 1 -- start start 2023-11-27T15:30:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f1499b9d700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14941022d0 0x7f1494198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f1499b9d700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14941034c0 0x7f14941985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f1499b9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1494103e50 0x7f149419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f1499b9d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149419fbf0 con 0x7f1494103e50 2023-11-27T15:30:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f1499b9d700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149419fd30 con 0x7f14941034c0 2023-11-27T15:30:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f1499b9d700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149419fec0 con 0x7f14941022d0 2023-11-27T15:30:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f1498b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14941022d0 0x7f1494198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f1498b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14941022d0 0x7f1494198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58888/0 (socket says 172.21.15.8:58888) 2023-11-27T15:30:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f1498b9b700 1 -- 172.21.15.8:0/1287232182 learned_addr learned my addr 172.21.15.8:0/1287232182 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f1493fff700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14941034c0 0x7f14941985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.756+0000 7f149939c700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1494103e50 0x7f149419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.757+0000 7f1498b9b700 1 -- 172.21.15.8:0/1287232182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14941034c0 msgr2=0x7f14941985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.757+0000 7f1498b9b700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14941034c0 0x7f14941985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.757+0000 7f1498b9b700 1 -- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1494103e50 msgr2=0x7f149419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.757+0000 7f1498b9b700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1494103e50 0x7f149419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.757+0000 7f1498b9b700 1 -- 172.21.15.8:0/1287232182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880095f0 con 0x7f14941022d0 2023-11-27T15:30:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.757+0000 7f149939c700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1494103e50 0x7f149419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:30:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.757+0000 7f1498b9b700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14941022d0 0x7f1494198070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1488000c00 tx=0x7f1488005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.758+0000 7f1491ffb700 1 -- 172.21.15.8:0/1287232182 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148801d070 con 0x7f14941022d0 2023-11-27T15:30:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.758+0000 7f1491ffb700 1 -- 172.21.15.8:0/1287232182 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1488021410 con 0x7f14941022d0 2023-11-27T15:30:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.758+0000 7f1499b9d700 1 -- 172.21.15.8:0/1287232182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149410dca0 con 0x7f14941022d0 2023-11-27T15:30:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.758+0000 7f1491ffb700 1 -- 172.21.15.8:0/1287232182 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488017650 con 0x7f14941022d0 2023-11-27T15:30:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.758+0000 7f1499b9d700 1 -- 172.21.15.8:0/1287232182 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149410db10 con 0x7f14941022d0 2023-11-27T15:30:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.761+0000 7f1491ffb700 1 -- 172.21.15.8:0/1287232182 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1488021580 con 0x7f14941022d0 2023-11-27T15:30:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.763+0000 7f1491ffb700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f14840a1150 0x7f14840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.763+0000 7f1491ffb700 1 -- 172.21.15.8:0/1287232182 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f14880cdea0 con 0x7f14941022d0 2023-11-27T15:30:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.763+0000 7f1493fff700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f14840a1150 0x7f14840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.764+0000 7f1493fff700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f14840a1150 0x7f14840a3600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1480009c60 tx=0x7f1480009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.764+0000 7f14ae6e4b80 1 Processor -- start 2023-11-27T15:30:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.765+0000 7f14ae6e4b80 1 -- start start 2023-11-27T15:30:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.765+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b638ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.765+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638cfb0 0x5629b638d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.765+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638dd60 0x5629b638e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.765+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b6292000 con 0x5629b638dd60 2023-11-27T15:30:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.765+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b62e4840 con 0x5629b638cfb0 2023-11-27T15:30:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.765+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b62ddcd0 con 0x5629b61e86b0 2023-11-27T15:30:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.765+0000 7f1493fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638cfb0 0x5629b638d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.765+0000 7f1493fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638cfb0 0x5629b638d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50002/0 (socket says 172.21.15.8:50002) 2023-11-27T15:30:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.765+0000 7f1493fff700 1 -- 172.21.15.8:0/538967828 learned_addr learned my addr 172.21.15.8:0/538967828 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.766+0000 7f1498b9b700 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b638ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.766+0000 7f149939c700 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638dd60 0x5629b638e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.766+0000 7f1493fff700 1 -- 172.21.15.8:0/538967828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 msgr2=0x5629b638ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.766+0000 7f1493fff700 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b638ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.766+0000 7f1493fff700 1 -- 172.21.15.8:0/538967828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638dd60 msgr2=0x5629b638e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.766+0000 7f1493fff700 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638dd60 0x5629b638e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.766+0000 7f1493fff700 1 -- 172.21.15.8:0/538967828 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629b615f5d0 con 0x5629b638cfb0 2023-11-27T15:30:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.766+0000 7f1498b9b700 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b638ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:30:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.766+0000 7f149939c700 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638dd60 0x5629b638e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:30:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.766+0000 7f1493fff700 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638cfb0 0x5629b638d820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f148000e4e0 tx=0x7f1480014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a51a6d4b3f93af39 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.767+0000 7f147effd700 1 -- 172.21.15.8:0/538967828 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1480026430 con 0x5629b638cfb0 2023-11-27T15:30:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.767+0000 7f147effd700 1 -- 172.21.15.8:0/538967828 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1480021070 con 0x5629b638cfb0 2023-11-27T15:30:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.767+0000 7f147effd700 1 -- 172.21.15.8:0/538967828 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148001d6e0 con 0x5629b638cfb0 2023-11-27T15:30:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.768+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/538967828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638cfb0 msgr2=0x5629b638d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.768+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638cfb0 0x5629b638d820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f148000e4e0 tx=0x7f1480014000 comp rx=0 tx=0).stop 2023-11-27T15:30:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.768+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/538967828 shutdown_connections 2023-11-27T15:30:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.768+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b638ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.768+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638dd60 0x5629b638e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.768+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/538967828 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638cfb0 0x5629b638d820 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.768+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/538967828 >> 172.21.15.8:0/538967828 conn(0x5629b63821a0 msgr2=0x5629b63872e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.768+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/538967828 shutdown_connections 2023-11-27T15:30:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.768+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/538967828 wait complete. 2023-11-27T15:30:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.769+0000 7f14ae6e4b80 1 -- start start 2023-11-27T15:30:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.769+0000 7f14ae6e4b80 1 Processor -- start 2023-11-27T15:30:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.769+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6395e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b63963d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b627f880 con 0x5629b638cfb0 2023-11-27T15:30:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b627f520 con 0x5629b638dd60 2023-11-27T15:30:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b61618c0 con 0x5629b61e86b0 2023-11-27T15:30:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f14ae6e4b80 10 client.0 _ll_register_callbacks cb 0x5629b629d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:30:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f1493fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b63963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f1493fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b63963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54720/0 (socket says 172.21.15.8:54720) 2023-11-27T15:30:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f1493fff700 1 -- 172.21.15.8:0/3330039852 learned_addr learned my addr 172.21.15.8:0/3330039852 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f149939c700 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f1498b9b700 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6395e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f1493fff700 1 -- 172.21.15.8:0/3330039852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 msgr2=0x5629b6395e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f1493fff700 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6395e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f1493fff700 1 -- 172.21.15.8:0/3330039852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 msgr2=0x5629b639d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f1493fff700 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.770+0000 7f1493fff700 1 -- 172.21.15.8:0/3330039852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1480017070 con 0x5629b638cfb0 2023-11-27T15:30:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.771+0000 7f1493fff700 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b63963d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f148000eb80 tx=0x7f148000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c963dcac50d2d11 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.771+0000 7f1473fff700 10 client.0 ms_handle_connect on v2:172.21.15.8:3300/0 2023-11-27T15:30:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.771+0000 7f1473fff700 1 -- 172.21.15.8:0/3330039852 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14800053d0 con 0x5629b638cfb0 2023-11-27T15:30:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.771+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3330039852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629b637f960 con 0x5629b638cfb0 2023-11-27T15:30:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.771+0000 7f1473fff700 1 -- 172.21.15.8:0/3330039852 <== mon.0 v2:172.21.15.8:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1480005530 con 0x5629b638cfb0 2023-11-27T15:30:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.772+0000 7f1473fff700 1 -- 172.21.15.8:0/3330039852 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14800056f0 con 0x5629b638cfb0 2023-11-27T15:30:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.772+0000 7f14ae6e4b80 10 client.15675 fetch_fsmap learned FSMap version 36 2023-11-27T15:30:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.772+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3330039852 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5629b62a4130 con 0x5629b638cfb0 2023-11-27T15:30:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.772+0000 7f1473fff700 1 -- 172.21.15.8:0/3330039852 <== mon.0 v2:172.21.15.8:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629b637f960 con 0x5629b638cfb0 2023-11-27T15:30:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.772+0000 7f1473fff700 1 -- 172.21.15.8:0/3330039852 <== mon.0 v2:172.21.15.8:3300/0 5 ==== fsmap(e 36) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1480003c10 con 0x5629b638cfb0 2023-11-27T15:30:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.772+0000 7f14ae6e4b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 36 2023-11-27T15:30:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.772+0000 7f14ae6e4b80 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:30:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.773+0000 7f14ae6e4b80 20 client.15675 populate_metadata read hostname 'smithi008' 2023-11-27T15:30:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.773+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5629b63a1c10 0x5629b63a4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.773+0000 7f14ae6e4b80 4 client.15675 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:30:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.773+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3330039852 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629b62a0040 con 0x5629b63a1c10 2023-11-27T15:30:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.773+0000 7f1498b9b700 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5629b63a1c10 0x5629b63a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.774+0000 7f1498b9b700 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5629b63a1c10 0x5629b63a4020 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8528891a3c14c87 server_cookie=e5e7eecf2442b2cc in_seq=0 out_seq=0 2023-11-27T15:30:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.774+0000 7f1473fff700 10 client.15675 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f1473fff700 1 -- 172.21.15.8:0/3330039852 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5629b62a0040 con 0x5629b63a1c10 2023-11-27T15:30:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f1473fff700 10 client.15675 handle_command_reply: tid=0 2023-11-27T15:30:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 client.15675 shutdown 2023-11-27T15:30:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3330039852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 msgr2=0x5629b63963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b63963d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f148000eb80 tx=0x7f148000eef0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3330039852 shutdown_connections 2023-11-27T15:30:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5629b63a1c10 0x5629b63a4020 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6395e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f1473fff700 0 client.15675 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3330039852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b63963d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3330039852 >> 172.21.15.8:0/3330039852 conn(0x5629b63821a0 msgr2=0x5629b6386a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3330039852 shutdown_connections 2023-11-27T15:30:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3330039852 wait complete. 2023-11-27T15:30:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.775+0000 7f14ae6e4b80 20 client.15675 trim_cache size 0 max 16384 2023-11-27T15:30:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f14ae6e4b80 1 Processor -- start 2023-11-27T15:30:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f14ae6e4b80 1 -- start start 2023-11-27T15:30:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6390b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b6391070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b6395a00 con 0x5629b638cfb0 2023-11-27T15:30:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b6387f50 con 0x5629b638dd60 2023-11-27T15:30:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b6397010 con 0x5629b61e86b0 2023-11-27T15:30:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f1493fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b6391070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f149939c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f1498b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6390b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f149939c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:50020/0 (socket says 172.21.15.8:50020) 2023-11-27T15:30:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.795+0000 7f149939c700 1 -- 172.21.15.8:0/3354432266 learned_addr learned my addr 172.21.15.8:0/3354432266 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.796+0000 7f1498b9b700 1 -- 172.21.15.8:0/3354432266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 msgr2=0x5629b639d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.796+0000 7f1498b9b700 1 --2- 172.21.15.8:0/3354432266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.796+0000 7f1498b9b700 1 -- 172.21.15.8:0/3354432266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 msgr2=0x5629b6391070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.796+0000 7f1498b9b700 1 --2- 172.21.15.8:0/3354432266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b6391070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.796+0000 7f1498b9b700 1 -- 172.21.15.8:0/3354432266 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629b62a3e80 con 0x5629b61e86b0 2023-11-27T15:30:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.796+0000 7f1498b9b700 1 --2- 172.21.15.8:0/3354432266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6390b30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f148809af80 tx=0x7f14880cb520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e4f36e36bfcfb10 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.796+0000 7f147f7fe700 1 -- 172.21.15.8:0/3354432266 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880b7070 con 0x5629b61e86b0 2023-11-27T15:30:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.796+0000 7f147f7fe700 1 -- 172.21.15.8:0/3354432266 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f148809f020 con 0x5629b61e86b0 2023-11-27T15:30:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.796+0000 7f147f7fe700 1 -- 172.21.15.8:0/3354432266 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488021860 con 0x5629b61e86b0 2023-11-27T15:30:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.797+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3354432266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 msgr2=0x5629b6390b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.797+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3354432266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6390b30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f148809af80 tx=0x7f14880cb520 comp rx=0 tx=0).stop 2023-11-27T15:30:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.797+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3354432266 shutdown_connections 2023-11-27T15:30:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.797+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3354432266 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6390b30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.797+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3354432266 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.797+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3354432266 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b6391070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.797+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3354432266 >> 172.21.15.8:0/3354432266 conn(0x5629b63821a0 msgr2=0x5629b63a8940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.797+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3354432266 shutdown_connections 2023-11-27T15:30:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.797+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3354432266 wait complete. 2023-11-27T15:30:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.798+0000 7f14ae6e4b80 1 -- start start 2023-11-27T15:30:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.798+0000 7f14ae6e4b80 1 Processor -- start 2023-11-27T15:30:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6398c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b63991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f14ae6e4b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b639c700 con 0x5629b638cfb0 2023-11-27T15:30:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b639c870 con 0x5629b638dd60 2023-11-27T15:30:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f14ae6e4b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629b639ca30 con 0x5629b61e86b0 2023-11-27T15:30:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f14ae6e4b80 10 client.0 _ll_register_callbacks cb 0x7f148000e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:30:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f1498b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6398c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f1498b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6398c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:58932/0 (socket says 172.21.15.8:58932) 2023-11-27T15:30:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f1498b9b700 1 -- 172.21.15.8:0/3909895258 learned_addr learned my addr 172.21.15.8:0/3909895258 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f1493fff700 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b63991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.799+0000 7f149939c700 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f1498b9b700 1 -- 172.21.15.8:0/3909895258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 msgr2=0x5629b639c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f1498b9b700 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f1498b9b700 1 -- 172.21.15.8:0/3909895258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 msgr2=0x5629b63991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f1498b9b700 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b63991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f1498b9b700 1 -- 172.21.15.8:0/3909895258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149410db10 con 0x5629b61e86b0 2023-11-27T15:30:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f1498b9b700 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6398c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1488097a60 tx=0x7f1488021c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7c5867132e8c199 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f1473fff700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:30:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f1473fff700 1 -- 172.21.15.8:0/3909895258 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880b7070 con 0x5629b61e86b0 2023-11-27T15:30:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f1473fff700 1 -- 172.21.15.8:0/3909895258 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1488099390 con 0x5629b61e86b0 2023-11-27T15:30:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3909895258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1480021070 con 0x5629b61e86b0 2023-11-27T15:30:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f1473fff700 1 -- 172.21.15.8:0/3909895258 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1488021860 con 0x5629b61e86b0 2023-11-27T15:30:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.800+0000 7f14ae6e4b80 10 client.25753 fetch_fsmap learned FSMap version 36 2023-11-27T15:30:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.801+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3909895258 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5629b60f4c00 con 0x5629b61e86b0 2023-11-27T15:30:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.801+0000 7f1473fff700 1 -- 172.21.15.8:0/3909895258 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1480021070 con 0x5629b61e86b0 2023-11-27T15:30:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.801+0000 7f1473fff700 1 -- 172.21.15.8:0/3909895258 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 36) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f14880a2f20 con 0x5629b61e86b0 2023-11-27T15:30:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.801+0000 7f14ae6e4b80 10 client.25753 fetch_fsmap finished waiting for FSMap version 36 2023-11-27T15:30:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.801+0000 7f14ae6e4b80 10 client.25753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:30:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.801+0000 7f14ae6e4b80 20 client.25753 populate_metadata read hostname 'smithi008' 2023-11-27T15:30:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.801+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5629b63f61a0 0x5629b63f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.801+0000 7f14ae6e4b80 4 client.25753 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:30:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.801+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3909895258 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629b62ac600 con 0x5629b63f61a0 2023-11-27T15:30:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.801+0000 7f1493fff700 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5629b63f61a0 0x5629b63f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.802+0000 7f1493fff700 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5629b63f61a0 0x5629b63f85b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241eaff624ea6775 server_cookie=daf934d4d218c98 in_seq=0 out_seq=0 2023-11-27T15:30:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:43.802+0000 7f1473fff700 10 client.25753 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:43.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:43 smithi060 ceph-mon[131518]: pgmap v506: 129 pgs: 129 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 472 op/s 2023-11-27T15:30:43.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:43 smithi008 ceph-mon[128788]: pgmap v506: 129 pgs: 129 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 472 op/s 2023-11-27T15:30:43.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:43 smithi029 ceph-mon[131166]: pgmap v506: 129 pgs: 129 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 472 op/s 2023-11-27T15:30:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f1473fff700 1 -- 172.21.15.8:0/3909895258 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629b62ac600 con 0x5629b63f61a0 2023-11-27T15:30:44.132 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f1473fff700 10 client.25753 handle_command_reply: tid=0 2023-11-27T15:30:44.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f14ae6e4b80 1 client.25753 shutdown 2023-11-27T15:30:44.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3909895258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 msgr2=0x5629b6398c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:44.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6398c60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1488097a60 tx=0x7f1488021c30 comp rx=0 tx=0).stop 2023-11-27T15:30:44.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3909895258 shutdown_connections 2023-11-27T15:30:44.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x5629b63f61a0 0x5629b63f85b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:44.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x5629b61e86b0 0x5629b6398c60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:44.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x5629b638dd60 0x5629b639c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:44.133 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f14ae6e4b80 1 --2- 172.21.15.8:0/3909895258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x5629b638cfb0 0x5629b63991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:44.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3909895258 >> 172.21.15.8:0/3909895258 conn(0x5629b63821a0 msgr2=0x5629b6395e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:44.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f1473fff700 0 client.25753 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:44.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.128+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3909895258 shutdown_connections 2023-11-27T15:30:44.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.129+0000 7f14ae6e4b80 1 -- 172.21.15.8:0/3909895258 wait complete. 2023-11-27T15:30:44.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.129+0000 7f14ae6e4b80 20 client.25753 trim_cache size 0 max 16384 2023-11-27T15:30:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:30:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:30:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:30:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:30:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.129+0000 7f1499b9d700 1 -- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f14840a1150 msgr2=0x7f14840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.129+0000 7f1499b9d700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f14840a1150 0x7f14840a3600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1480009c60 tx=0x7f1480009260 comp rx=0 tx=0).stop 2023-11-27T15:30:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.129+0000 7f1499b9d700 1 -- 172.21.15.8:0/1287232182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14941022d0 msgr2=0x7f1494198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.129+0000 7f1499b9d700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14941022d0 0x7f1494198070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1488000c00 tx=0x7f1488005740 comp rx=0 tx=0).stop 2023-11-27T15:30:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.130+0000 7f1499b9d700 1 -- 172.21.15.8:0/1287232182 shutdown_connections 2023-11-27T15:30:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.130+0000 7f1499b9d700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f14840a1150 0x7f14840a3600 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.130+0000 7f1499b9d700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f14941022d0 0x7f1494198070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.130+0000 7f1499b9d700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1494103e50 0x7f149419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.130+0000 7f1499b9d700 1 --2- 172.21.15.8:0/1287232182 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f14941034c0 0x7f14941985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.130+0000 7f1499b9d700 1 -- 172.21.15.8:0/1287232182 >> 172.21.15.8:0/1287232182 conn(0x7f14940fd8f0 msgr2=0x7f14940ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.130+0000 7f1499b9d700 1 -- 172.21.15.8:0/1287232182 shutdown_connections 2023-11-27T15:30:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:44.130+0000 7f1499b9d700 1 -- 172.21.15.8:0/1287232182 wait complete. 2023-11-27T15:30:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:45 smithi060 ceph-mon[131518]: pgmap v507: 129 pgs: 129 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 407 op/s 2023-11-27T15:30:45.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:45 smithi008 ceph-mon[128788]: pgmap v507: 129 pgs: 129 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 407 op/s 2023-11-27T15:30:45.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:45 smithi029 ceph-mon[131166]: pgmap v507: 129 pgs: 129 active+clean; 283 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 407 op/s 2023-11-27T15:30:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:47 smithi060 ceph-mon[131518]: pgmap v508: 129 pgs: 129 active+clean; 284 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 426 op/s 2023-11-27T15:30:47.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:47 smithi008 ceph-mon[128788]: pgmap v508: 129 pgs: 129 active+clean; 284 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 426 op/s 2023-11-27T15:30:47.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:47 smithi029 ceph-mon[131166]: pgmap v508: 129 pgs: 129 active+clean; 284 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 426 op/s 2023-11-27T15:30:49.649 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:49 smithi029 ceph-mon[131166]: pgmap v509: 129 pgs: 129 active+clean; 286 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 467 op/s 2023-11-27T15:30:49.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:49 smithi060 ceph-mon[131518]: pgmap v509: 129 pgs: 129 active+clean; 286 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 467 op/s 2023-11-27T15:30:49.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:49 smithi008 ceph-mon[128788]: pgmap v509: 129 pgs: 129 active+clean; 286 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 467 op/s 2023-11-27T15:30:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:51 smithi060 ceph-mon[131518]: pgmap v510: 129 pgs: 129 active+clean; 291 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 424 op/s 2023-11-27T15:30:51.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:51 smithi008 ceph-mon[128788]: pgmap v510: 129 pgs: 129 active+clean; 291 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 424 op/s 2023-11-27T15:30:51.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:51 smithi029 ceph-mon[131166]: pgmap v510: 129 pgs: 129 active+clean; 291 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 424 op/s 2023-11-27T15:30:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:53 smithi060 ceph-mon[131518]: pgmap v511: 129 pgs: 129 active+clean; 293 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 485 op/s 2023-11-27T15:30:53.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:53 smithi008 ceph-mon[128788]: pgmap v511: 129 pgs: 129 active+clean; 293 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 485 op/s 2023-11-27T15:30:53.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:53 smithi029 ceph-mon[131166]: pgmap v511: 129 pgs: 129 active+clean; 293 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 485 op/s 2023-11-27T15:30:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:55 smithi060 ceph-mon[131518]: pgmap v512: 129 pgs: 129 active+clean; 300 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 436 op/s 2023-11-27T15:30:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:55 smithi008 ceph-mon[128788]: pgmap v512: 129 pgs: 129 active+clean; 300 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 436 op/s 2023-11-27T15:30:55.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:55 smithi029 ceph-mon[131166]: pgmap v512: 129 pgs: 129 active+clean; 300 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 436 op/s 2023-11-27T15:30:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.323+0000 7f48193ff700 1 -- 172.21.15.8:0/1548948445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4814101e70 msgr2=0x7f4814108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.323+0000 7f48193ff700 1 --2- 172.21.15.8:0/1548948445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4814101e70 0x7f4814108710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f480c009f90 tx=0x7f480c00b330 comp rx=0 tx=0).stop 2023-11-27T15:30:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.323+0000 7f48193ff700 1 -- 172.21.15.8:0/1548948445 shutdown_connections 2023-11-27T15:30:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.323+0000 7f48193ff700 1 --2- 172.21.15.8:0/1548948445 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4814100230 0x7f4814100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.323+0000 7f48193ff700 1 --2- 172.21.15.8:0/1548948445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4814101e70 0x7f4814108710 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.323+0000 7f48193ff700 1 --2- 172.21.15.8:0/1548948445 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f48141014c0 0x7f4814101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.323+0000 7f48193ff700 1 -- 172.21.15.8:0/1548948445 >> 172.21.15.8:0/1548948445 conn(0x7f48140fb810 msgr2=0x7f48140fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.323+0000 7f48193ff700 1 -- 172.21.15.8:0/1548948445 shutdown_connections 2023-11-27T15:30:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.323+0000 7f48193ff700 1 -- 172.21.15.8:0/1548948445 wait complete. 2023-11-27T15:30:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.324+0000 7f48193ff700 1 Processor -- start 2023-11-27T15:30:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48193ff700 1 -- start start 2023-11-27T15:30:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48193ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4814100230 0x7f4814198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48193ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48141014c0 0x7f4814198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48193ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4814101e70 0x7f481419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48193ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481419fe80 con 0x7f48141014c0 2023-11-27T15:30:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48193ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481419ffc0 con 0x7f4814101e70 2023-11-27T15:30:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48193ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48141a0150 con 0x7f4814100230 2023-11-27T15:30:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48127fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48141014c0 0x7f4814198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48137fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4814101e70 0x7f481419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48127fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48141014c0 0x7f4814198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47024/0 (socket says 172.21.15.8:47024) 2023-11-27T15:30:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48137fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4814101e70 0x7f481419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:59790/0 (socket says 172.21.15.8:59790) 2023-11-27T15:30:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f48127fc700 1 -- 172.21.15.8:0/674500504 learned_addr learned my addr 172.21.15.8:0/674500504 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.325+0000 7f4812ffd700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4814100230 0x7f4814198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f48137fe700 1 -- 172.21.15.8:0/674500504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4814100230 msgr2=0x7f4814198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f48137fe700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4814100230 0x7f4814198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f48137fe700 1 -- 172.21.15.8:0/674500504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48141014c0 msgr2=0x7f4814198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f48137fe700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48141014c0 0x7f4814198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f48137fe700 1 -- 172.21.15.8:0/674500504 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48080095f0 con 0x7f4814101e70 2023-11-27T15:30:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f48137fe700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4814101e70 0x7f481419f8b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f480c009f90 tx=0x7f480c004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f47fbfff700 1 -- 172.21.15.8:0/674500504 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c00e040 con 0x7f4814101e70 2023-11-27T15:30:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f47fbfff700 1 -- 172.21.15.8:0/674500504 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f480c0093c0 con 0x7f4814101e70 2023-11-27T15:30:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f48193ff700 1 -- 172.21.15.8:0/674500504 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480c009c40 con 0x7f4814101e70 2023-11-27T15:30:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f47fbfff700 1 -- 172.21.15.8:0/674500504 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c01b6c0 con 0x7f4814101e70 2023-11-27T15:30:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.326+0000 7f48193ff700 1 -- 172.21.15.8:0/674500504 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481410bc80 con 0x7f4814101e70 2023-11-27T15:30:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.330+0000 7f47fbfff700 1 -- 172.21.15.8:0/674500504 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f480c01d470 con 0x7f4814101e70 2023-11-27T15:30:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.331+0000 7f47fbfff700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47fc0a1220 0x7f47fc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.331+0000 7f482cd15b80 1 Processor -- start 2023-11-27T15:30:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.332+0000 7f47fbfff700 1 -- 172.21.15.8:0/674500504 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f480c0cdd60 con 0x7f4814101e70 2023-11-27T15:30:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.332+0000 7f482cd15b80 1 -- start start 2023-11-27T15:30:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.332+0000 7f4812ffd700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47fc0a1220 0x7f47fc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.332+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 0x55d8d11bc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.332+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11bd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.332+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bdb10 0x55d8d11bdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.332+0000 7f482cd15b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d10c2000 con 0x55d8d11bdb10 2023-11-27T15:30:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.332+0000 7f482cd15b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d1114840 con 0x55d8d10186b0 2023-11-27T15:30:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.332+0000 7f482cd15b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d110dcd0 con 0x55d8d11bcdb0 2023-11-27T15:30:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f48127fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11bd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f48127fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11bd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:53018/0 (socket says 172.21.15.8:53018) 2023-11-27T15:30:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f48127fc700 1 -- 172.21.15.8:0/2938091776 learned_addr learned my addr 172.21.15.8:0/2938091776 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f48137fe700 1 --2- 172.21.15.8:0/2938091776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bdb10 0x55d8d11bdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f4812ffd700 1 --2- 172.21.15.8:0/2938091776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 0x55d8d11bc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f48127fc700 1 -- 172.21.15.8:0/2938091776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 msgr2=0x55d8d11bc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:57.340 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f48127fc700 1 --2- 172.21.15.8:0/2938091776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 0x55d8d11bc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f48127fc700 1 -- 172.21.15.8:0/2938091776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bdb10 msgr2=0x55d8d11bdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f48127fc700 1 --2- 172.21.15.8:0/2938091776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bdb10 0x55d8d11bdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f48127fc700 1 -- 172.21.15.8:0/2938091776 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8d0f8f5d0 con 0x55d8d11bcdb0 2023-11-27T15:30:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f4812ffd700 1 --2- 172.21.15.8:0/2938091776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 0x55d8d11bc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:30:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f4812ffd700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47fc0a1220 0x7f47fc0a36d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4804009cb0 tx=0x7f4804011720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.333+0000 7f48127fc700 1 --2- 172.21.15.8:0/2938091776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11bd5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4808011510 tx=0x7f4808011820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c7c70b069a55094 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.334+0000 7f47f97fa700 1 -- 172.21.15.8:0/2938091776 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808019630 con 0x55d8d11bcdb0 2023-11-27T15:30:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.334+0000 7f47f97fa700 1 -- 172.21.15.8:0/2938091776 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4808019c50 con 0x55d8d11bcdb0 2023-11-27T15:30:57.341 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.334+0000 7f47f97fa700 1 -- 172.21.15.8:0/2938091776 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480800ecb0 con 0x55d8d11bcdb0 2023-11-27T15:30:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.335+0000 7f482cd15b80 1 -- 172.21.15.8:0/2938091776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 msgr2=0x55d8d11bd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.335+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2938091776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11bd5d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4808011510 tx=0x7f4808011820 comp rx=0 tx=0).stop 2023-11-27T15:30:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.335+0000 7f482cd15b80 1 -- 172.21.15.8:0/2938091776 shutdown_connections 2023-11-27T15:30:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.335+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2938091776 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bdb10 0x55d8d11bdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.335+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2938091776 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 0x55d8d11bc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.335+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2938091776 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11bd5d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.335+0000 7f482cd15b80 1 -- 172.21.15.8:0/2938091776 >> 172.21.15.8:0/2938091776 conn(0x55d8d11b1f50 msgr2=0x55d8d11b7100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:57.342 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.336+0000 7f482cd15b80 1 -- 172.21.15.8:0/2938091776 shutdown_connections 2023-11-27T15:30:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.336+0000 7f482cd15b80 1 -- 172.21.15.8:0/2938091776 wait complete. 2023-11-27T15:30:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.337+0000 7f482cd15b80 1 -- start start 2023-11-27T15:30:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.337+0000 7f482cd15b80 1 Processor -- start 2023-11-27T15:30:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.337+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 0x55d8d11ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.337+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bcdb0 0x55d8d11ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.337+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 0x55d8d11d5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.337+0000 7f482cd15b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d10af880 con 0x55d8d11bcdb0 2023-11-27T15:30:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.337+0000 7f482cd15b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d10af520 con 0x55d8d10186b0 2023-11-27T15:30:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.337+0000 7f482cd15b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d0f918c0 con 0x55d8d11bdb10 2023-11-27T15:30:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f482cd15b80 10 client.0 _ll_register_callbacks cb 0x55d8d10cd720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:30:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f48127fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bcdb0 0x55d8d11ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f48137fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 0x55d8d11d5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f48127fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bcdb0 0x55d8d11ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47038/0 (socket says 172.21.15.8:47038) 2023-11-27T15:30:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f4812ffd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 0x55d8d11ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f48127fc700 1 -- 172.21.15.8:0/3241466495 learned_addr learned my addr 172.21.15.8:0/3241466495 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f48137fe700 1 -- 172.21.15.8:0/3241466495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 msgr2=0x55d8d11ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f48137fe700 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 0x55d8d11ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f48137fe700 1 -- 172.21.15.8:0/3241466495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bcdb0 msgr2=0x55d8d11ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f48137fe700 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bcdb0 0x55d8d11ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.338+0000 7f48137fe700 1 -- 172.21.15.8:0/3241466495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4808009cf0 con 0x55d8d11bdb10 2023-11-27T15:30:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.339+0000 7f48137fe700 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 0x55d8d11d5a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f480c00b6c0 tx=0x7f480c09a7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b9d40e973e0ee5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.339+0000 7f47f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:30:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.339+0000 7f47f6ff5700 1 -- 172.21.15.8:0/3241466495 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c09ae70 con 0x55d8d11bdb10 2023-11-27T15:30:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.339+0000 7f47f6ff5700 1 -- 172.21.15.8:0/3241466495 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f480c09eea0 con 0x55d8d11bdb10 2023-11-27T15:30:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.339+0000 7f482cd15b80 1 -- 172.21.15.8:0/3241466495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8d11af710 con 0x55d8d11bdb10 2023-11-27T15:30:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.339+0000 7f47f6ff5700 1 -- 172.21.15.8:0/3241466495 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f480c0a3230 con 0x55d8d11bdb10 2023-11-27T15:30:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.339+0000 7f482cd15b80 10 client.25765 fetch_fsmap learned FSMap version 36 2023-11-27T15:30:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.339+0000 7f482cd15b80 1 -- 172.21.15.8:0/3241466495 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d8d10d4130 con 0x55d8d11bdb10 2023-11-27T15:30:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.339+0000 7f47f6ff5700 1 -- 172.21.15.8:0/3241466495 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8d11af710 con 0x55d8d11bdb10 2023-11-27T15:30:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.340+0000 7f47f6ff5700 1 -- 172.21.15.8:0/3241466495 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 36) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f480c0a2da0 con 0x55d8d11bdb10 2023-11-27T15:30:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.340+0000 7f482cd15b80 10 client.25765 fetch_fsmap finished waiting for FSMap version 36 2023-11-27T15:30:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.340+0000 7f482cd15b80 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:30:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.340+0000 7f482cd15b80 20 client.25765 populate_metadata read hostname 'smithi008' 2023-11-27T15:30:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.340+0000 7f482cd15b80 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d8d11d9f70 0x55d8d11dc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.340+0000 7f482cd15b80 4 client.25765 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:30:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.340+0000 7f482cd15b80 1 -- 172.21.15.8:0/3241466495 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8d10d0040 con 0x55d8d11d9f70 2023-11-27T15:30:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.340+0000 7f4812ffd700 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d8d11d9f70 0x55d8d11dc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.341+0000 7f4812ffd700 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d8d11d9f70 0x55d8d11dc380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf06ecd643350aa4 server_cookie=e7e5971104af1deb in_seq=0 out_seq=0 2023-11-27T15:30:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.341+0000 7f47f6ff5700 10 client.25765 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.342+0000 7f47f6ff5700 1 -- 172.21.15.8:0/3241466495 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d8d10d0040 con 0x55d8d11d9f70 2023-11-27T15:30:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.342+0000 7f47f6ff5700 10 client.25765 handle_command_reply: tid=0 2023-11-27T15:30:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.342+0000 7f482cd15b80 1 client.25765 shutdown 2023-11-27T15:30:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 1 -- 172.21.15.8:0/3241466495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 msgr2=0x55d8d11d5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 0x55d8d11d5a00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f480c00b6c0 tx=0x7f480c09a7d0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 1 -- 172.21.15.8:0/3241466495 shutdown_connections 2023-11-27T15:30:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d8d11d9f70 0x55d8d11dc380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 0x55d8d11d5a00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d10186b0 0x55d8d11ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f47f6ff5700 0 client.25765 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 1 --2- 172.21.15.8:0/3241466495 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d11bcdb0 0x55d8d11ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 1 -- 172.21.15.8:0/3241466495 >> 172.21.15.8:0/3241466495 conn(0x55d8d11b1f50 msgr2=0x55d8d11b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 1 -- 172.21.15.8:0/3241466495 shutdown_connections 2023-11-27T15:30:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 1 -- 172.21.15.8:0/3241466495 wait complete. 2023-11-27T15:30:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.343+0000 7f482cd15b80 20 client.25765 trim_cache size 0 max 16384 2023-11-27T15:30:57.369 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.366+0000 7f482cd15b80 1 Processor -- start 2023-11-27T15:30:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.366+0000 7f482cd15b80 1 -- start start 2023-11-27T15:30:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.366+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 0x55d8d11b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.366+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.366+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bdb10 0x55d8d11cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.366+0000 7f482cd15b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d10d0040 con 0x55d8d10186b0 2023-11-27T15:30:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.366+0000 7f482cd15b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d11cd930 con 0x55d8d11bdb10 2023-11-27T15:30:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.366+0000 7f482cd15b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d11cdff0 con 0x55d8d11bcdb0 2023-11-27T15:30:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f48127fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f4812ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 0x55d8d11b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.371 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f4812ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 0x55d8d11b81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47046/0 (socket says 172.21.15.8:47046) 2023-11-27T15:30:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f48137fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bdb10 0x55d8d11cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f4812ffd700 1 -- 172.21.15.8:0/2827073689 learned_addr learned my addr 172.21.15.8:0/2827073689 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f48127fc700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:53032/0 (socket says 172.21.15.8:53032) 2023-11-27T15:30:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f48127fc700 1 -- 172.21.15.8:0/2827073689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bdb10 msgr2=0x55d8d11cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f48127fc700 1 --2- 172.21.15.8:0/2827073689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bdb10 0x55d8d11cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f48127fc700 1 -- 172.21.15.8:0/2827073689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 msgr2=0x55d8d11b81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f48127fc700 1 --2- 172.21.15.8:0/2827073689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 0x55d8d11b81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.367+0000 7f48127fc700 1 -- 172.21.15.8:0/2827073689 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8d10d3e80 con 0x55d8d11bcdb0 2023-11-27T15:30:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.368+0000 7f48127fc700 1 --2- 172.21.15.8:0/2827073689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d59c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4808005f50 tx=0x7f4808011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a6b74d83fdc03d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.368+0000 7f47f9ffb700 1 -- 172.21.15.8:0/2827073689 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48080050c0 con 0x55d8d11bcdb0 2023-11-27T15:30:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.368+0000 7f47f9ffb700 1 -- 172.21.15.8:0/2827073689 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4808019430 con 0x55d8d11bcdb0 2023-11-27T15:30:57.375 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.368+0000 7f47f9ffb700 1 -- 172.21.15.8:0/2827073689 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48080038e0 con 0x55d8d11bcdb0 2023-11-27T15:30:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.369+0000 7f482cd15b80 1 -- 172.21.15.8:0/2827073689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 msgr2=0x55d8d11d59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.369+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2827073689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d59c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4808005f50 tx=0x7f4808011440 comp rx=0 tx=0).stop 2023-11-27T15:30:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.369+0000 7f482cd15b80 1 -- 172.21.15.8:0/2827073689 shutdown_connections 2023-11-27T15:30:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.369+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2827073689 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bdb10 0x55d8d11cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.369+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2827073689 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 0x55d8d11b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.369+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2827073689 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d59c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.369+0000 7f482cd15b80 1 -- 172.21.15.8:0/2827073689 >> 172.21.15.8:0/2827073689 conn(0x55d8d11b1f50 msgr2=0x55d8d11b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.369+0000 7f482cd15b80 1 -- 172.21.15.8:0/2827073689 shutdown_connections 2023-11-27T15:30:57.376 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.369+0000 7f482cd15b80 1 -- 172.21.15.8:0/2827073689 wait complete. 2023-11-27T15:30:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.370+0000 7f482cd15b80 1 -- start start 2023-11-27T15:30:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.370+0000 7f482cd15b80 1 Processor -- start 2023-11-27T15:30:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.370+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 0x55d8d11d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.370+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.370+0000 7f482cd15b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 0x55d8d11d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.370+0000 7f482cd15b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d11d4d00 con 0x55d8d10186b0 2023-11-27T15:30:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.370+0000 7f482cd15b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d11d4e70 con 0x55d8d11bcdb0 2023-11-27T15:30:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.370+0000 7f482cd15b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d11d5030 con 0x55d8d11bdb10 2023-11-27T15:30:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f482cd15b80 10 client.0 _ll_register_callbacks cb 0x55d8d10cd720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:30:57.377 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f48127fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f4812ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 0x55d8d11d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f48137fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 0x55d8d11d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f4812ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 0x55d8d11d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47050/0 (socket says 172.21.15.8:47050) 2023-11-27T15:30:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f48127fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:59828/0 (socket says 172.21.15.8:59828) 2023-11-27T15:30:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f4812ffd700 1 -- 172.21.15.8:0/2961659560 learned_addr learned my addr 172.21.15.8:0/2961659560 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:30:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f48137fe700 1 -- 172.21.15.8:0/2961659560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bcdb0 msgr2=0x55d8d11d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f48137fe700 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f48137fe700 1 -- 172.21.15.8:0/2961659560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 msgr2=0x55d8d11d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:30:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f48137fe700 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 0x55d8d11d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f48137fe700 1 -- 172.21.15.8:0/2961659560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8d0f8f5d0 con 0x55d8d11bdb10 2023-11-27T15:30:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.371+0000 7f48127fc700 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:30:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.372+0000 7f48137fe700 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 0x55d8d11d47c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f480c0a2d00 tx=0x7f480c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30ad7e6759d635a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:30:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.372+0000 7f47f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:30:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.372+0000 7f47f6ff5700 1 -- 172.21.15.8:0/2961659560 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c09b2d0 con 0x55d8d11bdb10 2023-11-27T15:30:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.372+0000 7f47f6ff5700 1 -- 172.21.15.8:0/2961659560 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f480c09eea0 con 0x55d8d11bdb10 2023-11-27T15:30:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.372+0000 7f482cd15b80 1 -- 172.21.15.8:0/2961659560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4808019c50 con 0x55d8d11bdb10 2023-11-27T15:30:57.379 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.372+0000 7f47f6ff5700 1 -- 172.21.15.8:0/2961659560 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f480c09ae70 con 0x55d8d11bdb10 2023-11-27T15:30:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.373+0000 7f482cd15b80 10 client.25771 fetch_fsmap learned FSMap version 36 2023-11-27T15:30:57.380 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.373+0000 7f482cd15b80 1 -- 172.21.15.8:0/2961659560 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55d8d0f24c00 con 0x55d8d11bdb10 2023-11-27T15:30:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.373+0000 7f47f6ff5700 1 -- 172.21.15.8:0/2961659560 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f480c0b7070 con 0x55d8d11bdb10 2023-11-27T15:30:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.373+0000 7f47f6ff5700 1 -- 172.21.15.8:0/2961659560 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 36) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f480c096c60 con 0x55d8d11bdb10 2023-11-27T15:30:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.373+0000 7f482cd15b80 10 client.25771 fetch_fsmap finished waiting for FSMap version 36 2023-11-27T15:30:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.373+0000 7f482cd15b80 10 client.25771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:30:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.373+0000 7f482cd15b80 20 client.25771 populate_metadata read hostname 'smithi008' 2023-11-27T15:30:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.373+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d8d122e280 0x55d8d1230690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:30:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.373+0000 7f482cd15b80 4 client.25771 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:30:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.373+0000 7f482cd15b80 1 -- 172.21.15.8:0/2961659560 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d8d10dc600 con 0x55d8d122e280 2023-11-27T15:30:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.374+0000 7f4812ffd700 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d8d122e280 0x55d8d1230690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:30:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.375+0000 7f47f6ff5700 10 client.25771 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:57.382 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:57.375+0000 7f4812ffd700 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d8d122e280 0x55d8d1230690 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f47bd50809eb592 server_cookie=f36ea4c0fc4e2ea0 in_seq=0 out_seq=0 2023-11-27T15:30:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:57 smithi060 ceph-mon[131518]: pgmap v513: 129 pgs: 129 active+clean; 302 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 423 op/s 2023-11-27T15:30:57.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:57 smithi008 ceph-mon[128788]: pgmap v513: 129 pgs: 129 active+clean; 302 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 423 op/s 2023-11-27T15:30:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:57 smithi029 ceph-mon[131166]: pgmap v513: 129 pgs: 129 active+clean; 302 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 423 op/s 2023-11-27T15:30:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.785+0000 7f47f6ff5700 1 -- 172.21.15.8:0/2961659560 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d8d10dc600 con 0x55d8d122e280 2023-11-27T15:30:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.785+0000 7f47f6ff5700 10 client.25771 handle_command_reply: tid=0 2023-11-27T15:30:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.785+0000 7f482cd15b80 1 client.25771 shutdown 2023-11-27T15:30:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.786+0000 7f482cd15b80 1 -- 172.21.15.8:0/2961659560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 msgr2=0x55d8d11d47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.786+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 0x55d8d11d47c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f480c0a2d00 tx=0x7f480c009920 comp rx=0 tx=0).stop 2023-11-27T15:30:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.786+0000 7f482cd15b80 1 -- 172.21.15.8:0/2961659560 shutdown_connections 2023-11-27T15:30:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.786+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55d8d122e280 0x55d8d1230690 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.786+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55d8d11bdb10 0x55d8d11d47c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.786+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55d8d10186b0 0x55d8d11d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.786+0000 7f482cd15b80 1 --2- 172.21.15.8:0/2961659560 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55d8d11bcdb0 0x55d8d11d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.786+0000 7f482cd15b80 1 -- 172.21.15.8:0/2961659560 >> 172.21.15.8:0/2961659560 conn(0x55d8d11b1f50 msgr2=0x55d8d11c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:58.790 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.786+0000 7f47f6ff5700 0 client.25771 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:30:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.787+0000 7f482cd15b80 1 -- 172.21.15.8:0/2961659560 shutdown_connections 2023-11-27T15:30:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.787+0000 7f482cd15b80 1 -- 172.21.15.8:0/2961659560 wait complete. 2023-11-27T15:30:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.787+0000 7f482cd15b80 20 client.25771 trim_cache size 0 max 16384 2023-11-27T15:30:58.791 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:30:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:30:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:30:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:30:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.789+0000 7f48193ff700 1 -- 172.21.15.8:0/674500504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47fc0a1220 msgr2=0x7f47fc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.789+0000 7f48193ff700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47fc0a1220 0x7f47fc0a36d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4804009cb0 tx=0x7f4804011720 comp rx=0 tx=0).stop 2023-11-27T15:30:58.792 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.789+0000 7f48193ff700 1 -- 172.21.15.8:0/674500504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4814101e70 msgr2=0x7f481419f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:30:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.789+0000 7f48193ff700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4814101e70 0x7f481419f8b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f480c009f90 tx=0x7f480c004c60 comp rx=0 tx=0).stop 2023-11-27T15:30:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.789+0000 7f48193ff700 1 -- 172.21.15.8:0/674500504 shutdown_connections 2023-11-27T15:30:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.789+0000 7f48193ff700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f47fc0a1220 0x7f47fc0a36d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.789+0000 7f48193ff700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f4814100230 0x7f4814198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.789+0000 7f48193ff700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f4814101e70 0x7f481419f8b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:58.793 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.789+0000 7f48193ff700 1 --2- 172.21.15.8:0/674500504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48141014c0 0x7f4814198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:30:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.789+0000 7f48193ff700 1 -- 172.21.15.8:0/674500504 >> 172.21.15.8:0/674500504 conn(0x7f48140fb810 msgr2=0x7f48140fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:30:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.790+0000 7f48193ff700 1 -- 172.21.15.8:0/674500504 shutdown_connections 2023-11-27T15:30:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:30:58.790+0000 7f48193ff700 1 -- 172.21.15.8:0/674500504 wait complete. 2023-11-27T15:30:59.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:30:59 smithi029 ceph-mon[131166]: pgmap v514: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 412 op/s 2023-11-27T15:30:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:30:59 smithi060 ceph-mon[131518]: pgmap v514: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 412 op/s 2023-11-27T15:30:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:30:59 smithi008 ceph-mon[128788]: pgmap v514: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 412 op/s 2023-11-27T15:31:00.398 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:00 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-e[155797]: *** Caught signal (Segmentation fault) ** 2023-11-27T15:31:00.398 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:00 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-e[155797]: in thread 7fe339d83700 thread_name:MR_Finisher 2023-11-27T15:31:01.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:01 smithi029 ceph-mon[131166]: pgmap v515: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 376 op/s 2023-11-27T15:31:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:01 smithi060 ceph-mon[131518]: pgmap v515: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 376 op/s 2023-11-27T15:31:01.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:01 smithi008 ceph-mon[128788]: pgmap v515: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 376 op/s 2023-11-27T15:31:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:03 smithi060 ceph-mon[131518]: pgmap v516: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 453 op/s 2023-11-27T15:31:03.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:03 smithi008 ceph-mon[128788]: pgmap v516: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 453 op/s 2023-11-27T15:31:03.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:03 smithi029 ceph-mon[131166]: pgmap v516: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 453 op/s 2023-11-27T15:31:04.300 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:03 smithi029 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.e.service: Main process exited, code=exited, status=139/n/a 2023-11-27T15:31:04.647 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:04 smithi029 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.e.service: Failed with result 'exit-code'. 2023-11-27T15:31:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:05 smithi060 ceph-mon[131518]: pgmap v517: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 400 op/s 2023-11-27T15:31:05.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:05 smithi008 ceph-mon[128788]: pgmap v517: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 400 op/s 2023-11-27T15:31:05.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:05 smithi029 ceph-mon[131166]: pgmap v517: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 400 op/s 2023-11-27T15:31:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:07 smithi060 ceph-mon[131518]: pgmap v518: 129 pgs: 129 active+clean; 310 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 398 op/s 2023-11-27T15:31:07.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:07 smithi008 ceph-mon[128788]: pgmap v518: 129 pgs: 129 active+clean; 310 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 398 op/s 2023-11-27T15:31:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:07 smithi029 ceph-mon[131166]: pgmap v518: 129 pgs: 129 active+clean; 310 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 398 op/s 2023-11-27T15:31:09.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:09 smithi029 ceph-mon[131166]: pgmap v519: 129 pgs: 129 active+clean; 315 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 439 op/s 2023-11-27T15:31:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:09 smithi060 ceph-mon[131518]: pgmap v519: 129 pgs: 129 active+clean; 315 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 439 op/s 2023-11-27T15:31:09.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:09 smithi008 ceph-mon[128788]: pgmap v519: 129 pgs: 129 active+clean; 315 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 439 op/s 2023-11-27T15:31:10.647 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:10 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:31:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:10 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:31:10.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:10 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:31:11.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:11 smithi060 ceph-mon[131518]: pgmap v520: 129 pgs: 129 active+clean; 319 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 396 op/s 2023-11-27T15:31:11.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:31:11.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:31:11.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:31:11.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:11 smithi008 ceph-mon[128788]: pgmap v520: 129 pgs: 129 active+clean; 319 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 396 op/s 2023-11-27T15:31:11.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:31:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:31:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:31:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:11 smithi029 ceph-mon[131166]: pgmap v520: 129 pgs: 129 active+clean; 319 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 396 op/s 2023-11-27T15:31:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:31:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:31:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:31:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.043+0000 7f917a9ff700 1 -- 172.21.15.8:0/1801333897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174109da0 msgr2=0x7f917410c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.043+0000 7f917a9ff700 1 --2- 172.21.15.8:0/1801333897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174109da0 0x7f917410c220 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f916c00b0a0 tx=0x7f916c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.045+0000 7f917a9ff700 1 -- 172.21.15.8:0/1801333897 shutdown_connections 2023-11-27T15:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.045+0000 7f917a9ff700 1 --2- 172.21.15.8:0/1801333897 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f917410c760 0x7f917410ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.045+0000 7f917a9ff700 1 --2- 172.21.15.8:0/1801333897 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174071630 0x7f9174071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.045+0000 7f917a9ff700 1 --2- 172.21.15.8:0/1801333897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174109da0 0x7f917410c220 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.045+0000 7f917a9ff700 1 -- 172.21.15.8:0/1801333897 >> 172.21.15.8:0/1801333897 conn(0x7f917406d1a0 msgr2=0x7f917406f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.046+0000 7f917a9ff700 1 -- 172.21.15.8:0/1801333897 shutdown_connections 2023-11-27T15:31:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.047+0000 7f917a9ff700 1 -- 172.21.15.8:0/1801333897 wait complete. 2023-11-27T15:31:12.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.049+0000 7f917a9ff700 1 Processor -- start 2023-11-27T15:31:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.049+0000 7f917a9ff700 1 -- start start 2023-11-27T15:31:12.053 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.050+0000 7f917a9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9174071630 0x7f91741a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.050+0000 7f917a9ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174109da0 0x7f91741a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.050+0000 7f917a9ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f917410c760 0x7f91741ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.050+0000 7f917a9ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91741ac970 con 0x7f917410c760 2023-11-27T15:31:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.050+0000 7f917a9ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91741acab0 con 0x7f9174109da0 2023-11-27T15:31:12.054 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.050+0000 7f917a9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91741acc40 con 0x7f9174071630 2023-11-27T15:31:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.050+0000 7f9173fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9174071630 0x7f91741a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.051+0000 7f9173fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9174071630 0x7f91741a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:38632/0 (socket says 172.21.15.8:38632) 2023-11-27T15:31:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.051+0000 7f9173fff700 1 -- 172.21.15.8:0/669716942 learned_addr learned my addr 172.21.15.8:0/669716942 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.051+0000 7f91737fe700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174109da0 0x7f91741a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.052+0000 7f9173fff700 1 -- 172.21.15.8:0/669716942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174109da0 msgr2=0x7f91741a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.052+0000 7f9173fff700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174109da0 0x7f91741a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.052+0000 7f9173fff700 1 -- 172.21.15.8:0/669716942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f917410c760 msgr2=0x7f91741ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.052+0000 7f9173fff700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f917410c760 0x7f91741ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.056 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.052+0000 7f9173fff700 1 -- 172.21.15.8:0/669716942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c009d00 con 0x7f9174071630 2023-11-27T15:31:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.056+0000 7f9173fff700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9174071630 0x7f91741a4df0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f916800c920 tx=0x7f916800cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:12.060 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.057+0000 7f91717fa700 1 -- 172.21.15.8:0/669716942 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168007b00 con 0x7f9174071630 2023-11-27T15:31:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.057+0000 7f91717fa700 1 -- 172.21.15.8:0/669716942 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9168004cf0 con 0x7f9174071630 2023-11-27T15:31:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.057+0000 7f91717fa700 1 -- 172.21.15.8:0/669716942 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168005610 con 0x7f9174071630 2023-11-27T15:31:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.057+0000 7f917a9ff700 1 -- 172.21.15.8:0/669716942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917411aa30 con 0x7f9174071630 2023-11-27T15:31:12.061 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.057+0000 7f917a9ff700 1 -- 172.21.15.8:0/669716942 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917411a8a0 con 0x7f9174071630 2023-11-27T15:31:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.065+0000 7f91717fa700 1 -- 172.21.15.8:0/669716942 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9168004750 con 0x7f9174071630 2023-11-27T15:31:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.067+0000 7f91717fa700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f915c0a12a0 0x7f915c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.068+0000 7f91717fa700 1 -- 172.21.15.8:0/669716942 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f91680ccbb0 con 0x7f9174071630 2023-11-27T15:31:12.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.068+0000 7f91737fe700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f915c0a12a0 0x7f915c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.069+0000 7f918e332b80 1 Processor -- start 2023-11-27T15:31:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.069+0000 7f918e332b80 1 -- start start 2023-11-27T15:31:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.070+0000 7f918e332b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c5e76b0 0x55ef3c78b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.070+0000 7f918e332b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c78c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.070+0000 7f918e332b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78cb10 0x55ef3c78cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.070+0000 7f918e332b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c691000 con 0x55ef3c5e76b0 2023-11-27T15:31:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.070+0000 7f918e332b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c6e3840 con 0x55ef3c78cb10 2023-11-27T15:31:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.070+0000 7f918e332b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c6dccd0 con 0x55ef3c78bdb0 2023-11-27T15:31:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.071+0000 7f9178f9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78cb10 0x55ef3c78cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.071+0000 7f9173fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c5e76b0 0x55ef3c78b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.071+0000 7f91737fe700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f915c0a12a0 0x7f915c0a3750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f916c000f80 tx=0x7f916c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.071+0000 7f91737fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c78c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.071+0000 7f9173fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c5e76b0 0x55ef3c78b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45570/0 (socket says 172.21.15.8:45570) 2023-11-27T15:31:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.071+0000 7f9173fff700 1 -- 172.21.15.8:0/1443995478 learned_addr learned my addr 172.21.15.8:0/1443995478 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:12.075 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.071+0000 7f91737fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c78c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:38634/0 (socket says 172.21.15.8:38634) 2023-11-27T15:31:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.071+0000 7f9173fff700 1 -- 172.21.15.8:0/1443995478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 msgr2=0x55ef3c78c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.071+0000 7f9173fff700 1 --2- 172.21.15.8:0/1443995478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c78c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.072+0000 7f9173fff700 1 -- 172.21.15.8:0/1443995478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78cb10 msgr2=0x55ef3c78cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.072+0000 7f9173fff700 1 --2- 172.21.15.8:0/1443995478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78cb10 0x55ef3c78cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.072+0000 7f9173fff700 1 -- 172.21.15.8:0/1443995478 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef3c55e5d0 con 0x55ef3c5e76b0 2023-11-27T15:31:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.072+0000 7f9173fff700 1 --2- 172.21.15.8:0/1443995478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c5e76b0 0x55ef3c78b870 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f91680a1b50 tx=0x7f9168099590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06a07d360aa91d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.072+0000 7f915a7fc700 1 -- 172.21.15.8:0/1443995478 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916809d030 con 0x55ef3c5e76b0 2023-11-27T15:31:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.073+0000 7f915a7fc700 1 -- 172.21.15.8:0/1443995478 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f916809dcc0 con 0x55ef3c5e76b0 2023-11-27T15:31:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.073+0000 7f915a7fc700 1 -- 172.21.15.8:0/1443995478 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168096340 con 0x55ef3c5e76b0 2023-11-27T15:31:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.074+0000 7f918e332b80 1 -- 172.21.15.8:0/1443995478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c5e76b0 msgr2=0x55ef3c78b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.074+0000 7f918e332b80 1 --2- 172.21.15.8:0/1443995478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c5e76b0 0x55ef3c78b870 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f91680a1b50 tx=0x7f9168099590 comp rx=0 tx=0).stop 2023-11-27T15:31:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.074+0000 7f918e332b80 1 -- 172.21.15.8:0/1443995478 shutdown_connections 2023-11-27T15:31:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.074+0000 7f918e332b80 1 --2- 172.21.15.8:0/1443995478 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78cb10 0x55ef3c78cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.074+0000 7f918e332b80 1 --2- 172.21.15.8:0/1443995478 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c5e76b0 0x55ef3c78b870 secure :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f91680a1b50 tx=0x7f9168099590 comp rx=0 tx=0).stop 2023-11-27T15:31:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.074+0000 7f918e332b80 1 --2- 172.21.15.8:0/1443995478 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c78c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.074+0000 7f918e332b80 1 -- 172.21.15.8:0/1443995478 >> 172.21.15.8:0/1443995478 conn(0x55ef3c780f50 msgr2=0x55ef3c786100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.075+0000 7f918e332b80 1 -- 172.21.15.8:0/1443995478 shutdown_connections 2023-11-27T15:31:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.075+0000 7f918e332b80 1 -- 172.21.15.8:0/1443995478 wait complete. 2023-11-27T15:31:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.076+0000 7f918e332b80 1 -- start start 2023-11-27T15:31:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.076+0000 7f918e332b80 1 Processor -- start 2023-11-27T15:31:12.080 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.077+0000 7f918e332b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78bdb0 0x55ef3c794f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.077+0000 7f918e332b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78cb10 0x55ef3c7954c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.077+0000 7f918e332b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c79a4f0 0x55ef3c79c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.077+0000 7f918e332b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c67e880 con 0x55ef3c79a4f0 2023-11-27T15:31:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.077+0000 7f918e332b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c67e520 con 0x55ef3c78bdb0 2023-11-27T15:31:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.077+0000 7f918e332b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c5608c0 con 0x55ef3c78cb10 2023-11-27T15:31:12.081 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.077+0000 7f918e332b80 10 client.0 _ll_register_callbacks cb 0x55ef3c69c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:31:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.078+0000 7f91737fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78cb10 0x55ef3c7954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.078+0000 7f91737fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78cb10 0x55ef3c7954c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:38648/0 (socket says 172.21.15.8:38648) 2023-11-27T15:31:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.078+0000 7f91737fe700 1 -- 172.21.15.8:0/2438564426 learned_addr learned my addr 172.21.15.8:0/2438564426 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:12.082 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.078+0000 7f9173fff700 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78bdb0 0x55ef3c794f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.078+0000 7f9178f9c700 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c79a4f0 0x55ef3c79c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.079+0000 7f9173fff700 1 -- 172.21.15.8:0/2438564426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78cb10 msgr2=0x55ef3c7954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.079+0000 7f9173fff700 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78cb10 0x55ef3c7954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.079+0000 7f9173fff700 1 -- 172.21.15.8:0/2438564426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c79a4f0 msgr2=0x55ef3c79c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.079+0000 7f9173fff700 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c79a4f0 0x55ef3c79c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.079+0000 7f9173fff700 1 -- 172.21.15.8:0/2438564426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917411a8a0 con 0x55ef3c78bdb0 2023-11-27T15:31:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.080+0000 7f9173fff700 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78bdb0 0x55ef3c794f80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55ef3c787570 tx=0x7f916800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b59050795541b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.080+0000 7f91537fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:31:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.080+0000 7f91537fe700 1 -- 172.21.15.8:0/2438564426 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916809d030 con 0x55ef3c78bdb0 2023-11-27T15:31:12.084 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.080+0000 7f91537fe700 1 -- 172.21.15.8:0/2438564426 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9168004a00 con 0x55ef3c78bdb0 2023-11-27T15:31:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.081+0000 7f918e332b80 1 -- 172.21.15.8:0/2438564426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef3c77e710 con 0x55ef3c78bdb0 2023-11-27T15:31:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.081+0000 7f91537fe700 1 -- 172.21.15.8:0/2438564426 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91680b2290 con 0x55ef3c78bdb0 2023-11-27T15:31:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.082+0000 7f918e332b80 10 client.25757 fetch_fsmap learned FSMap version 36 2023-11-27T15:31:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.082+0000 7f918e332b80 1 -- 172.21.15.8:0/2438564426 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ef3c6a3130 con 0x55ef3c78bdb0 2023-11-27T15:31:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.084+0000 7f91537fe700 1 -- 172.21.15.8:0/2438564426 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef3c77e710 con 0x55ef3c78bdb0 2023-11-27T15:31:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.084+0000 7f91537fe700 1 -- 172.21.15.8:0/2438564426 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 36) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f91680a1bf0 con 0x55ef3c78bdb0 2023-11-27T15:31:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.084+0000 7f918e332b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 36 2023-11-27T15:31:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.084+0000 7f918e332b80 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:31:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.084+0000 7f918e332b80 20 client.25757 populate_metadata read hostname 'smithi008' 2023-11-27T15:31:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.084+0000 7f918e332b80 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55ef3c7b1bd0 0x55ef3c7b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.084+0000 7f918e332b80 4 client.25757 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:31:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.084+0000 7f918e332b80 1 -- 172.21.15.8:0/2438564426 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef3c69f040 con 0x55ef3c7b1bd0 2023-11-27T15:31:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.085+0000 7f91737fe700 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55ef3c7b1bd0 0x55ef3c7b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.088+0000 7f91737fe700 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55ef3c7b1bd0 0x55ef3c7b3fe0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40e89d05cbbfb98 server_cookie=2154d6c08c6b3aa2 in_seq=0 out_seq=0 2023-11-27T15:31:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.088+0000 7f91537fe700 10 client.25757 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.092+0000 7f91537fe700 1 -- 172.21.15.8:0/2438564426 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ef3c69f040 con 0x55ef3c7b1bd0 2023-11-27T15:31:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.092+0000 7f91537fe700 10 client.25757 handle_command_reply: tid=0 2023-11-27T15:31:12.096 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.093+0000 7f918e332b80 1 client.25757 shutdown 2023-11-27T15:31:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.094+0000 7f918e332b80 1 -- 172.21.15.8:0/2438564426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78bdb0 msgr2=0x55ef3c794f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.094+0000 7f918e332b80 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78bdb0 0x55ef3c794f80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55ef3c787570 tx=0x7f916800a620 comp rx=0 tx=0).stop 2023-11-27T15:31:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.094+0000 7f918e332b80 1 -- 172.21.15.8:0/2438564426 shutdown_connections 2023-11-27T15:31:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.094+0000 7f918e332b80 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c79a4f0 0x55ef3c79c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.094+0000 7f918e332b80 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c78bdb0 0x55ef3c794f80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.094+0000 7f918e332b80 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55ef3c7b1bd0 0x55ef3c7b3fe0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.094+0000 7f918e332b80 1 --2- 172.21.15.8:0/2438564426 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78cb10 0x55ef3c7954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.094+0000 7f918e332b80 1 -- 172.21.15.8:0/2438564426 >> 172.21.15.8:0/2438564426 conn(0x55ef3c780f50 msgr2=0x55ef3c7857e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.095+0000 7f91537fe700 0 client.25757 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.095+0000 7f918e332b80 1 -- 172.21.15.8:0/2438564426 shutdown_connections 2023-11-27T15:31:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.095+0000 7f918e332b80 1 -- 172.21.15.8:0/2438564426 wait complete. 2023-11-27T15:31:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.097+0000 7f918e332b80 20 client.25757 trim_cache size 0 max 16384 2023-11-27T15:31:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.130+0000 7f918e332b80 1 Processor -- start 2023-11-27T15:31:12.134 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.131+0000 7f918e332b80 1 -- start start 2023-11-27T15:31:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.132+0000 7f918e332b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.132+0000 7f918e332b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c78cb10 0x55ef3c7c1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.132+0000 7f918e332b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 0x55ef3c78fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.133+0000 7f918e332b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c7875a0 con 0x55ef3c78cb10 2023-11-27T15:31:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.133+0000 7f918e332b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c794280 con 0x55ef3c79a4f0 2023-11-27T15:31:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.133+0000 7f918e332b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c794430 con 0x55ef3c78bdb0 2023-11-27T15:31:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.133+0000 7f9173fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.133+0000 7f9173fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7931d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:38664/0 (socket says 172.21.15.8:38664) 2023-11-27T15:31:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.133+0000 7f9173fff700 1 -- 172.21.15.8:0/2781010091 learned_addr learned my addr 172.21.15.8:0/2781010091 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.133+0000 7f9178f9c700 1 --2- 172.21.15.8:0/2781010091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 0x55ef3c78fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.134+0000 7f9173fff700 1 -- 172.21.15.8:0/2781010091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 msgr2=0x55ef3c78fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.134+0000 7f9173fff700 1 --2- 172.21.15.8:0/2781010091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 0x55ef3c78fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.134+0000 7f9173fff700 1 -- 172.21.15.8:0/2781010091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c78cb10 msgr2=0x55ef3c7c1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.134+0000 7f9173fff700 1 --2- 172.21.15.8:0/2781010091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c78cb10 0x55ef3c7c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.134+0000 7f9173fff700 1 -- 172.21.15.8:0/2781010091 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef3c6a2e80 con 0x55ef3c78bdb0 2023-11-27T15:31:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.135+0000 7f9173fff700 1 --2- 172.21.15.8:0/2781010091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7931d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9168000c00 tx=0x7f9168099590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4817a8ebf248e8c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.136+0000 7f915affd700 1 -- 172.21.15.8:0/2781010091 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916809d040 con 0x55ef3c78bdb0 2023-11-27T15:31:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.136+0000 7f915affd700 1 -- 172.21.15.8:0/2781010091 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9168099340 con 0x55ef3c78bdb0 2023-11-27T15:31:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.136+0000 7f915affd700 1 -- 172.21.15.8:0/2781010091 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91680b2290 con 0x55ef3c78bdb0 2023-11-27T15:31:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.137+0000 7f918e332b80 1 -- 172.21.15.8:0/2781010091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 msgr2=0x55ef3c7931d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.137+0000 7f918e332b80 1 --2- 172.21.15.8:0/2781010091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7931d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9168000c00 tx=0x7f9168099590 comp rx=0 tx=0).stop 2023-11-27T15:31:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.138+0000 7f918e332b80 1 -- 172.21.15.8:0/2781010091 shutdown_connections 2023-11-27T15:31:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.138+0000 7f918e332b80 1 --2- 172.21.15.8:0/2781010091 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7931d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.138+0000 7f918e332b80 1 --2- 172.21.15.8:0/2781010091 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 0x55ef3c78fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.138+0000 7f918e332b80 1 --2- 172.21.15.8:0/2781010091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c78cb10 0x55ef3c7c1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.138+0000 7f918e332b80 1 -- 172.21.15.8:0/2781010091 >> 172.21.15.8:0/2781010091 conn(0x55ef3c780f50 msgr2=0x55ef3c7b8940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.139+0000 7f918e332b80 1 -- 172.21.15.8:0/2781010091 shutdown_connections 2023-11-27T15:31:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.140+0000 7f918e332b80 1 -- 172.21.15.8:0/2781010091 wait complete. 2023-11-27T15:31:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.141+0000 7f918e332b80 1 -- start start 2023-11-27T15:31:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.142+0000 7f918e332b80 1 Processor -- start 2023-11-27T15:31:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.143+0000 7f918e332b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7c0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.143+0000 7f918e332b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c78cb10 0x55ef3c7c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.144+0000 7f9173fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.144+0000 7f9173fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7c0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:38678/0 (socket says 172.21.15.8:38678) 2023-11-27T15:31:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.145+0000 7f918e332b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 0x55ef3c804200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.145+0000 7f91737fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c78cb10 0x55ef3c7c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.145+0000 7f9173fff700 1 -- 172.21.15.8:0/1177801206 learned_addr learned my addr 172.21.15.8:0/1177801206 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.145+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c7c1590 con 0x55ef3c78cb10 2023-11-27T15:31:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.145+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c804740 con 0x55ef3c79a4f0 2023-11-27T15:31:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.145+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef3c804900 con 0x55ef3c78bdb0 2023-11-27T15:31:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.145+0000 7f918e332b80 10 client.0 _ll_register_callbacks cb 0x55ef3c69c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:31:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.146+0000 7f9178f9c700 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 0x55ef3c804200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.147+0000 7f9178f9c700 1 -- 172.21.15.8:0/1177801206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 msgr2=0x55ef3c7c0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.147+0000 7f9178f9c700 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7c0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.147+0000 7f9178f9c700 1 -- 172.21.15.8:0/1177801206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c78cb10 msgr2=0x55ef3c7c0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.147+0000 7f9178f9c700 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c78cb10 0x55ef3c7c0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.147+0000 7f9178f9c700 1 -- 172.21.15.8:0/1177801206 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef3c6a3130 con 0x55ef3c79a4f0 2023-11-27T15:31:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.150+0000 7f9178f9c700 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 0x55ef3c804200 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55ef3c796030 tx=0x7f91640119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e7639c08dbe4c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.150+0000 7f91537fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:31:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.150+0000 7f91537fe700 1 -- 172.21.15.8:0/1177801206 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164016070 con 0x55ef3c79a4f0 2023-11-27T15:31:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.150+0000 7f91537fe700 1 -- 172.21.15.8:0/1177801206 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f916400f420 con 0x55ef3c79a4f0 2023-11-27T15:31:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.151+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f916809dcc0 con 0x55ef3c79a4f0 2023-11-27T15:31:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.151+0000 7f91537fe700 1 -- 172.21.15.8:0/1177801206 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91640112e0 con 0x55ef3c79a4f0 2023-11-27T15:31:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.151+0000 7f918e332b80 10 client.25763 fetch_fsmap learned FSMap version 36 2023-11-27T15:31:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.151+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55ef3c4f3c00 con 0x55ef3c79a4f0 2023-11-27T15:31:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.152+0000 7f91537fe700 1 -- 172.21.15.8:0/1177801206 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f916809dcc0 con 0x55ef3c79a4f0 2023-11-27T15:31:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.152+0000 7f91537fe700 1 -- 172.21.15.8:0/1177801206 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 36) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f91640258a0 con 0x55ef3c79a4f0 2023-11-27T15:31:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.152+0000 7f918e332b80 10 client.25763 fetch_fsmap finished waiting for FSMap version 36 2023-11-27T15:31:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.152+0000 7f918e332b80 10 client.25763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:31:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.152+0000 7f918e332b80 20 client.25763 populate_metadata read hostname 'smithi008' 2023-11-27T15:31:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.152+0000 7f918e332b80 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55ef3c8063e0 0x55ef3c8087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.152+0000 7f918e332b80 4 client.25763 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:31:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.152+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ef3c6ab600 con 0x55ef3c8063e0 2023-11-27T15:31:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.155+0000 7f9173fff700 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55ef3c8063e0 0x55ef3c8087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.158+0000 7f91537fe700 10 client.25763 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.158+0000 7f9173fff700 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55ef3c8063e0 0x55ef3c8087f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e40df96712c0c48 server_cookie=e564e515f06837cc in_seq=0 out_seq=0 2023-11-27T15:31:12.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:12 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:31:12.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:12 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:31:12.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:12 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:31:12.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:12 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:31:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:12 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:31:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:12 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:31:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:12 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:31:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:12 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:31:12.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:12 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:31:12.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:12 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:31:12.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:12 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:31:12.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:12 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:31:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.922+0000 7f91537fe700 1 -- 172.21.15.8:0/1177801206 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ef3c6ab600 con 0x55ef3c8063e0 2023-11-27T15:31:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.922+0000 7f91537fe700 10 client.25763 handle_command_reply: tid=0 2023-11-27T15:31:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.922+0000 7f918e332b80 1 client.25763 shutdown 2023-11-27T15:31:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.924+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 msgr2=0x55ef3c804200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.924+0000 7f918e332b80 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 0x55ef3c804200 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55ef3c796030 tx=0x7f91640119a0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.925+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 shutdown_connections 2023-11-27T15:31:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.925+0000 7f918e332b80 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55ef3c8063e0 0x55ef3c8087f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.926+0000 7f918e332b80 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55ef3c78bdb0 0x55ef3c7c0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.926+0000 7f918e332b80 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55ef3c79a4f0 0x55ef3c804200 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.926+0000 7f918e332b80 1 --2- 172.21.15.8:0/1177801206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55ef3c78cb10 0x55ef3c7c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.926+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 >> 172.21.15.8:0/1177801206 conn(0x55ef3c780f50 msgr2=0x55ef3c7931d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.926+0000 7f91537fe700 0 client.25763 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.927+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 shutdown_connections 2023-11-27T15:31:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.927+0000 7f918e332b80 1 -- 172.21.15.8:0/1177801206 wait complete. 2023-11-27T15:31:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.928+0000 7f918e332b80 20 client.25763 trim_cache size 0 max 16384 2023-11-27T15:31:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:31:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:31:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:31:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:31:12.933 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 -- 172.21.15.8:0/669716942 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f915c0a12a0 msgr2=0x7f915c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f915c0a12a0 0x7f915c0a3750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f916c000f80 tx=0x7f916c00b7b0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 -- 172.21.15.8:0/669716942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9174071630 msgr2=0x7f91741a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9174071630 0x7f91741a4df0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f916800c920 tx=0x7f916800cce0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 -- 172.21.15.8:0/669716942 shutdown_connections 2023-11-27T15:31:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f915c0a12a0 0x7f915c0a3750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9174071630 0x7f91741a4df0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f917410c760 0x7f91741ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 --2- 172.21.15.8:0/669716942 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9174109da0 0x7f91741a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 -- 172.21.15.8:0/669716942 >> 172.21.15.8:0/669716942 conn(0x7f917406d1a0 msgr2=0x7f917410b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.930+0000 7f917a9ff700 1 -- 172.21.15.8:0/669716942 shutdown_connections 2023-11-27T15:31:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:12.931+0000 7f917a9ff700 1 -- 172.21.15.8:0/669716942 wait complete. 2023-11-27T15:31:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:13 smithi060 ceph-mon[131518]: pgmap v521: 129 pgs: 129 active+clean; 322 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 399 op/s 2023-11-27T15:31:13.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:13 smithi008 ceph-mon[128788]: pgmap v521: 129 pgs: 129 active+clean; 322 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 399 op/s 2023-11-27T15:31:13.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:13 smithi029 ceph-mon[131166]: pgmap v521: 129 pgs: 129 active+clean; 322 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 399 op/s 2023-11-27T15:31:14.897 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:14 smithi029 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.e.service: Service RestartSec=10s expired, scheduling restart. 2023-11-27T15:31:14.897 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:14 smithi029 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.e.service: Scheduled restart job, restart counter is at 1. 2023-11-27T15:31:14.897 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:14 smithi029 systemd[1]: Stopped Ceph mds.e for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:31:14.898 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:14 smithi029 systemd[1]: Starting Ceph mds.e for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:15 smithi060 ceph-mon[131518]: pgmap v522: 129 pgs: 129 active+clean; 327 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 356 op/s 2023-11-27T15:31:15.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:15 smithi008 ceph-mon[128788]: pgmap v522: 129 pgs: 129 active+clean; 327 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 356 op/s 2023-11-27T15:31:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:15 smithi029 ceph-mon[131166]: pgmap v522: 129 pgs: 129 active+clean; 327 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 356 op/s 2023-11-27T15:31:16.897 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:16 smithi029 podman[159293]: 2023-11-27T15:31:17.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:17 smithi060 ceph-mon[131518]: pgmap v523: 129 pgs: 129 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 398 op/s 2023-11-27T15:31:17.850 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:17 smithi029 ceph-mon[131166]: pgmap v523: 129 pgs: 129 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 398 op/s 2023-11-27T15:31:17.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:17 smithi008 ceph-mon[128788]: pgmap v523: 129 pgs: 129 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 398 op/s 2023-11-27T15:31:18.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:18 smithi060 ceph-mon[131518]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2023-11-27T15:31:18.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:18 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 5 up:standby 2023-11-27T15:31:18.831 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:18 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-e[159308]: starting mds.e at 2023-11-27T15:31:18.832 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:18 smithi029 ceph-mon[131166]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2023-11-27T15:31:18.832 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:18 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 5 up:standby 2023-11-27T15:31:18.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:18 smithi008 ceph-mon[128788]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2023-11-27T15:31:18.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:18 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 5 up:standby 2023-11-27T15:31:19.397 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:19 smithi029 bash[159293]: 4a3a35d9cfe651482dfcb0ef9743a8b27470e0fab757868eaf55e21267aa628d 2023-11-27T15:31:19.397 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:31:19 smithi029 systemd[1]: Started Ceph mds.e for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:31:19.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:19 smithi060 ceph-mon[131518]: pgmap v524: 129 pgs: 129 active+clean; 330 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 448 op/s 2023-11-27T15:31:19.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:19 smithi008 ceph-mon[128788]: pgmap v524: 129 pgs: 129 active+clean; 330 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 448 op/s 2023-11-27T15:31:19.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:19 smithi029 ceph-mon[131166]: pgmap v524: 129 pgs: 129 active+clean; 330 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 448 op/s 2023-11-27T15:31:20.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:20 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.29:6834/1212732321,v1:172.21.15.29:6835/1212732321] up:boot 2023-11-27T15:31:20.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:20 smithi060 ceph-mon[131518]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:31:20.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:20 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:31:20.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:20 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.29:6834/1212732321,v1:172.21.15.29:6835/1212732321] up:boot 2023-11-27T15:31:20.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:20 smithi008 ceph-mon[128788]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:31:20.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:20 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:31:20.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:20 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.29:6834/1212732321,v1:172.21.15.29:6835/1212732321] up:boot 2023-11-27T15:31:20.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:20 smithi029 ceph-mon[131166]: fsmap cephfs:3 {0=a=up:active,1=b=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:31:20.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:20 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:31:21.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:21 smithi060 ceph-mon[131518]: pgmap v525: 129 pgs: 129 active+clean; 331 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 388 op/s 2023-11-27T15:31:21.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:21 smithi008 ceph-mon[128788]: pgmap v525: 129 pgs: 129 active+clean; 331 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 388 op/s 2023-11-27T15:31:21.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:21 smithi029 ceph-mon[131166]: pgmap v525: 129 pgs: 129 active+clean; 331 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 388 op/s 2023-11-27T15:31:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:23 smithi060 ceph-mon[131518]: pgmap v526: 129 pgs: 129 active+clean; 335 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 441 op/s 2023-11-27T15:31:23.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:23 smithi008 ceph-mon[128788]: pgmap v526: 129 pgs: 129 active+clean; 335 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 441 op/s 2023-11-27T15:31:23.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:23 smithi029 ceph-mon[131166]: pgmap v526: 129 pgs: 129 active+clean; 335 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 441 op/s 2023-11-27T15:31:25.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:25 smithi060 ceph-mon[131518]: pgmap v527: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 439 op/s 2023-11-27T15:31:25.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:25 smithi008 ceph-mon[128788]: pgmap v527: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 439 op/s 2023-11-27T15:31:25.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:25 smithi029 ceph-mon[131166]: pgmap v527: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 439 op/s 2023-11-27T15:31:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.188+0000 7f3fe55ff700 1 -- 172.21.15.8:0/1863362280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe0100250 msgr2=0x7f3fe0100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.188+0000 7f3fe55ff700 1 --2- 172.21.15.8:0/1863362280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe0100250 0x7f3fe0100660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000dd20 tx=0x7f3fd000f330 comp rx=0 tx=0).stop 2023-11-27T15:31:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.188+0000 7f3fe55ff700 1 -- 172.21.15.8:0/1863362280 shutdown_connections 2023-11-27T15:31:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.188+0000 7f3fe55ff700 1 --2- 172.21.15.8:0/1863362280 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe0100250 0x7f3fe0100660 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.188+0000 7f3fe55ff700 1 --2- 172.21.15.8:0/1863362280 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fe0101dd0 0x7f3fe0108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.188+0000 7f3fe55ff700 1 --2- 172.21.15.8:0/1863362280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe0101440 0x7f3fe0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.188+0000 7f3fe55ff700 1 -- 172.21.15.8:0/1863362280 >> 172.21.15.8:0/1863362280 conn(0x7f3fe00fb850 msgr2=0x7f3fe00fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.188+0000 7f3fe55ff700 1 -- 172.21.15.8:0/1863362280 shutdown_connections 2023-11-27T15:31:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.189+0000 7f3fe55ff700 1 -- 172.21.15.8:0/1863362280 wait complete. 2023-11-27T15:31:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fe55ff700 1 Processor -- start 2023-11-27T15:31:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fe55ff700 1 -- start start 2023-11-27T15:31:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fe55ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe0100250 0x7f3fe0193f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fe55ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fe0101440 0x7f3fe0194450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fe55ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe0101dd0 0x7f3fe019b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fe55ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe019ba90 con 0x7f3fe0100250 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fe55ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe019bbd0 con 0x7f3fe0101440 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fe55ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe019bd60 con 0x7f3fe0101dd0 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe0100250 0x7f3fe0193f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe0100250 0x7f3fe0193f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56704/0 (socket says 172.21.15.8:56704) 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.190+0000 7f3fdeffd700 1 -- 172.21.15.8:0/2507069281 learned_addr learned my addr 172.21.15.8:0/2507069281 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.191+0000 7f3fde7fc700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fe0101440 0x7f3fe0194450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.191+0000 7f3fdf7fe700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe0101dd0 0x7f3fe019b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.191+0000 7f3fdeffd700 1 -- 172.21.15.8:0/2507069281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe0101dd0 msgr2=0x7f3fe019b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.191+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe0101dd0 0x7f3fe019b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.191+0000 7f3fdeffd700 1 -- 172.21.15.8:0/2507069281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fe0101440 msgr2=0x7f3fe0194450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.191+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fe0101440 0x7f3fe0194450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.195 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.191+0000 7f3fdeffd700 1 -- 172.21.15.8:0/2507069281 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd000da00 con 0x7f3fe0100250 2023-11-27T15:31:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.191+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe0100250 0x7f3fe0193f10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00042c0 tx=0x7f3fd000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.192+0000 7f3fc7fff700 1 -- 172.21.15.8:0/2507069281 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0025070 con 0x7f3fe0100250 2023-11-27T15:31:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.192+0000 7f3fe55ff700 1 -- 172.21.15.8:0/2507069281 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe0109a10 con 0x7f3fe0100250 2023-11-27T15:31:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.192+0000 7f3fe55ff700 1 -- 172.21.15.8:0/2507069281 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe0109880 con 0x7f3fe0100250 2023-11-27T15:31:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.193+0000 7f3fc7fff700 1 -- 172.21.15.8:0/2507069281 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fd00054c0 con 0x7f3fe0100250 2023-11-27T15:31:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.193+0000 7f3fc7fff700 1 -- 172.21.15.8:0/2507069281 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0012040 con 0x7f3fe0100250 2023-11-27T15:31:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.197+0000 7f3fc7fff700 1 -- 172.21.15.8:0/2507069281 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3fd0003940 con 0x7f3fe0100250 2023-11-27T15:31:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.198+0000 7f3ff8f96b80 1 Processor -- start 2023-11-27T15:31:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.198+0000 7f3ff8f96b80 1 -- start start 2023-11-27T15:31:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.198+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a118080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.198+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a118de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.198+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 0x564e2a119790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.198+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a015000 con 0x564e2a119320 2023-11-27T15:31:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.198+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a067840 con 0x564e2a1185c0 2023-11-27T15:31:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.198+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a060cd0 con 0x564e29f6b6b0 2023-11-27T15:31:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a118080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a118080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:56188/0 (socket says 172.21.15.8:56188) 2023-11-27T15:31:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fdf7fe700 1 -- 172.21.15.8:0/2094690408 learned_addr learned my addr 172.21.15.8:0/2094690408 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a118de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fde7fc700 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 0x564e2a119790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fdeffd700 1 -- 172.21.15.8:0/2094690408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 msgr2=0x564e2a118080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a118080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fc7fff700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3fc80a11d0 0x7f3fc80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fdeffd700 1 -- 172.21.15.8:0/2094690408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 msgr2=0x564e2a119790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 0x564e2a119790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.199+0000 7f3fdeffd700 1 -- 172.21.15.8:0/2094690408 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e29ee25d0 con 0x564e2a1185c0 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.200+0000 7f3fdf7fe700 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a118080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.200+0000 7f3fde7fc700 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 0x564e2a119790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.200+0000 7f3fc7fff700 1 -- 172.21.15.8:0/2507069281 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3fd00d1270 con 0x7f3fe0100250 2023-11-27T15:31:26.204 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.200+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a118de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00ce9b0 tx=0x7f3fd0069820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa763e68481ffe1a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.200+0000 7f3fc57fa700 1 -- 172.21.15.8:0/2094690408 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd007d080 con 0x564e2a1185c0 2023-11-27T15:31:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.200+0000 7f3fc57fa700 1 -- 172.21.15.8:0/2094690408 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fd00890c0 con 0x564e2a1185c0 2023-11-27T15:31:26.205 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.200+0000 7f3fdf7fe700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3fc80a11d0 0x7f3fc80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.200+0000 7f3fc57fa700 1 -- 172.21.15.8:0/2094690408 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd009dce0 con 0x564e2a1185c0 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.201+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2094690408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 msgr2=0x564e2a118de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.201+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a118de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00ce9b0 tx=0x7f3fd0069820 comp rx=0 tx=0).stop 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.201+0000 7f3fdf7fe700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3fc80a11d0 0x7f3fc80a3680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009550 tx=0x7f3fd8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.201+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2094690408 shutdown_connections 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.201+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a118080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.201+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 0x564e2a119790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.201+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/2094690408 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a118de0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.201+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2094690408 >> 172.21.15.8:0/2094690408 conn(0x564e2a104f50 msgr2=0x564e2a10a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.201+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2094690408 shutdown_connections 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.201+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2094690408 wait complete. 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.202+0000 7f3ff8f96b80 1 -- start start 2023-11-27T15:31:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.202+0000 7f3ff8f96b80 1 Processor -- start 2023-11-27T15:31:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.203+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e29f6b6b0 0x564e2a1212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.203+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e2a1185c0 0x564e2a121800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.203+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a119320 0x564e2a128840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.203+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a002880 con 0x564e29f6b6b0 2023-11-27T15:31:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.203+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a002520 con 0x564e2a119320 2023-11-27T15:31:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.203+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e29ee48c0 con 0x564e2a1185c0 2023-11-27T15:31:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.203+0000 7f3ff8f96b80 10 client.0 _ll_register_callbacks cb 0x564e2a020720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:31:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a119320 0x564e2a128840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fdf7fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a119320 0x564e2a128840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45424/0 (socket says 172.21.15.8:45424) 2023-11-27T15:31:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fdf7fe700 1 -- 172.21.15.8:0/2891475822 learned_addr learned my addr 172.21.15.8:0/2891475822 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fde7fc700 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e2a1185c0 0x564e2a121800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fdf7fe700 1 -- 172.21.15.8:0/2891475822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e2a1185c0 msgr2=0x564e2a121800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fdf7fe700 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e2a1185c0 0x564e2a121800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.209 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fdf7fe700 1 -- 172.21.15.8:0/2891475822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e29f6b6b0 msgr2=0x564e2a1212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fdf7fe700 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e29f6b6b0 0x564e2a1212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fdf7fe700 1 -- 172.21.15.8:0/2891475822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe0109880 con 0x564e2a119320 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fdf7fe700 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a119320 0x564e2a128840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8018e40 tx=0x7f3fd8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bca188f5afca47ca server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.204+0000 7f3fc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.205+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/2891475822 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8016cc0 con 0x564e2a119320 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.205+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2891475822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e2a102710 con 0x564e2a119320 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.205+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/2891475822 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3fd8009de0 con 0x564e2a119320 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.205+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/2891475822 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd8016e20 con 0x564e2a119320 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.205+0000 7f3ff8f96b80 10 client.25778 fetch_fsmap learned FSMap version 38 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.205+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2891475822 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x564e2a027130 con 0x564e2a119320 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.205+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/2891475822 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564e2a102710 con 0x564e2a119320 2023-11-27T15:31:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.205+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/2891475822 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3fd80154e0 con 0x564e2a119320 2023-11-27T15:31:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.206+0000 7f3ff8f96b80 10 client.25778 fetch_fsmap finished waiting for FSMap version 38 2023-11-27T15:31:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.206+0000 7f3ff8f96b80 10 client.25778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:31:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.206+0000 7f3ff8f96b80 20 client.25778 populate_metadata read hostname 'smithi008' 2023-11-27T15:31:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.206+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x564e2a12d280 0x564e2a12f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.206+0000 7f3ff8f96b80 4 client.25778 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:31:26.211 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.206+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2891475822 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e2a023040 con 0x564e2a12d280 2023-11-27T15:31:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.206+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x564e2a12d280 0x564e2a12f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.207+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x564e2a12d280 0x564e2a12f660 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102cc39a9508ec67 server_cookie=c07206237a1d0501 in_seq=0 out_seq=0 2023-11-27T15:31:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.207+0000 7f3fc2ff5700 10 client.25778 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.208+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/2891475822 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564e2a023040 con 0x564e2a12d280 2023-11-27T15:31:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.208+0000 7f3fc2ff5700 10 client.25778 handle_command_reply: tid=0 2023-11-27T15:31:26.212 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.208+0000 7f3ff8f96b80 1 client.25778 shutdown 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.208+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2891475822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a119320 msgr2=0x564e2a128840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.208+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a119320 0x564e2a128840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8018e40 tx=0x7f3fd8004ad0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.208+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2891475822 shutdown_connections 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.208+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a119320 0x564e2a128840 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.208+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e29f6b6b0 0x564e2a1212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.208+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x564e2a12d280 0x564e2a12f660 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.209+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/2891475822 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e2a1185c0 0x564e2a121800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.209+0000 7f3fc2ff5700 0 client.25778 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.209+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2891475822 >> 172.21.15.8:0/2891475822 conn(0x564e2a104f50 msgr2=0x564e2a1097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.209+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2891475822 shutdown_connections 2023-11-27T15:31:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.209+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/2891475822 wait complete. 2023-11-27T15:31:26.214 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.209+0000 7f3ff8f96b80 20 client.25778 trim_cache size 0 max 16384 2023-11-27T15:31:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.230+0000 7f3ff8f96b80 1 Processor -- start 2023-11-27T15:31:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3ff8f96b80 1 -- start start 2023-11-27T15:31:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a121070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a128760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 0x564e2a1215b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a023040 con 0x564e2a119320 2023-11-27T15:31:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a1224b0 con 0x564e2a1185c0 2023-11-27T15:31:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a120770 con 0x564e29f6b6b0 2023-11-27T15:31:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a121070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3fdeffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a121070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:56216/0 (socket says 172.21.15.8:56216) 2023-11-27T15:31:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3fdeffd700 1 -- 172.21.15.8:0/1744231939 learned_addr learned my addr 172.21.15.8:0/1744231939 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.231+0000 7f3fde7fc700 1 --2- 172.21.15.8:0/1744231939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a128760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.232+0000 7f3fdf7fe700 1 --2- 172.21.15.8:0/1744231939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 0x564e2a1215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.232+0000 7f3fdeffd700 1 -- 172.21.15.8:0/1744231939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 msgr2=0x564e2a128760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.232+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/1744231939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a128760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.232+0000 7f3fdeffd700 1 -- 172.21.15.8:0/1744231939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 msgr2=0x564e2a1215b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.232+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/1744231939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 0x564e2a1215b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.232+0000 7f3fdeffd700 1 -- 172.21.15.8:0/1744231939 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e2a026e80 con 0x564e29f6b6b0 2023-11-27T15:31:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.232+0000 7f3fdf7fe700 1 --2- 172.21.15.8:0/1744231939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 0x564e2a1215b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:31:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.232+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/1744231939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a121070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000b5a0 tx=0x7f3fd000b510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eb7779c233fc507 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.232+0000 7f3fc5ffb700 1 -- 172.21.15.8:0/1744231939 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00ceea0 con 0x564e29f6b6b0 2023-11-27T15:31:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.233+0000 7f3fc5ffb700 1 -- 172.21.15.8:0/1744231939 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fd009dce0 con 0x564e29f6b6b0 2023-11-27T15:31:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.233+0000 7f3fc5ffb700 1 -- 172.21.15.8:0/1744231939 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00a23b0 con 0x564e29f6b6b0 2023-11-27T15:31:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.234+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1744231939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 msgr2=0x564e2a121070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.234+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/1744231939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a121070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000b5a0 tx=0x7f3fd000b510 comp rx=0 tx=0).stop 2023-11-27T15:31:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.234+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1744231939 shutdown_connections 2023-11-27T15:31:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.234+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/1744231939 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e29f6b6b0 0x564e2a121070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.234+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/1744231939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e2a119320 0x564e2a1215b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.234+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/1744231939 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a128760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.234+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1744231939 >> 172.21.15.8:0/1744231939 conn(0x564e2a104f50 msgr2=0x564e2a133fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.234+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1744231939 shutdown_connections 2023-11-27T15:31:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.234+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1744231939 wait complete. 2023-11-27T15:31:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.235+0000 7f3ff8f96b80 1 -- start start 2023-11-27T15:31:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.235+0000 7f3ff8f96b80 1 Processor -- start 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e29f6b6b0 0x564e2a124300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a124840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3ff8f96b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e2a119320 0x564e2a127860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a127da0 con 0x564e29f6b6b0 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a127f10 con 0x564e2a1185c0 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3ff8f96b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2a1280d0 con 0x564e2a119320 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3ff8f96b80 10 client.0 _ll_register_callbacks cb 0x7f3fd8018810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a124840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a124840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:45448/0 (socket says 172.21.15.8:45448) 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3fde7fc700 1 -- 172.21.15.8:0/1588605112 learned_addr learned my addr 172.21.15.8:0/1588605112 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3fde7fc700 1 -- 172.21.15.8:0/1588605112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e2a119320 msgr2=0x564e2a127860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:31:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.236+0000 7f3fde7fc700 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e2a119320 0x564e2a127860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.237+0000 7f3fde7fc700 1 -- 172.21.15.8:0/1588605112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e29f6b6b0 msgr2=0x564e2a124300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.237+0000 7f3fde7fc700 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e29f6b6b0 0x564e2a124300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.237+0000 7f3fde7fc700 1 -- 172.21.15.8:0/1588605112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e29ee25d0 con 0x564e2a1185c0 2023-11-27T15:31:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.237+0000 7f3fde7fc700 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a124840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400bb70 tx=0x7f3fd400bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43b175b26087677c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.237+0000 7f3fc2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:31:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.237+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/1588605112 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4018040 con 0x564e2a1185c0 2023-11-27T15:31:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.238+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/1588605112 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3fd401d480 con 0x564e2a1185c0 2023-11-27T15:31:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.238+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1588605112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fd00890c0 con 0x564e2a1185c0 2023-11-27T15:31:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.238+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/1588605112 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd401d640 con 0x564e2a1185c0 2023-11-27T15:31:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.238+0000 7f3ff8f96b80 10 client.25784 fetch_fsmap learned FSMap version 38 2023-11-27T15:31:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.238+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1588605112 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x564e29e77c00 con 0x564e2a1185c0 2023-11-27T15:31:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.240+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/1588605112 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fd00890c0 con 0x564e2a1185c0 2023-11-27T15:31:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.240+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/1588605112 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3fd4011930 con 0x564e2a1185c0 2023-11-27T15:31:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.240+0000 7f3ff8f96b80 10 client.25784 fetch_fsmap finished waiting for FSMap version 38 2023-11-27T15:31:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.240+0000 7f3ff8f96b80 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:31:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.240+0000 7f3ff8f96b80 20 client.25784 populate_metadata read hostname 'smithi008' 2023-11-27T15:31:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.240+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x564e2a181f90 0x564e2a1843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.240+0000 7f3ff8f96b80 4 client.25784 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:31:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.240+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1588605112 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564e2a02f600 con 0x564e2a181f90 2023-11-27T15:31:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.241+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x564e2a181f90 0x564e2a1843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.241+0000 7f3fdeffd700 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x564e2a181f90 0x564e2a1843a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fbb399e5ac1092 server_cookie=f07d0b1b80d81144 in_seq=0 out_seq=0 2023-11-27T15:31:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.241+0000 7f3fc2ff5700 10 client.25784 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.543+0000 7f3fc2ff5700 1 -- 172.21.15.8:0/1588605112 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564e2a02f600 con 0x564e2a181f90 2023-11-27T15:31:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.543+0000 7f3fc2ff5700 10 client.25784 handle_command_reply: tid=0 2023-11-27T15:31:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.543+0000 7f3ff8f96b80 1 client.25784 shutdown 2023-11-27T15:31:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1588605112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 msgr2=0x564e2a124840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a124840 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400bb70 tx=0x7f3fd400bf30 comp rx=0 tx=0).stop 2023-11-27T15:31:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1588605112 shutdown_connections 2023-11-27T15:31:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x564e2a181f90 0x564e2a1843a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x564e2a119320 0x564e2a127860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x564e29f6b6b0 0x564e2a124300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3ff8f96b80 1 --2- 172.21.15.8:0/1588605112 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x564e2a1185c0 0x564e2a124840 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1588605112 >> 172.21.15.8:0/1588605112 conn(0x564e2a104f50 msgr2=0x564e2a121070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3fc2ff5700 0 client.25784 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1588605112 shutdown_connections 2023-11-27T15:31:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.545+0000 7f3ff8f96b80 1 -- 172.21.15.8:0/1588605112 wait complete. 2023-11-27T15:31:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:31:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:31:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:31:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:31:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.546+0000 7f3ff8f96b80 20 client.25784 trim_cache size 0 max 16384 2023-11-27T15:31:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.548+0000 7f3fe55ff700 1 -- 172.21.15.8:0/2507069281 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3fc80a11d0 msgr2=0x7f3fc80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.548+0000 7f3fe55ff700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3fc80a11d0 0x7f3fc80a3680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009550 tx=0x7f3fd8009040 comp rx=0 tx=0).stop 2023-11-27T15:31:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.548+0000 7f3fe55ff700 1 -- 172.21.15.8:0/2507069281 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe0100250 msgr2=0x7f3fe0193f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.548+0000 7f3fe55ff700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe0100250 0x7f3fe0193f10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00042c0 tx=0x7f3fd000df90 comp rx=0 tx=0).stop 2023-11-27T15:31:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.548+0000 7f3fe55ff700 1 -- 172.21.15.8:0/2507069281 shutdown_connections 2023-11-27T15:31:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.548+0000 7f3fe55ff700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3fc80a11d0 0x7f3fc80a3680 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.548+0000 7f3fe55ff700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3fe0101dd0 0x7f3fe019b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.548+0000 7f3fe55ff700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fe0100250 0x7f3fe0193f10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.548+0000 7f3fe55ff700 1 --2- 172.21.15.8:0/2507069281 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3fe0101440 0x7f3fe0194450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.548+0000 7f3fe55ff700 1 -- 172.21.15.8:0/2507069281 >> 172.21.15.8:0/2507069281 conn(0x7f3fe00fb850 msgr2=0x7f3fe00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.549+0000 7f3fe55ff700 1 -- 172.21.15.8:0/2507069281 shutdown_connections 2023-11-27T15:31:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:26.549+0000 7f3fe55ff700 1 -- 172.21.15.8:0/2507069281 wait complete. 2023-11-27T15:31:27.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:27 smithi060 ceph-mon[131518]: pgmap v528: 129 pgs: 129 active+clean; 341 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 408 op/s 2023-11-27T15:31:27.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:27 smithi008 ceph-mon[128788]: pgmap v528: 129 pgs: 129 active+clean; 341 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 408 op/s 2023-11-27T15:31:27.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:27 smithi029 ceph-mon[131166]: pgmap v528: 129 pgs: 129 active+clean; 341 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 408 op/s 2023-11-27T15:31:29.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:29 smithi060 ceph-mon[131518]: pgmap v529: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 402 op/s 2023-11-27T15:31:29.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:29 smithi008 ceph-mon[128788]: pgmap v529: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 402 op/s 2023-11-27T15:31:29.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:29 smithi029 ceph-mon[131166]: pgmap v529: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 402 op/s 2023-11-27T15:31:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:31 smithi060 ceph-mon[131518]: pgmap v530: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 337 op/s 2023-11-27T15:31:31.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:31 smithi008 ceph-mon[128788]: pgmap v530: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 337 op/s 2023-11-27T15:31:31.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:31 smithi029 ceph-mon[131166]: pgmap v530: 129 pgs: 129 active+clean; 343 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 337 op/s 2023-11-27T15:31:33.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:33 smithi060 ceph-mon[131518]: pgmap v531: 129 pgs: 129 active+clean; 348 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 433 op/s 2023-11-27T15:31:33.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:33 smithi008 ceph-mon[128788]: pgmap v531: 129 pgs: 129 active+clean; 348 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 433 op/s 2023-11-27T15:31:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:33 smithi029 ceph-mon[131166]: pgmap v531: 129 pgs: 129 active+clean; 348 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 433 op/s 2023-11-27T15:31:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:35 smithi060 ceph-mon[131518]: pgmap v532: 129 pgs: 129 active+clean; 350 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 409 op/s 2023-11-27T15:31:35.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:35 smithi008 ceph-mon[128788]: pgmap v532: 129 pgs: 129 active+clean; 350 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 409 op/s 2023-11-27T15:31:35.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:35 smithi029 ceph-mon[131166]: pgmap v532: 129 pgs: 129 active+clean; 350 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 409 op/s 2023-11-27T15:31:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:37 smithi060 ceph-mon[131518]: pgmap v533: 129 pgs: 129 active+clean; 355 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 417 op/s 2023-11-27T15:31:37.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:37 smithi008 ceph-mon[128788]: pgmap v533: 129 pgs: 129 active+clean; 355 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 417 op/s 2023-11-27T15:31:37.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:37 smithi029 ceph-mon[131166]: pgmap v533: 129 pgs: 129 active+clean; 355 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 417 op/s 2023-11-27T15:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.817+0000 7f1955fff700 1 -- 172.21.15.8:0/314155861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1950101470 msgr2=0x7f19501018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.817+0000 7f1955fff700 1 --2- 172.21.15.8:0/314155861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1950101470 0x7f19501018c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1944009990 tx=0x7f1944009ca0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.817+0000 7f1955fff700 1 -- 172.21.15.8:0/314155861 shutdown_connections 2023-11-27T15:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.817+0000 7f1955fff700 1 --2- 172.21.15.8:0/314155861 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1950101e00 0x7f19501086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.817+0000 7f1955fff700 1 --2- 172.21.15.8:0/314155861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1950100280 0x7f1950100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.817+0000 7f1955fff700 1 --2- 172.21.15.8:0/314155861 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1950101470 0x7f19501018c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.817+0000 7f1955fff700 1 -- 172.21.15.8:0/314155861 >> 172.21.15.8:0/314155861 conn(0x7f19500fb8a0 msgr2=0x7f19500fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.817+0000 7f1955fff700 1 -- 172.21.15.8:0/314155861 shutdown_connections 2023-11-27T15:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.817+0000 7f1955fff700 1 -- 172.21.15.8:0/314155861 wait complete. 2023-11-27T15:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.818+0000 7f1955fff700 1 Processor -- start 2023-11-27T15:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.818+0000 7f1955fff700 1 -- start start 2023-11-27T15:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f1955fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1950100280 0x7f1950195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f1955fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1950101470 0x7f1950196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f1955fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1950101e00 0x7f195019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f1955fff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195019db60 con 0x7f1950101470 2023-11-27T15:31:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f1955fff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195019dca0 con 0x7f1950101e00 2023-11-27T15:31:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f1955fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195019de30 con 0x7f1950100280 2023-11-27T15:31:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:39 smithi060 ceph-mon[131518]: pgmap v534: 129 pgs: 129 active+clean; 355 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 445 op/s 2023-11-27T15:31:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f194ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1950101e00 0x7f195019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f194f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1950100280 0x7f1950195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f194ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1950101e00 0x7f195019d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:47186/0 (socket says 172.21.15.8:47186) 2023-11-27T15:31:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f194ffff700 1 -- 172.21.15.8:0/78879537 learned_addr learned my addr 172.21.15.8:0/78879537 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.819+0000 7f194effd700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1950101470 0x7f1950196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.820+0000 7f194f7fe700 1 -- 172.21.15.8:0/78879537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1950101e00 msgr2=0x7f195019d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.820+0000 7f194f7fe700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1950101e00 0x7f195019d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.820+0000 7f194f7fe700 1 -- 172.21.15.8:0/78879537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1950101470 msgr2=0x7f1950196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.820+0000 7f194f7fe700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1950101470 0x7f1950196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.820+0000 7f194f7fe700 1 -- 172.21.15.8:0/78879537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19440095f0 con 0x7f1950100280 2023-11-27T15:31:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.820+0000 7f194f7fe700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1950100280 0x7f1950195fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f194000e9c0 tx=0x7f194000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.820+0000 7f194cff9700 1 -- 172.21.15.8:0/78879537 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940004220 con 0x7f1950100280 2023-11-27T15:31:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.820+0000 7f194cff9700 1 -- 172.21.15.8:0/78879537 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1940004380 con 0x7f1950100280 2023-11-27T15:31:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.820+0000 7f194cff9700 1 -- 172.21.15.8:0/78879537 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940003c70 con 0x7f1950100280 2023-11-27T15:31:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.820+0000 7f1955fff700 1 -- 172.21.15.8:0/78879537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195010bbe0 con 0x7f1950100280 2023-11-27T15:31:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.821+0000 7f1955fff700 1 -- 172.21.15.8:0/78879537 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195010ba50 con 0x7f1950100280 2023-11-27T15:31:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.825+0000 7f194cff9700 1 -- 172.21.15.8:0/78879537 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f194000c7c0 con 0x7f1950100280 2023-11-27T15:31:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.826+0000 7f194cff9700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f19380a11d0 0x7f19380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.826+0000 7f194cff9700 1 -- 172.21.15.8:0/78879537 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1940014070 con 0x7f1950100280 2023-11-27T15:31:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.827+0000 7f194effd700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f19380a11d0 0x7f19380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.828+0000 7f194effd700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f19380a11d0 0x7f19380a3680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f194400b2e0 tx=0x7f1944005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.828+0000 7f196999cb80 1 Processor -- start 2023-11-27T15:31:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.828+0000 7f196999cb80 1 -- start start 2023-11-27T15:31:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.828+0000 7f196999cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 0x55fc12d8e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.828+0000 7f196999cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8edb0 0x55fc12d8f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.828+0000 7f196999cb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8fb10 0x55fc12d8ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f196999cb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12c94000 con 0x55fc12d8fb10 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f196999cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12ce6840 con 0x55fc12d8edb0 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f194effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8edb0 0x55fc12d8f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f196999cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12cdfcd0 con 0x55fc12bea6b0 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f194effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8edb0 0x55fc12d8f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:47202/0 (socket says 172.21.15.8:47202) 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f194effd700 1 -- 172.21.15.8:0/4172819158 learned_addr learned my addr 172.21.15.8:0/4172819158 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f194f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 0x55fc12d8e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f194ffff700 1 --2- 172.21.15.8:0/4172819158 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8fb10 0x55fc12d8ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f194effd700 1 -- 172.21.15.8:0/4172819158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 msgr2=0x55fc12d8e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f194effd700 1 --2- 172.21.15.8:0/4172819158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 0x55fc12d8e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f194effd700 1 -- 172.21.15.8:0/4172819158 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8fb10 msgr2=0x55fc12d8ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f194effd700 1 --2- 172.21.15.8:0/4172819158 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8fb10 0x55fc12d8ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.829+0000 7f194effd700 1 -- 172.21.15.8:0/4172819158 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc12b615d0 con 0x55fc12d8edb0 2023-11-27T15:31:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.830+0000 7f194effd700 1 --2- 172.21.15.8:0/4172819158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8edb0 0x55fc12d8f5d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f194400bef0 tx=0x7f1944020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b493f6acea029b6d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.830+0000 7f1935ffb700 1 -- 172.21.15.8:0/4172819158 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194402b070 con 0x55fc12d8edb0 2023-11-27T15:31:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.830+0000 7f1935ffb700 1 -- 172.21.15.8:0/4172819158 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194401e040 con 0x55fc12d8edb0 2023-11-27T15:31:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.830+0000 7f1935ffb700 1 -- 172.21.15.8:0/4172819158 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944030820 con 0x55fc12d8edb0 2023-11-27T15:31:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.831+0000 7f196999cb80 1 -- 172.21.15.8:0/4172819158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8edb0 msgr2=0x55fc12d8f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.831+0000 7f196999cb80 1 --2- 172.21.15.8:0/4172819158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8edb0 0x55fc12d8f5d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f194400bef0 tx=0x7f1944020000 comp rx=0 tx=0).stop 2023-11-27T15:31:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.832+0000 7f196999cb80 1 -- 172.21.15.8:0/4172819158 shutdown_connections 2023-11-27T15:31:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.832+0000 7f196999cb80 1 --2- 172.21.15.8:0/4172819158 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 0x55fc12d8e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.832+0000 7f196999cb80 1 --2- 172.21.15.8:0/4172819158 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8fb10 0x55fc12d8ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.832+0000 7f196999cb80 1 --2- 172.21.15.8:0/4172819158 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8edb0 0x55fc12d8f5d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.832+0000 7f196999cb80 1 -- 172.21.15.8:0/4172819158 >> 172.21.15.8:0/4172819158 conn(0x55fc12d83f50 msgr2=0x55fc12d89100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.832+0000 7f196999cb80 1 -- 172.21.15.8:0/4172819158 shutdown_connections 2023-11-27T15:31:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.832+0000 7f196999cb80 1 -- 172.21.15.8:0/4172819158 wait complete. 2023-11-27T15:31:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.833+0000 7f196999cb80 1 -- start start 2023-11-27T15:31:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.833+0000 7f196999cb80 1 Processor -- start 2023-11-27T15:31:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.833+0000 7f196999cb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12bea6b0 0x55fc12d9c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.833+0000 7f196999cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12d8edb0 0x55fc12d9c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.833+0000 7f196999cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12da3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.833+0000 7f196999cb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12c81880 con 0x55fc12bea6b0 2023-11-27T15:31:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.833+0000 7f196999cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12c81520 con 0x55fc12d8fb10 2023-11-27T15:31:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.833+0000 7f196999cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12b638c0 con 0x55fc12d8edb0 2023-11-27T15:31:39.838 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.833+0000 7f196999cb80 10 client.0 _ll_register_callbacks cb 0x55fc12c9f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:31:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12d8edb0 0x55fc12d9c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194effd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12d8edb0 0x55fc12d9c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:59710/0 (socket says 172.21.15.8:59710) 2023-11-27T15:31:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194effd700 1 -- 172.21.15.8:0/686713688 learned_addr learned my addr 172.21.15.8:0/686713688 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194f7fe700 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12bea6b0 0x55fc12d9c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194ffff700 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12da3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194effd700 1 -- 172.21.15.8:0/686713688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 msgr2=0x55fc12da3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194effd700 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12da3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194effd700 1 -- 172.21.15.8:0/686713688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12bea6b0 msgr2=0x55fc12d9c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194effd700 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12bea6b0 0x55fc12d9c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194effd700 1 -- 172.21.15.8:0/686713688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1950108ff0 con 0x55fc12d8edb0 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f194effd700 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12d8edb0 0x55fc12d9c5c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1944005920 tx=0x7f1944030d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f5b7dd1e1d63d1d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.834+0000 7f192effd700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.835+0000 7f192effd700 1 -- 172.21.15.8:0/686713688 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944034070 con 0x55fc12d8edb0 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.835+0000 7f192effd700 1 -- 172.21.15.8:0/686713688 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1944004710 con 0x55fc12d8edb0 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.835+0000 7f196999cb80 1 -- 172.21.15.8:0/686713688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc12d81710 con 0x55fc12d8edb0 2023-11-27T15:31:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.835+0000 7f192effd700 1 -- 172.21.15.8:0/686713688 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f194402b070 con 0x55fc12d8edb0 2023-11-27T15:31:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.836+0000 7f196999cb80 10 client.25819 fetch_fsmap learned FSMap version 38 2023-11-27T15:31:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.836+0000 7f196999cb80 1 -- 172.21.15.8:0/686713688 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55fc12ca6130 con 0x55fc12d8edb0 2023-11-27T15:31:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.836+0000 7f192effd700 1 -- 172.21.15.8:0/686713688 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc12d81710 con 0x55fc12d8edb0 2023-11-27T15:31:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.836+0000 7f192effd700 1 -- 172.21.15.8:0/686713688 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f194400fb10 con 0x55fc12d8edb0 2023-11-27T15:31:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.836+0000 7f196999cb80 10 client.25819 fetch_fsmap finished waiting for FSMap version 38 2023-11-27T15:31:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.836+0000 7f196999cb80 10 client.25819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:31:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.836+0000 7f196999cb80 20 client.25819 populate_metadata read hostname 'smithi008' 2023-11-27T15:31:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.836+0000 7f196999cb80 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55fc12da7930 0x55fc12da9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.836+0000 7f196999cb80 4 client.25819 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:31:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.836+0000 7f196999cb80 1 -- 172.21.15.8:0/686713688 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc12ca2040 con 0x55fc12da7930 2023-11-27T15:31:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.837+0000 7f194f7fe700 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55fc12da7930 0x55fc12da9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.837+0000 7f194f7fe700 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55fc12da7930 0x55fc12da9d40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24865957ce8e428e server_cookie=67e5144ec19dbd01 in_seq=0 out_seq=0 2023-11-27T15:31:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.837+0000 7f192effd700 10 client.25819 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.840+0000 7f192effd700 1 -- 172.21.15.8:0/686713688 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fc12ca2040 con 0x55fc12da7930 2023-11-27T15:31:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.840+0000 7f192effd700 10 client.25819 handle_command_reply: tid=0 2023-11-27T15:31:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.840+0000 7f196999cb80 1 client.25819 shutdown 2023-11-27T15:31:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 1 -- 172.21.15.8:0/686713688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12d8edb0 msgr2=0x55fc12d9c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12d8edb0 0x55fc12d9c5c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1944005920 tx=0x7f1944030d90 comp rx=0 tx=0).stop 2023-11-27T15:31:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 1 -- 172.21.15.8:0/686713688 shutdown_connections 2023-11-27T15:31:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12da3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12bea6b0 0x55fc12d9c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55fc12da7930 0x55fc12da9d40 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 1 --2- 172.21.15.8:0/686713688 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12d8edb0 0x55fc12d9c5c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f192effd700 0 client.25819 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 1 -- 172.21.15.8:0/686713688 >> 172.21.15.8:0/686713688 conn(0x55fc12d83f50 msgr2=0x55fc12d887e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 1 -- 172.21.15.8:0/686713688 shutdown_connections 2023-11-27T15:31:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 1 -- 172.21.15.8:0/686713688 wait complete. 2023-11-27T15:31:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.841+0000 7f196999cb80 20 client.25819 trim_cache size 0 max 16384 2023-11-27T15:31:39.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:39 smithi008 ceph-mon[128788]: pgmap v534: 129 pgs: 129 active+clean; 355 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 445 op/s 2023-11-27T15:31:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f196999cb80 1 Processor -- start 2023-11-27T15:31:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f196999cb80 1 -- start start 2023-11-27T15:31:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f196999cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 0x55fc12d929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f196999cb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8edb0 0x55fc12da35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f196999cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d92f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f196999cb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12d8f1c0 con 0x55fc12d8edb0 2023-11-27T15:31:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f196999cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12d8f3c0 con 0x55fc12d8fb10 2023-11-27T15:31:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f196999cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12d95c60 con 0x55fc12bea6b0 2023-11-27T15:31:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f194f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 0x55fc12d929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f194effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8edb0 0x55fc12da35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.868+0000 7f194ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d92f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.869+0000 7f194f7fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 0x55fc12d929c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:59726/0 (socket says 172.21.15.8:59726) 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.869+0000 7f194f7fe700 1 -- 172.21.15.8:0/1404691260 learned_addr learned my addr 172.21.15.8:0/1404691260 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.869+0000 7f194effd700 1 -- 172.21.15.8:0/1404691260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 msgr2=0x55fc12d929c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.869+0000 7f194effd700 1 --2- 172.21.15.8:0/1404691260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 0x55fc12d929c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.869+0000 7f194effd700 1 -- 172.21.15.8:0/1404691260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 msgr2=0x55fc12d92f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.869+0000 7f194effd700 1 --2- 172.21.15.8:0/1404691260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d92f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.869+0000 7f194effd700 1 -- 172.21.15.8:0/1404691260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc12ca5e80 con 0x55fc12d8edb0 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.869+0000 7f194f7fe700 1 --2- 172.21.15.8:0/1404691260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 0x55fc12d929c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.869+0000 7f194ffff700 1 --2- 172.21.15.8:0/1404691260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d92f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:31:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.869+0000 7f194effd700 1 --2- 172.21.15.8:0/1404691260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8edb0 0x55fc12da35c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1944022640 tx=0x7f194400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9365fce5cf63da server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.870+0000 7f19367fc700 1 -- 172.21.15.8:0/1404691260 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944034020 con 0x55fc12d8edb0 2023-11-27T15:31:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.870+0000 7f19367fc700 1 -- 172.21.15.8:0/1404691260 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194402c040 con 0x55fc12d8edb0 2023-11-27T15:31:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.870+0000 7f19367fc700 1 -- 172.21.15.8:0/1404691260 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194402b070 con 0x55fc12d8edb0 2023-11-27T15:31:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.870+0000 7f196999cb80 1 -- 172.21.15.8:0/1404691260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8edb0 msgr2=0x55fc12da35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.870+0000 7f196999cb80 1 --2- 172.21.15.8:0/1404691260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8edb0 0x55fc12da35c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1944022640 tx=0x7f194400b920 comp rx=0 tx=0).stop 2023-11-27T15:31:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.871+0000 7f196999cb80 1 -- 172.21.15.8:0/1404691260 shutdown_connections 2023-11-27T15:31:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.871+0000 7f196999cb80 1 --2- 172.21.15.8:0/1404691260 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12bea6b0 0x55fc12d929c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.871+0000 7f196999cb80 1 --2- 172.21.15.8:0/1404691260 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d92f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.871+0000 7f196999cb80 1 --2- 172.21.15.8:0/1404691260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12d8edb0 0x55fc12da35c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.871+0000 7f196999cb80 1 -- 172.21.15.8:0/1404691260 >> 172.21.15.8:0/1404691260 conn(0x55fc12d83f50 msgr2=0x55fc12dae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.871+0000 7f196999cb80 1 -- 172.21.15.8:0/1404691260 shutdown_connections 2023-11-27T15:31:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.871+0000 7f196999cb80 1 -- 172.21.15.8:0/1404691260 wait complete. 2023-11-27T15:31:39.877 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.871+0000 7f196999cb80 1 -- start start 2023-11-27T15:31:39.877 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.871+0000 7f196999cb80 1 Processor -- start 2023-11-27T15:31:39.877 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.872+0000 7f196999cb80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12bea6b0 0x55fc12d9eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.877 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.872+0000 7f196999cb80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d9f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.877 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.872+0000 7f196999cb80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12da2300 0x55fc12da2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.877 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.872+0000 7f196999cb80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12da2cb0 con 0x55fc12bea6b0 2023-11-27T15:31:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.872+0000 7f196999cb80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12da2e20 con 0x55fc12d8fb10 2023-11-27T15:31:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.872+0000 7f196999cb80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc12da2fe0 con 0x55fc12da2300 2023-11-27T15:31:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f196999cb80 10 client.0 _ll_register_callbacks cb 0x7f1940004750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:31:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f194f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12bea6b0 0x55fc12d9eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f194effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d9f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f194effd700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d9f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:47218/0 (socket says 172.21.15.8:47218) 2023-11-27T15:31:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f194effd700 1 -- 172.21.15.8:0/3260680514 learned_addr learned my addr 172.21.15.8:0/3260680514 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f194ffff700 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12da2300 0x55fc12da2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f194effd700 1 -- 172.21.15.8:0/3260680514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12da2300 msgr2=0x55fc12da2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f194effd700 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12da2300 0x55fc12da2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f194effd700 1 -- 172.21.15.8:0/3260680514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12bea6b0 msgr2=0x55fc12d9eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f194effd700 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12bea6b0 0x55fc12d9eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.873+0000 7f194effd700 1 -- 172.21.15.8:0/3260680514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195010ba50 con 0x55fc12d8fb10 2023-11-27T15:31:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.874+0000 7f194effd700 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d9f2e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1944005600 tx=0x7f194400b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c256612c60da634 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.874+0000 7f192effd700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:31:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.874+0000 7f192effd700 1 -- 172.21.15.8:0/3260680514 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944034030 con 0x55fc12d8fb10 2023-11-27T15:31:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.874+0000 7f196999cb80 1 -- 172.21.15.8:0/3260680514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f194401e040 con 0x55fc12d8fb10 2023-11-27T15:31:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.874+0000 7f192effd700 1 -- 172.21.15.8:0/3260680514 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f1944004910 con 0x55fc12d8fb10 2023-11-27T15:31:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.875+0000 7f192effd700 1 -- 172.21.15.8:0/3260680514 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f194402b040 con 0x55fc12d8fb10 2023-11-27T15:31:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.875+0000 7f196999cb80 10 client.25802 fetch_fsmap learned FSMap version 38 2023-11-27T15:31:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.875+0000 7f196999cb80 1 -- 172.21.15.8:0/3260680514 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f1948009350 con 0x55fc12d8fb10 2023-11-27T15:31:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.875+0000 7f192effd700 1 -- 172.21.15.8:0/3260680514 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194401e040 con 0x55fc12d8fb10 2023-11-27T15:31:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.875+0000 7f192effd700 1 -- 172.21.15.8:0/3260680514 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f194404d070 con 0x55fc12d8fb10 2023-11-27T15:31:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.875+0000 7f196999cb80 10 client.25802 fetch_fsmap finished waiting for FSMap version 38 2023-11-27T15:31:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.875+0000 7f196999cb80 10 client.25802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:31:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.876+0000 7f196999cb80 20 client.25802 populate_metadata read hostname 'smithi008' 2023-11-27T15:31:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.876+0000 7f196999cb80 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55fc12dfc060 0x55fc12dfe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.876+0000 7f196999cb80 4 client.25802 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:31:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.876+0000 7f196999cb80 1 -- 172.21.15.8:0/3260680514 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc12cae600 con 0x55fc12dfc060 2023-11-27T15:31:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.876+0000 7f194f7fe700 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55fc12dfc060 0x55fc12dfe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.876+0000 7f194f7fe700 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55fc12dfc060 0x55fc12dfe470 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60ee795f0d0c5e22 server_cookie=6f3c89b221f79dcc in_seq=0 out_seq=0 2023-11-27T15:31:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:39.877+0000 7f192effd700 10 client.25802 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:39.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:39 smithi029 ceph-mon[131166]: pgmap v534: 129 pgs: 129 active+clean; 355 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 445 op/s 2023-11-27T15:31:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.813+0000 7f192effd700 1 -- 172.21.15.8:0/3260680514 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc12cae600 con 0x55fc12dfc060 2023-11-27T15:31:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.813+0000 7f192effd700 10 client.25802 handle_command_reply: tid=0 2023-11-27T15:31:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.813+0000 7f196999cb80 1 client.25802 shutdown 2023-11-27T15:31:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.815+0000 7f196999cb80 1 -- 172.21.15.8:0/3260680514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 msgr2=0x55fc12d9f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.815+0000 7f196999cb80 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d9f2e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1944005600 tx=0x7f194400b400 comp rx=0 tx=0).stop 2023-11-27T15:31:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.816+0000 7f196999cb80 1 -- 172.21.15.8:0/3260680514 shutdown_connections 2023-11-27T15:31:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.816+0000 7f196999cb80 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55fc12dfc060 0x55fc12dfe470 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.816+0000 7f196999cb80 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55fc12da2300 0x55fc12da2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.816+0000 7f196999cb80 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55fc12bea6b0 0x55fc12d9eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.816+0000 7f196999cb80 1 --2- 172.21.15.8:0/3260680514 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55fc12d8fb10 0x55fc12d9f2e0 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1944005600 tx=0x7f194400b400 comp rx=0 tx=0).stop 2023-11-27T15:31:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.816+0000 7f192effd700 0 client.25802 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.816+0000 7f196999cb80 1 -- 172.21.15.8:0/3260680514 >> 172.21.15.8:0/3260680514 conn(0x55fc12d83f50 msgr2=0x55fc12d929c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.817+0000 7f196999cb80 1 -- 172.21.15.8:0/3260680514 shutdown_connections 2023-11-27T15:31:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.817+0000 7f196999cb80 1 -- 172.21.15.8:0/3260680514 wait complete. 2023-11-27T15:31:40.826 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.821+0000 7f196999cb80 20 client.25802 trim_cache size 0 max 16384 2023-11-27T15:31:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:31:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:31:40.827 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:31:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:31:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.824+0000 7f1955fff700 1 -- 172.21.15.8:0/78879537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f19380a11d0 msgr2=0x7f19380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.824+0000 7f1955fff700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f19380a11d0 0x7f19380a3680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f194400b2e0 tx=0x7f1944005fb0 comp rx=0 tx=0).stop 2023-11-27T15:31:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.824+0000 7f1955fff700 1 -- 172.21.15.8:0/78879537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1950100280 msgr2=0x7f1950195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.824+0000 7f1955fff700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1950100280 0x7f1950195fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f194000e9c0 tx=0x7f194000ecd0 comp rx=0 tx=0).stop 2023-11-27T15:31:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.825+0000 7f1955fff700 1 -- 172.21.15.8:0/78879537 shutdown_connections 2023-11-27T15:31:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.825+0000 7f1955fff700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f19380a11d0 0x7f19380a3680 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.825+0000 7f1955fff700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1950100280 0x7f1950195fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.825+0000 7f1955fff700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1950101e00 0x7f195019d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.825+0000 7f1955fff700 1 --2- 172.21.15.8:0/78879537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1950101470 0x7f1950196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.825+0000 7f1955fff700 1 -- 172.21.15.8:0/78879537 >> 172.21.15.8:0/78879537 conn(0x7f19500fb8a0 msgr2=0x7f1950104900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.825+0000 7f1955fff700 1 -- 172.21.15.8:0/78879537 shutdown_connections 2023-11-27T15:31:40.830 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:40.826+0000 7f1955fff700 1 -- 172.21.15.8:0/78879537 wait complete. 2023-11-27T15:31:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:41 smithi060 ceph-mon[131518]: pgmap v535: 129 pgs: 129 active+clean; 359 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 394 op/s 2023-11-27T15:31:41.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:41 smithi008 ceph-mon[128788]: pgmap v535: 129 pgs: 129 active+clean; 359 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 394 op/s 2023-11-27T15:31:41.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:41 smithi029 ceph-mon[131166]: pgmap v535: 129 pgs: 129 active+clean; 359 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 394 op/s 2023-11-27T15:31:43.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:43 smithi060 ceph-mon[131518]: pgmap v536: 129 pgs: 129 active+clean; 360 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 454 op/s 2023-11-27T15:31:43.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:43 smithi008 ceph-mon[128788]: pgmap v536: 129 pgs: 129 active+clean; 360 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 454 op/s 2023-11-27T15:31:43.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:43 smithi029 ceph-mon[131166]: pgmap v536: 129 pgs: 129 active+clean; 360 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 454 op/s 2023-11-27T15:31:45.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:45 smithi060 ceph-mon[131518]: pgmap v537: 129 pgs: 129 active+clean; 360 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 386 op/s 2023-11-27T15:31:45.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:45 smithi008 ceph-mon[128788]: pgmap v537: 129 pgs: 129 active+clean; 360 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 386 op/s 2023-11-27T15:31:45.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:45 smithi029 ceph-mon[131166]: pgmap v537: 129 pgs: 129 active+clean; 360 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 386 op/s 2023-11-27T15:31:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:47 smithi060 ceph-mon[131518]: pgmap v538: 129 pgs: 129 active+clean; 365 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 379 op/s 2023-11-27T15:31:47.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:47 smithi008 ceph-mon[128788]: pgmap v538: 129 pgs: 129 active+clean; 365 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 379 op/s 2023-11-27T15:31:47.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:47 smithi029 ceph-mon[131166]: pgmap v538: 129 pgs: 129 active+clean; 365 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 379 op/s 2023-11-27T15:31:49.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:49 smithi060 ceph-mon[131518]: pgmap v539: 129 pgs: 129 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2023-11-27T15:31:49.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:49 smithi008 ceph-mon[128788]: pgmap v539: 129 pgs: 129 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2023-11-27T15:31:49.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:49 smithi029 ceph-mon[131166]: pgmap v539: 129 pgs: 129 active+clean; 367 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2023-11-27T15:31:51.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:51 smithi060 ceph-mon[131518]: pgmap v540: 129 pgs: 129 active+clean; 370 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 338 op/s 2023-11-27T15:31:51.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:51 smithi008 ceph-mon[128788]: pgmap v540: 129 pgs: 129 active+clean; 370 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 338 op/s 2023-11-27T15:31:51.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:51 smithi029 ceph-mon[131166]: pgmap v540: 129 pgs: 129 active+clean; 370 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 338 op/s 2023-11-27T15:31:52.956 INFO:tasks.workunit.client.0.smithi008.stdout:cleaned up hierarchy 2023-11-27T15:31:53.799 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:31:53.799 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:31:53.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:53 smithi060 ceph-mon[131518]: pgmap v541: 129 pgs: 129 active+clean; 370 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 315 op/s 2023-11-27T15:31:53.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:53 smithi008 ceph-mon[128788]: pgmap v541: 129 pgs: 129 active+clean; 370 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 315 op/s 2023-11-27T15:31:53.896 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:53 smithi029 ceph-mon[131166]: pgmap v541: 129 pgs: 129 active+clean; 370 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 315 op/s 2023-11-27T15:31:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.028+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/363678436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1e8101e70 msgr2=0x7fc1e8108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.028+0000 7fc1ee9ff700 1 --2- 172.21.15.8:0/363678436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1e8101e70 0x7fc1e8108710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000b0a0 tx=0x7fc1e000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.028+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/363678436 shutdown_connections 2023-11-27T15:31:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.028+0000 7fc1ee9ff700 1 --2- 172.21.15.8:0/363678436 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e8100230 0x7fc1e8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.028+0000 7fc1ee9ff700 1 --2- 172.21.15.8:0/363678436 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1e8101e70 0x7fc1e8108710 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.028+0000 7fc1ee9ff700 1 --2- 172.21.15.8:0/363678436 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1e81014c0 0x7fc1e8101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.028+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/363678436 >> 172.21.15.8:0/363678436 conn(0x7fc1e80fb810 msgr2=0x7fc1e80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:54.032 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.028+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/363678436 shutdown_connections 2023-11-27T15:31:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.028+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/363678436 wait complete. 2023-11-27T15:31:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.030+0000 7fc1ee9ff700 1 Processor -- start 2023-11-27T15:31:54.033 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.030+0000 7fc1ee9ff700 1 -- start start 2023-11-27T15:31:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1ee9ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1e8100230 0x7fc1e81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1ee9ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e81014c0 0x7fc1e8198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1ee9ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1e8101e70 0x7fc1e819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.035 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1ee9ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e819fe70 con 0x7fc1e8100230 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1ee9ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e819ffb0 con 0x7fc1e8101e70 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1ee9ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e81a0140 con 0x7fc1e81014c0 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e81014c0 0x7fc1e8198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1e8100230 0x7fc1e81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e81014c0 0x7fc1e8198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:56848/0 (socket says 172.21.15.8:56848) 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1e8100230 0x7fc1e81982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:32824/0 (socket says 172.21.15.8:32824) 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e77fe700 1 -- 172.21.15.8:0/2787337413 learned_addr learned my addr 172.21.15.8:0/2787337413 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1ecf9c700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1e8101e70 0x7fc1e819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e77fe700 1 -- 172.21.15.8:0/2787337413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1e8101e70 msgr2=0x7fc1e819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e77fe700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1e8101e70 0x7fc1e819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e77fe700 1 -- 172.21.15.8:0/2787337413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1e8100230 msgr2=0x7fc1e81982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:54.036 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e77fe700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1e8100230 0x7fc1e81982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e77fe700 1 -- 172.21.15.8:0/2787337413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc009520 con 0x7fc1e81014c0 2023-11-27T15:31:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.031+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1e8100230 0x7fc1e81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:31:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.032+0000 7fc1e77fe700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e81014c0 0x7fc1e8198830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00eb30 tx=0x7fc1dc00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.032+0000 7fc1e57fa700 1 -- 172.21.15.8:0/2787337413 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc009880 con 0x7fc1e81014c0 2023-11-27T15:31:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.032+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/2787337413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e0009d00 con 0x7fc1e81014c0 2023-11-27T15:31:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.032+0000 7fc1e57fa700 1 -- 172.21.15.8:0/2787337413 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1dc0044e0 con 0x7fc1e81014c0 2023-11-27T15:31:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.032+0000 7fc1e57fa700 1 -- 172.21.15.8:0/2787337413 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc003d80 con 0x7fc1e81014c0 2023-11-27T15:31:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.032+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/2787337413 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e810bc80 con 0x7fc1e81014c0 2023-11-27T15:31:54.038 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.035+0000 7fc1e57fa700 1 -- 172.21.15.8:0/2787337413 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc1dc003f20 con 0x7fc1e81014c0 2023-11-27T15:31:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.037+0000 7fc1e57fa700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc1d00a1150 0x7fc1d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.037+0000 7fc2023f2b80 1 Processor -- start 2023-11-27T15:31:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.037+0000 7fc1e57fa700 1 -- 172.21.15.8:0/2787337413 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc1dc014070 con 0x7fc1e81014c0 2023-11-27T15:31:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.037+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc1d00a1150 0x7fc1d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.040 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.037+0000 7fc2023f2b80 1 -- start start 2023-11-27T15:31:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.037+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b65d6b0 0x55c93b801870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.037+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b801db0 0x55c93b8025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.037+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 0x55c93b802f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b707000 con 0x55c93b801db0 2023-11-27T15:31:54.042 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b759840 con 0x55c93b802b10 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b752cd0 con 0x55c93b65d6b0 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc1d00a1150 0x7fc1d00a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8011490 tx=0x7fc1d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b65d6b0 0x55c93b801870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e7fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b65d6b0 0x55c93b801870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:56860/0 (socket says 172.21.15.8:56860) 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e7fff700 1 -- 172.21.15.8:0/1132596700 learned_addr learned my addr 172.21.15.8:0/1132596700 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e77fe700 1 --2- 172.21.15.8:0/1132596700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b801db0 0x55c93b8025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1ecf9c700 1 --2- 172.21.15.8:0/1132596700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 0x55c93b802f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e7fff700 1 -- 172.21.15.8:0/1132596700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 msgr2=0x55c93b802f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/1132596700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 0x55c93b802f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e7fff700 1 -- 172.21.15.8:0/1132596700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b801db0 msgr2=0x55c93b8025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/1132596700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b801db0 0x55c93b8025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e7fff700 1 -- 172.21.15.8:0/1132596700 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c93b5d45d0 con 0x55c93b65d6b0 2023-11-27T15:31:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.038+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/1132596700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b65d6b0 0x55c93b801870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8004390 tx=0x7fc1d8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbfc1ee75ab4336f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.039+0000 7fc1ce7fc700 1 -- 172.21.15.8:0/1132596700 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8003d10 con 0x55c93b65d6b0 2023-11-27T15:31:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.039+0000 7fc1ce7fc700 1 -- 172.21.15.8:0/1132596700 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1d8003e70 con 0x55c93b65d6b0 2023-11-27T15:31:54.044 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.039+0000 7fc1ce7fc700 1 -- 172.21.15.8:0/1132596700 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d801e7f0 con 0x55c93b65d6b0 2023-11-27T15:31:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.039+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1132596700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b65d6b0 msgr2=0x55c93b801870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.039+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1132596700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b65d6b0 0x55c93b801870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8004390 tx=0x7fc1d8014000 comp rx=0 tx=0).stop 2023-11-27T15:31:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.040+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1132596700 shutdown_connections 2023-11-27T15:31:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.040+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1132596700 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b65d6b0 0x55c93b801870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.040+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1132596700 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 0x55c93b802f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.040+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1132596700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b801db0 0x55c93b8025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.040+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1132596700 >> 172.21.15.8:0/1132596700 conn(0x55c93b7f6f50 msgr2=0x55c93b7fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.040+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1132596700 shutdown_connections 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.040+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1132596700 wait complete. 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.040+0000 7fc2023f2b80 1 -- start start 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.040+0000 7fc2023f2b80 1 Processor -- start 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b65d6b0 0x55c93b80ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b801db0 0x55c93b80b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 0x55c93b812210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b6f4880 con 0x55c93b801db0 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b6f4520 con 0x55c93b65d6b0 2023-11-27T15:31:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b5d68c0 con 0x55c93b802b10 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc2023f2b80 10 client.0 _ll_register_callbacks cb 0x55c93b712720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc1ecf9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 0x55c93b812210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc1e77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b801db0 0x55c93b80b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc1ecf9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 0x55c93b812210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:56866/0 (socket says 172.21.15.8:56866) 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc1ecf9c700 1 -- 172.21.15.8:0/1379768516 learned_addr learned my addr 172.21.15.8:0/1379768516 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.041+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b65d6b0 0x55c93b80ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc1ecf9c700 1 -- 172.21.15.8:0/1379768516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b65d6b0 msgr2=0x55c93b80ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc1ecf9c700 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b65d6b0 0x55c93b80ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc1ecf9c700 1 -- 172.21.15.8:0/1379768516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b801db0 msgr2=0x55c93b80b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc1ecf9c700 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b801db0 0x55c93b80b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc1ecf9c700 1 -- 172.21.15.8:0/1379768516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d8017070 con 0x55c93b802b10 2023-11-27T15:31:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc1ecf9c700 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 0x55c93b812210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000b0a0 tx=0x7fc1e000bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c60385a30a75ae4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc1c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.60:3300/0 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1379768516 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000e040 con 0x55c93b802b10 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1379768516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c93b7f4710 con 0x55c93b802b10 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1379768516 <== mon.1 v2:172.21.15.60:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc1e000be30 con 0x55c93b802b10 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1379768516 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1e0003c00 con 0x55c93b802b10 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc2023f2b80 10 client.25837 fetch_fsmap learned FSMap version 38 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.042+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1379768516 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c93b719130 con 0x55c93b802b10 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.043+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1379768516 <== mon.1 v2:172.21.15.60:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c93b7f4710 con 0x55c93b802b10 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.043+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1379768516 <== mon.1 v2:172.21.15.60:3300/0 5 ==== fsmap(e 38) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc1e0023020 con 0x55c93b802b10 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.043+0000 7fc2023f2b80 10 client.25837 fetch_fsmap finished waiting for FSMap version 38 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.043+0000 7fc2023f2b80 10 client.25837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.043+0000 7fc2023f2b80 20 client.25837 populate_metadata read hostname 'smithi008' 2023-11-27T15:31:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.043+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55c93b816b10 0x55c93b818f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.043+0000 7fc2023f2b80 4 client.25837 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:31:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.043+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1379768516 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c93b715040 con 0x55c93b816b10 2023-11-27T15:31:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.043+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55c93b816b10 0x55c93b818f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.044+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55c93b816b10 0x55c93b818f20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73faeadcecb8ba18 server_cookie=aaea8605ea628c3e in_seq=0 out_seq=0 2023-11-27T15:31:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.044+0000 7fc1c77fe700 10 client.25837 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.044+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1379768516 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c93b715040 con 0x55c93b816b10 2023-11-27T15:31:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.044+0000 7fc1c77fe700 10 client.25837 handle_command_reply: tid=0 2023-11-27T15:31:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 client.25837 shutdown 2023-11-27T15:31:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1379768516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 msgr2=0x55c93b812210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 0x55c93b812210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000b0a0 tx=0x7fc1e000bab0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1379768516 shutdown_connections 2023-11-27T15:31:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55c93b816b10 0x55c93b818f20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 0x55c93b812210 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b65d6b0 0x55c93b80ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1379768516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b801db0 0x55c93b80b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc1c77fe700 0 client.25837 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1379768516 >> 172.21.15.8:0/1379768516 conn(0x55c93b7f6f50 msgr2=0x55c93b7fb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1379768516 shutdown_connections 2023-11-27T15:31:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1379768516 wait complete. 2023-11-27T15:31:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.045+0000 7fc2023f2b80 20 client.25837 trim_cache size 0 max 16384 2023-11-27T15:31:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.060+0000 7fc2023f2b80 1 Processor -- start 2023-11-27T15:31:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.060+0000 7fc2023f2b80 1 -- start start 2023-11-27T15:31:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 0x55c93b808df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b801db0 0x55c93b8121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 0x55c93b809330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.064 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b7f4250 con 0x55c93b65d6b0 2023-11-27T15:31:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b802420 con 0x55c93b801db0 2023-11-27T15:31:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b7fccf0 con 0x55c93b802b10 2023-11-27T15:31:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc1e77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b801db0 0x55c93b8121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc1e7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 0x55c93b808df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc1ecf9c700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 0x55c93b809330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc1e77fe700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b801db0 0x55c93b8121d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:39304/0 (socket says 172.21.15.8:39304) 2023-11-27T15:31:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc1e77fe700 1 -- 172.21.15.8:0/1454208926 learned_addr learned my addr 172.21.15.8:0/1454208926 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc1e77fe700 1 -- 172.21.15.8:0/1454208926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 msgr2=0x55c93b809330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc1e77fe700 1 --2- 172.21.15.8:0/1454208926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 0x55c93b809330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc1e77fe700 1 -- 172.21.15.8:0/1454208926 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 msgr2=0x55c93b808df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc1e77fe700 1 --2- 172.21.15.8:0/1454208926 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 0x55c93b808df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.061+0000 7fc1e77fe700 1 -- 172.21.15.8:0/1454208926 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c93b718e80 con 0x55c93b801db0 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.062+0000 7fc1e77fe700 1 --2- 172.21.15.8:0/1454208926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b801db0 0x55c93b8121d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00c5b0 tx=0x7fc1dc085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e853803b62c54425 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.062+0000 7fc1ceffd700 1 -- 172.21.15.8:0/1454208926 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc00cbd0 con 0x55c93b801db0 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.062+0000 7fc1ceffd700 1 -- 172.21.15.8:0/1454208926 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1dc09a250 con 0x55c93b801db0 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.062+0000 7fc1ceffd700 1 -- 172.21.15.8:0/1454208926 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc099740 con 0x55c93b801db0 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1454208926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b801db0 msgr2=0x55c93b8121d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1454208926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b801db0 0x55c93b8121d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00c5b0 tx=0x7fc1dc085040 comp rx=0 tx=0).stop 2023-11-27T15:31:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1454208926 shutdown_connections 2023-11-27T15:31:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1454208926 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b802b10 0x55c93b809330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1454208926 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 0x55c93b808df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1454208926 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b801db0 0x55c93b8121d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1454208926 >> 172.21.15.8:0/1454208926 conn(0x55c93b7f6f50 msgr2=0x55c93b805920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1454208926 shutdown_connections 2023-11-27T15:31:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1454208926 wait complete. 2023-11-27T15:31:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 -- start start 2023-11-27T15:31:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.063+0000 7fc2023f2b80 1 Processor -- start 2023-11-27T15:31:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 0x55c93b811ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b801db0 0x55c93b8120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc2023f2b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 0x55c93b80e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b80ed90 con 0x55c93b65d6b0 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b80ef00 con 0x55c93b802b10 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc2023f2b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93b80f0c0 con 0x55c93b801db0 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc2023f2b80 10 client.0 _ll_register_callbacks cb 0x55c93b712720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1e7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 0x55c93b811ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1ecf9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 0x55c93b80e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1e77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b801db0 0x55c93b8120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1ecf9c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 0x55c93b80e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:39318/0 (socket says 172.21.15.8:39318) 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1e77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b801db0 0x55c93b8120e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:56872/0 (socket says 172.21.15.8:56872) 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1ecf9c700 1 -- 172.21.15.8:0/1053660108 learned_addr learned my addr 172.21.15.8:0/1053660108 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:31:54.071 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1ecf9c700 1 -- 172.21.15.8:0/1053660108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b801db0 msgr2=0x55c93b8120e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1ecf9c700 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b801db0 0x55c93b8120e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1ecf9c700 1 -- 172.21.15.8:0/1053660108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 msgr2=0x55c93b811ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1ecf9c700 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 0x55c93b811ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1ecf9c700 1 -- 172.21.15.8:0/1053660108 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e810bc80 con 0x55c93b802b10 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.064+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 0x55c93b811ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.065+0000 7fc1ecf9c700 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 0x55c93b80e680 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000b3f0 tx=0x7fc1e0003cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d1c4869833b9c91 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.065+0000 7fc1c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.065+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1053660108 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000e030 con 0x55c93b802b10 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.065+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1053660108 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc1e001d6b0 con 0x55c93b802b10 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.065+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1053660108 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1d8003e70 con 0x55c93b802b10 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.065+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1053660108 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1e0003ea0 con 0x55c93b802b10 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.065+0000 7fc2023f2b80 10 client.25820 fetch_fsmap learned FSMap version 38 2023-11-27T15:31:54.072 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.065+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1053660108 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c93b569c00 con 0x55c93b802b10 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.065+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1053660108 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1d8003e70 con 0x55c93b802b10 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.066+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1053660108 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc1e0019040 con 0x55c93b802b10 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.066+0000 7fc2023f2b80 10 client.25820 fetch_fsmap finished waiting for FSMap version 38 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.066+0000 7fc2023f2b80 10 client.25820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.066+0000 7fc2023f2b80 20 client.25820 populate_metadata read hostname 'smithi008' 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.066+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55c93b86be10 0x55c93b86e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.066+0000 7fc2023f2b80 4 client.25820 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.066+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1053660108 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c93b721600 con 0x55c93b86be10 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.066+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55c93b86be10 0x55c93b86e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.067+0000 7fc1e7fff700 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55c93b86be10 0x55c93b86e220 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b9eff5a58c4e13 server_cookie=5257d1aabdb0017 in_seq=0 out_seq=0 2023-11-27T15:31:54.073 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.067+0000 7fc1c77fe700 10 client.25820 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc1c77fe700 1 -- 172.21.15.8:0/1053660108 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c93b721600 con 0x55c93b86be10 2023-11-27T15:31:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc1c77fe700 10 client.25820 handle_command_reply: tid=0 2023-11-27T15:31:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc2023f2b80 1 client.25820 shutdown 2023-11-27T15:31:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1053660108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 msgr2=0x55c93b80e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 0x55c93b80e680 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000b3f0 tx=0x7fc1e0003cb0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1053660108 shutdown_connections 2023-11-27T15:31:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x55c93b802b10 0x55c93b80e680 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x55c93b65d6b0 0x55c93b811ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x55c93b86be10 0x55c93b86e220 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc2023f2b80 1 --2- 172.21.15.8:0/1053660108 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x55c93b801db0 0x55c93b8120e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc1c77fe700 0 client.25820 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:31:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.087+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1053660108 >> 172.21.15.8:0/1053660108 conn(0x55c93b7f6f50 msgr2=0x55c93b80ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.088+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1053660108 shutdown_connections 2023-11-27T15:31:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.088+0000 7fc2023f2b80 1 -- 172.21.15.8:0/1053660108 wait complete. 2023-11-27T15:31:54.092 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.088+0000 7fc2023f2b80 20 client.25820 trim_cache size 0 max 16384 2023-11-27T15:31:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:31:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:31:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:31:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:31:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.088+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/2787337413 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc1d00a1150 msgr2=0x7fc1d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.088+0000 7fc1ee9ff700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc1d00a1150 0x7fc1d00a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8011490 tx=0x7fc1d8009260 comp rx=0 tx=0).stop 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.088+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/2787337413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e81014c0 msgr2=0x7fc1e8198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.088+0000 7fc1ee9ff700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e81014c0 0x7fc1e8198830 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00eb30 tx=0x7fc1dc00ee40 comp rx=0 tx=0).stop 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.089+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/2787337413 shutdown_connections 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.089+0000 7fc1ee9ff700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc1e8101e70 0x7fc1e819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.089+0000 7fc1ee9ff700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1e8100230 0x7fc1e81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.089+0000 7fc1ee9ff700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc1d00a1150 0x7fc1d00a3600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.089+0000 7fc1ee9ff700 1 --2- 172.21.15.8:0/2787337413 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc1e81014c0 0x7fc1e8198830 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.089+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/2787337413 >> 172.21.15.8:0/2787337413 conn(0x7fc1e80fb810 msgr2=0x7fc1e8106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.089+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/2787337413 shutdown_connections 2023-11-27T15:31:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:31:54.089+0000 7fc1ee9ff700 1 -- 172.21.15.8:0/2787337413 wait complete. 2023-11-27T15:31:55.387 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2023-11-27T15:31:55.387 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2023-11-27T15:31:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:55 smithi060 ceph-mon[131518]: pgmap v542: 129 pgs: 129 active+clean; 371 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 899 KiB/s wr, 215 op/s 2023-11-27T15:31:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:55 smithi008 ceph-mon[128788]: pgmap v542: 129 pgs: 129 active+clean; 371 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 899 KiB/s wr, 215 op/s 2023-11-27T15:31:55.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:55 smithi029 ceph-mon[131166]: pgmap v542: 129 pgs: 129 active+clean; 371 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 899 KiB/s wr, 215 op/s 2023-11-27T15:31:57.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:57 smithi060 ceph-mon[131518]: pgmap v543: 129 pgs: 129 active+clean; 368 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 962 KiB/s wr, 179 op/s 2023-11-27T15:31:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:57 smithi008 ceph-mon[128788]: pgmap v543: 129 pgs: 129 active+clean; 368 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 962 KiB/s wr, 179 op/s 2023-11-27T15:31:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:57 smithi029 ceph-mon[131166]: pgmap v543: 129 pgs: 129 active+clean; 368 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 962 KiB/s wr, 179 op/s 2023-11-27T15:31:59.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:31:59 smithi060 ceph-mon[131518]: pgmap v544: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 619 KiB/s wr, 131 op/s 2023-11-27T15:31:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:31:59 smithi008 ceph-mon[128788]: pgmap v544: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 619 KiB/s wr, 131 op/s 2023-11-27T15:31:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:31:59 smithi029 ceph-mon[131166]: pgmap v544: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 619 KiB/s wr, 131 op/s 2023-11-27T15:32:01.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:01 smithi060 ceph-mon[131518]: pgmap v545: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 511 KiB/s wr, 67 op/s 2023-11-27T15:32:01.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:01 smithi008 ceph-mon[128788]: pgmap v545: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 511 KiB/s wr, 67 op/s 2023-11-27T15:32:01.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:01 smithi029 ceph-mon[131166]: pgmap v545: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 511 KiB/s wr, 67 op/s 2023-11-27T15:32:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:03 smithi060 ceph-mon[131518]: pgmap v546: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 235 KiB/s wr, 45 op/s 2023-11-27T15:32:03.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:03 smithi008 ceph-mon[128788]: pgmap v546: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 235 KiB/s wr, 45 op/s 2023-11-27T15:32:03.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:03 smithi029 ceph-mon[131166]: pgmap v546: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 235 KiB/s wr, 45 op/s 2023-11-27T15:32:05.752 INFO:tasks.workunit.client.0.smithi008.stdout:ok 2023-11-27T15:32:05.758 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:32:05.758 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:32:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:05 smithi060 ceph-mon[131518]: pgmap v547: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 224 KiB/s wr, 36 op/s 2023-11-27T15:32:05.825 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:05 smithi008 ceph-mon[128788]: pgmap v547: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 224 KiB/s wr, 36 op/s 2023-11-27T15:32:05.853 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2023-11-27T15:32:05.853 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2023-11-27T15:32:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:05 smithi029 ceph-mon[131166]: pgmap v547: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 224 KiB/s wr, 36 op/s 2023-11-27T15:32:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.257+0000 7f95dbdff700 1 -- 172.21.15.8:0/1917749827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d41014c0 msgr2=0x7f95d4101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.257+0000 7f95dbdff700 1 --2- 172.21.15.8:0/1917749827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d41014c0 0x7f95d4101930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009990 tx=0x7f95c4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.257+0000 7f95dbdff700 1 -- 172.21.15.8:0/1917749827 shutdown_connections 2023-11-27T15:32:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.257+0000 7f95dbdff700 1 --2- 172.21.15.8:0/1917749827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95d4101e70 0x7f95d4108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.257+0000 7f95dbdff700 1 --2- 172.21.15.8:0/1917749827 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95d4100230 0x7f95d4100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.257+0000 7f95dbdff700 1 --2- 172.21.15.8:0/1917749827 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d41014c0 0x7f95d4101930 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.257+0000 7f95dbdff700 1 -- 172.21.15.8:0/1917749827 >> 172.21.15.8:0/1917749827 conn(0x7f95d40fb810 msgr2=0x7f95d40fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.257+0000 7f95dbdff700 1 -- 172.21.15.8:0/1917749827 shutdown_connections 2023-11-27T15:32:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.257+0000 7f95dbdff700 1 -- 172.21.15.8:0/1917749827 wait complete. 2023-11-27T15:32:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.258+0000 7f95dbdff700 1 Processor -- start 2023-11-27T15:32:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.259+0000 7f95dbdff700 1 -- start start 2023-11-27T15:32:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.259+0000 7f95dbdff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d4100230 0x7f95d41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.259+0000 7f95dbdff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95d41014c0 0x7f95d4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.259+0000 7f95dbdff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95d4101e70 0x7f95d419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.259+0000 7f95dbdff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d419fd70 con 0x7f95d4101e70 2023-11-27T15:32:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.259+0000 7f95dbdff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d419feb0 con 0x7f95d41014c0 2023-11-27T15:32:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.259+0000 7f95dbdff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d41a0040 con 0x7f95d4100230 2023-11-27T15:32:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95d9b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d4100230 0x7f95d41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95da39c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95d4101e70 0x7f95d419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95d939a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95d41014c0 0x7f95d4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95d939a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95d41014c0 0x7f95d4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:34638/0 (socket says 172.21.15.8:34638) 2023-11-27T15:32:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95d939a700 1 -- 172.21.15.8:0/3549765283 learned_addr learned my addr 172.21.15.8:0/3549765283 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95d9b9b700 1 -- 172.21.15.8:0/3549765283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95d41014c0 msgr2=0x7f95d4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95d9b9b700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95d41014c0 0x7f95d4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95d9b9b700 1 -- 172.21.15.8:0/3549765283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95d4101e70 msgr2=0x7f95d419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95d9b9b700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95d4101e70 0x7f95d419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95d9b9b700 1 -- 172.21.15.8:0/3549765283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc009c40 con 0x7f95d4100230 2023-11-27T15:32:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95da39c700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95d4101e70 0x7f95d419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:32:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.260+0000 7f95d9b9b700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d4100230 0x7f95d41981f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f95d000e9f0 tx=0x7f95d000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.261+0000 7f95caffd700 1 -- 172.21.15.8:0/3549765283 <== mon.1 v2:172.21.15.60:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000c9c0 con 0x7f95d4100230 2023-11-27T15:32:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.261+0000 7f95caffd700 1 -- 172.21.15.8:0/3549765283 <== mon.1 v2:172.21.15.60:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95d0010430 con 0x7f95d4100230 2023-11-27T15:32:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.261+0000 7f95dbdff700 1 -- 172.21.15.8:0/3549765283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c40095f0 con 0x7f95d4100230 2023-11-27T15:32:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.261+0000 7f95caffd700 1 -- 172.21.15.8:0/3549765283 <== mon.1 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0018680 con 0x7f95d4100230 2023-11-27T15:32:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.261+0000 7f95dbdff700 1 -- 172.21.15.8:0/3549765283 --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d410bc80 con 0x7f95d4100230 2023-11-27T15:32:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.266+0000 7f95caffd700 1 -- 172.21.15.8:0/3549765283 <== mon.1 v2:172.21.15.60:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f95d0010860 con 0x7f95d4100230 2023-11-27T15:32:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.266+0000 7f95ef7b6b80 1 Processor -- start 2023-11-27T15:32:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95ef7b6b80 1 -- start start 2023-11-27T15:32:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126cfeb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cfec5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 0x56126cfecf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126cef1000 con 0x56126cfebdb0 2023-11-27T15:32:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126cf43840 con 0x56126cfecb10 2023-11-27T15:32:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126cf3ccd0 con 0x56126ce476b0 2023-11-27T15:32:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95d9b9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cfec5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95d9b9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cfec5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43032/0 (socket says 172.21.15.8:43032) 2023-11-27T15:32:07.272 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95d939a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 0x56126cfecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95d9b9b700 1 -- 172.21.15.8:0/1873747607 learned_addr learned my addr 172.21.15.8:0/1873747607 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95da39c700 1 --2- 172.21.15.8:0/1873747607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126cfeb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.267+0000 7f95caffd700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95c00a11d0 0x7f95c00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.268+0000 7f95d939a700 1 -- 172.21.15.8:0/1873747607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 msgr2=0x56126cfeb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.268+0000 7f95d939a700 1 --2- 172.21.15.8:0/1873747607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126cfeb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.268+0000 7f95d939a700 1 -- 172.21.15.8:0/1873747607 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 msgr2=0x56126cfec5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.268+0000 7f95d939a700 1 --2- 172.21.15.8:0/1873747607 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cfec5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.268+0000 7f95caffd700 1 -- 172.21.15.8:0/3549765283 <== mon.1 v2:172.21.15.60:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f95d0014070 con 0x7f95d4100230 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.268+0000 7f95d939a700 1 -- 172.21.15.8:0/1873747607 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56126cdbe5d0 con 0x56126cfecb10 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.268+0000 7f95da39c700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95c00a11d0 0x7f95c00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.268+0000 7f95d939a700 1 --2- 172.21.15.8:0/1873747607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 0x56126cfecf80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c4005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c70d74136c4ce4ce server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.268+0000 7f95da39c700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95c00a11d0 0x7f95c00a3680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009fa0 tx=0x7f95cc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.268+0000 7f95bffff700 1 -- 172.21.15.8:0/1873747607 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c401d070 con 0x56126cfecb10 2023-11-27T15:32:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95bffff700 1 -- 172.21.15.8:0/1873747607 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c401e040 con 0x56126cfecb10 2023-11-27T15:32:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95bffff700 1 -- 172.21.15.8:0/1873747607 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40049c0 con 0x56126cfecb10 2023-11-27T15:32:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/1873747607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 msgr2=0x56126cfecf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/1873747607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 0x56126cfecf80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95c4000c00 tx=0x7f95c4005ea0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/1873747607 shutdown_connections 2023-11-27T15:32:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/1873747607 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126cfeb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/1873747607 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 0x56126cfecf80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/1873747607 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cfec5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/1873747607 >> 172.21.15.8:0/1873747607 conn(0x56126cfe0f50 msgr2=0x56126cfe6100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/1873747607 shutdown_connections 2023-11-27T15:32:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.269+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/1873747607 wait complete. 2023-11-27T15:32:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.270+0000 7f95ef7b6b80 1 -- start start 2023-11-27T15:32:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.270+0000 7f95ef7b6b80 1 Processor -- start 2023-11-27T15:32:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.270+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126cff4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfebdb0 0x56126cff5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 0x56126cffc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126cede880 con 0x56126cfecb10 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126cede520 con 0x56126cfebdb0 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126cdc08c0 con 0x56126ce476b0 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95ef7b6b80 10 client.0 _ll_register_callbacks cb 0x56126cefc720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95d9b9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126cff4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95d939a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfebdb0 0x56126cff5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95da39c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 0x56126cffc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95d939a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfebdb0 0x56126cff5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:34644/0 (socket says 172.21.15.8:34644) 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95d939a700 1 -- 172.21.15.8:0/2073025378 learned_addr learned my addr 172.21.15.8:0/2073025378 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95d939a700 1 -- 172.21.15.8:0/2073025378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 msgr2=0x56126cff4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95d939a700 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126cff4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95d939a700 1 -- 172.21.15.8:0/2073025378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 msgr2=0x56126cffc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95d939a700 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 0x56126cffc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95d939a700 1 -- 172.21.15.8:0/2073025378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d4108f80 con 0x56126cfebdb0 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.271+0000 7f95d9b9b700 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126cff4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.272+0000 7f95d939a700 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfebdb0 0x56126cff5140 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009ce0 tx=0x7f95c4022b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49882a626f0df9ff server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.272+0000 7f95bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.272+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2073025378 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c401d070 con 0x56126cfebdb0 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.272+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2073025378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56126cfde710 con 0x56126cfebdb0 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.272+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2073025378 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f95c400f460 con 0x56126cfebdb0 2023-11-27T15:32:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.272+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2073025378 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95c4005680 con 0x56126cfebdb0 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.272+0000 7f95ef7b6b80 10 client.25832 fetch_fsmap learned FSMap version 38 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.272+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2073025378 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56126cf03130 con 0x56126cfebdb0 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.273+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2073025378 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56126cfde710 con 0x56126cfebdb0 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.273+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2073025378 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f95c402c520 con 0x56126cfebdb0 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.273+0000 7f95ef7b6b80 10 client.25832 fetch_fsmap finished waiting for FSMap version 38 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.273+0000 7f95ef7b6b80 10 client.25832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.273+0000 7f95ef7b6b80 20 client.25832 populate_metadata read hostname 'smithi008' 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.273+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56126d000710 0x56126d002b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.273+0000 7f95ef7b6b80 4 client.25832 mds_command: new command op to 24595 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.273+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2073025378 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56126ceff040 con 0x56126d000710 2023-11-27T15:32:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.273+0000 7f95d9b9b700 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56126d000710 0x56126d002b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.274+0000 7f95d9b9b700 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56126d000710 0x56126d002b20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76fffa3a630c06b8 server_cookie=9f05b03e60d75c3a in_seq=0 out_seq=0 2023-11-27T15:32:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.274+0000 7f95bd7fa700 10 client.25832 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:32:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2073025378 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56126ceff040 con 0x56126d000710 2023-11-27T15:32:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95bd7fa700 10 client.25832 handle_command_reply: tid=0 2023-11-27T15:32:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 client.25832 shutdown 2023-11-27T15:32:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2073025378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfebdb0 msgr2=0x56126cff5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfebdb0 0x56126cff5140 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009ce0 tx=0x7f95c4022b10 comp rx=0 tx=0).stop 2023-11-27T15:32:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2073025378 shutdown_connections 2023-11-27T15:32:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56126d000710 0x56126d002b20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126cff4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 0x56126cffc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95bd7fa700 0 client.25832 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:32:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2073025378 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfebdb0 0x56126cff5140 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2073025378 >> 172.21.15.8:0/2073025378 conn(0x56126cfe0f50 msgr2=0x56126cfe57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2073025378 shutdown_connections 2023-11-27T15:32:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2073025378 wait complete. 2023-11-27T15:32:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.275+0000 7f95ef7b6b80 20 client.25832 trim_cache size 0 max 16384 2023-11-27T15:32:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95ef7b6b80 1 Processor -- start 2023-11-27T15:32:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95ef7b6b80 1 -- start start 2023-11-27T15:32:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126d007480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cffc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 0x56126cfef980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126cff4770 con 0x56126cfebdb0 2023-11-27T15:32:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126cfde250 con 0x56126cfecb10 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126cfe7930 con 0x56126ce476b0 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95d939a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cffc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95d939a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cffc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43046/0 (socket says 172.21.15.8:43046) 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95d939a700 1 -- 172.21.15.8:0/276034577 learned_addr learned my addr 172.21.15.8:0/276034577 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95d9b9b700 1 --2- 172.21.15.8:0/276034577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126d007480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.291+0000 7f95da39c700 1 --2- 172.21.15.8:0/276034577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 0x56126cfef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.292+0000 7f95d939a700 1 -- 172.21.15.8:0/276034577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 msgr2=0x56126d007480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.292+0000 7f95d939a700 1 --2- 172.21.15.8:0/276034577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126d007480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.292+0000 7f95d939a700 1 -- 172.21.15.8:0/276034577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 msgr2=0x56126cfef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.292+0000 7f95d939a700 1 --2- 172.21.15.8:0/276034577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 0x56126cfef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.292+0000 7f95d939a700 1 -- 172.21.15.8:0/276034577 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56126cf02e80 con 0x56126cfebdb0 2023-11-27T15:32:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.292+0000 7f95d939a700 1 --2- 172.21.15.8:0/276034577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cffc140 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f95c402cad0 tx=0x7f95c4005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ef6cd53de97af3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.292+0000 7f95c8bff700 1 -- 172.21.15.8:0/276034577 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c401d040 con 0x56126cfebdb0 2023-11-27T15:32:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.292+0000 7f95c8bff700 1 -- 172.21.15.8:0/276034577 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c400bde0 con 0x56126cfebdb0 2023-11-27T15:32:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.292+0000 7f95c8bff700 1 -- 172.21.15.8:0/276034577 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c402be30 con 0x56126cfebdb0 2023-11-27T15:32:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.293+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/276034577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 msgr2=0x56126cffc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.293+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/276034577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cffc140 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f95c402cad0 tx=0x7f95c4005a80 comp rx=0 tx=0).stop 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.293+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/276034577 shutdown_connections 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.293+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/276034577 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126ce476b0 0x56126d007480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.293+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/276034577 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126cfecb10 0x56126cfef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.293+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/276034577 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfebdb0 0x56126cffc140 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.293+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/276034577 >> 172.21.15.8:0/276034577 conn(0x56126cfe0f50 msgr2=0x56126cfe60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.293+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/276034577 shutdown_connections 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.293+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/276034577 wait complete. 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.293+0000 7f95ef7b6b80 1 -- start start 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95ef7b6b80 1 Processor -- start 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126ce476b0 0x56126cffa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126cfebdb0 0x56126d0505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.301 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95ef7b6b80 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 0x56126d052b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d053290 con 0x56126cfecb10 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d053400 con 0x56126ce476b0 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95ef7b6b80 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x56126d0535c0 con 0x56126cfebdb0 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95ef7b6b80 10 client.0 _ll_register_callbacks cb 0x7f95d0096360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95da39c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 0x56126d052b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95d9b9b700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126ce476b0 0x56126cffa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95da39c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 0x56126d052b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43062/0 (socket says 172.21.15.8:43062) 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.294+0000 7f95da39c700 1 -- 172.21.15.8:0/2627707316 learned_addr learned my addr 172.21.15.8:0/2627707316 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95d939a700 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126cfebdb0 0x56126d0505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95d9b9b700 1 -- 172.21.15.8:0/2627707316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126cfebdb0 msgr2=0x56126d0505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95d9b9b700 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126cfebdb0 0x56126d0505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95d9b9b700 1 -- 172.21.15.8:0/2627707316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 msgr2=0x56126d052b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95d9b9b700 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 0x56126d052b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95d9b9b700 1 -- 172.21.15.8:0/2627707316 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56126cf03130 con 0x56126ce476b0 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95d9b9b700 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126ce476b0 0x56126cffa090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f95d000e9c0 tx=0x7f95d0010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8071cbdbdc388f3e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.29:3300/0 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2627707316 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0099fa0 con 0x56126ce476b0 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2627707316 <== mon.2 v2:172.21.15.29:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f95d009dfd0 con 0x56126ce476b0 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2627707316 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95c401e040 con 0x56126ce476b0 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2627707316 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95d00a23f0 con 0x56126ce476b0 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.295+0000 7f95ef7b6b80 10 client.25838 fetch_fsmap learned FSMap version 38 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.296+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2627707316 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56126cd53c00 con 0x56126ce476b0 2023-11-27T15:32:07.303 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.296+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2627707316 <== mon.2 v2:172.21.15.29:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c401e040 con 0x56126ce476b0 2023-11-27T15:32:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.296+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2627707316 <== mon.2 v2:172.21.15.29:3300/0 5 ==== fsmap(e 38) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f95d00a1f60 con 0x56126ce476b0 2023-11-27T15:32:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.296+0000 7f95ef7b6b80 10 client.25838 fetch_fsmap finished waiting for FSMap version 38 2023-11-27T15:32:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.296+0000 7f95ef7b6b80 10 client.25838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.a 2023-11-27T15:32:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.296+0000 7f95ef7b6b80 20 client.25838 populate_metadata read hostname 'smithi008' 2023-11-27T15:32:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.296+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56126d054fe0 0x56126d0573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.296+0000 7f95ef7b6b80 4 client.25838 mds_command: new command op to 24595 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:32:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.296+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2627707316 --> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56126cf0b600 con 0x56126d054fe0 2023-11-27T15:32:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.296+0000 7f95d939a700 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56126d054fe0 0x56126d0573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.297+0000 7f95d939a700 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56126d054fe0 0x56126d0573f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c43beb03655570 server_cookie=15953503174233a8 in_seq=0 out_seq=0 2023-11-27T15:32:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.297+0000 7f95bd7fa700 10 client.25838 ms_handle_connect on v2:172.21.15.8:6840/110161953 2023-11-27T15:32:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95bd7fa700 1 -- 172.21.15.8:0/2627707316 <== mds.0 v2:172.21.15.8:6840/110161953 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56126cf0b600 con 0x56126d054fe0 2023-11-27T15:32:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95bd7fa700 10 client.25838 handle_command_reply: tid=0 2023-11-27T15:32:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95ef7b6b80 1 client.25838 shutdown 2023-11-27T15:32:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2627707316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126ce476b0 msgr2=0x56126cffa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126ce476b0 0x56126cffa090 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f95d000e9c0 tx=0x7f95d0010c60 comp rx=0 tx=0).stop 2023-11-27T15:32:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2627707316 shutdown_connections 2023-11-27T15:32:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x56126cfecb10 0x56126d052b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x56126ce476b0 0x56126cffa090 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.8:6840/110161953,v1:172.21.15.8:6841/110161953] conn(0x56126d054fe0 0x56126d0573f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95ef7b6b80 1 --2- 172.21.15.8:0/2627707316 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x56126cfebdb0 0x56126d0505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95bd7fa700 0 client.25838 ms_handle_reset on v2:172.21.15.8:6840/110161953 2023-11-27T15:32:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2627707316 >> 172.21.15.8:0/2627707316 conn(0x56126cfe0f50 msgr2=0x56126cfef980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.320+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2627707316 shutdown_connections 2023-11-27T15:32:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95ef7b6b80 1 -- 172.21.15.8:0/2627707316 wait complete. 2023-11-27T15:32:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95ef7b6b80 20 client.25838 trim_cache size 0 max 16384 2023-11-27T15:32:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:{ 2023-11-27T15:32:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "message": "", 2023-11-27T15:32:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout: "return_code": 0 2023-11-27T15:32:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stdout:} 2023-11-27T15:32:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95dbdff700 1 -- 172.21.15.8:0/3549765283 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95c00a11d0 msgr2=0x7f95c00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95dbdff700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95c00a11d0 0x7f95c00a3680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009fa0 tx=0x7f95cc009040 comp rx=0 tx=0).stop 2023-11-27T15:32:07.326 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95dbdff700 1 -- 172.21.15.8:0/3549765283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d4100230 msgr2=0x7f95d41981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95dbdff700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d4100230 0x7f95d41981f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f95d000e9f0 tx=0x7f95d000ed00 comp rx=0 tx=0).stop 2023-11-27T15:32:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95dbdff700 1 -- 172.21.15.8:0/3549765283 shutdown_connections 2023-11-27T15:32:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95dbdff700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f95c00a11d0 0x7f95c00a3680 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95dbdff700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f95d4100230 0x7f95d41981f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95dbdff700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95d4101e70 0x7f95d419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95dbdff700 1 --2- 172.21.15.8:0/3549765283 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f95d41014c0 0x7f95d4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.321+0000 7f95dbdff700 1 -- 172.21.15.8:0/3549765283 >> 172.21.15.8:0/3549765283 conn(0x7f95d40fb810 msgr2=0x7f95d4106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.322+0000 7f95dbdff700 1 -- 172.21.15.8:0/3549765283 shutdown_connections 2023-11-27T15:32:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi008.front.sepia.ceph.com.smithi008.stderr:2023-11-27T15:32:07.322+0000 7f95dbdff700 1 -- 172.21.15.8:0/3549765283 wait complete. 2023-11-27T15:32:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:07 smithi060 ceph-mon[131518]: pgmap v548: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 31 op/s 2023-11-27T15:32:07.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:07 smithi008 ceph-mon[128788]: pgmap v548: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 31 op/s 2023-11-27T15:32:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:07 smithi029 ceph-mon[131166]: pgmap v548: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 171 KiB/s wr, 31 op/s 2023-11-27T15:32:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:09 smithi060 ceph-mon[131518]: pgmap v549: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 96 KiB/s wr, 19 op/s 2023-11-27T15:32:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:09 smithi008 ceph-mon[128788]: pgmap v549: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 96 KiB/s wr, 19 op/s 2023-11-27T15:32:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:09 smithi029 ceph-mon[131166]: pgmap v549: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 96 KiB/s wr, 19 op/s 2023-11-27T15:32:11.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:11 smithi060 ceph-mon[131518]: pgmap v550: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-11-27T15:32:11.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:11 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:32:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:11 smithi008 ceph-mon[128788]: pgmap v550: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-11-27T15:32:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:11 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:32:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:11 smithi029 ceph-mon[131166]: pgmap v550: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.9 KiB/s wr, 12 op/s 2023-11-27T15:32:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:11 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:32:12.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:32:12.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:32:12.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:32:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:32:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:32:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:32:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:32:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:32:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:32:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:32:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:13 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi008 to 223.0M 2023-11-27T15:32:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:13 smithi060 ceph-mon[131518]: Unable to set osd_memory_target on smithi008 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-11-27T15:32:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:13 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:32:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:13 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:32:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:13 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:32:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:13 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:32:13.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:13 smithi060 ceph-mon[131518]: pgmap v551: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.7 KiB/s wr, 14 op/s 2023-11-27T15:32:13.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:13 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi008 to 223.0M 2023-11-27T15:32:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:13 smithi008 ceph-mon[128788]: Unable to set osd_memory_target on smithi008 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-11-27T15:32:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:13 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:32:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:13 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:32:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:13 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:32:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:13 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:32:13.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:13 smithi008 ceph-mon[128788]: pgmap v551: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.7 KiB/s wr, 14 op/s 2023-11-27T15:32:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:13 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi008 to 223.0M 2023-11-27T15:32:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:13 smithi029 ceph-mon[131166]: Unable to set osd_memory_target on smithi008 to 233877504: error parsing value: Value '233877504' is below minimum 939524096 2023-11-27T15:32:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:13 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:32:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:13 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:32:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:13 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:32:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:13 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:32:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:13 smithi029 ceph-mon[131166]: pgmap v551: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.7 KiB/s wr, 14 op/s 2023-11-27T15:32:15.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:15 smithi060 ceph-mon[131518]: pgmap v552: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2023-11-27T15:32:15.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:15 smithi008 ceph-mon[128788]: pgmap v552: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2023-11-27T15:32:15.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:15 smithi029 ceph-mon[131166]: pgmap v552: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2023-11-27T15:32:15.945 INFO:tasks.workunit.client.0.smithi008.stdout:ok 2023-11-27T15:32:15.950 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:32:15.950 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:32:16.042 INFO:tasks.workunit:Running workunit fs/misc/general_vxattrs.sh... 2023-11-27T15:32:16.042 DEBUG:teuthology.orchestra.run.smithi008:workunit test fs/misc/general_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh 2023-11-27T15:32:16.118 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir -p dir 2023-11-27T15:32:16.123 INFO:tasks.workunit.client.0.smithi008.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-27T15:32:16.123 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-27T15:32:16.125 INFO:tasks.workunit.client.0.smithi008.stdout:ceph.dir.pin="-1" 2023-11-27T15:32:16.126 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -n ceph.dir.pin dir 2023-11-27T15:32:16.126 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:32:16.128 INFO:tasks.workunit.client.0.smithi008.stdout:setfattr: dir: Invalid argument 2023-11-27T15:32:16.129 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -n ceph.dir.pin -v 1 dir 2023-11-27T15:32:16.132 INFO:tasks.workunit.client.0.smithi008.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-27T15:32:16.132 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep 'ceph.dir.pin="1"' 2023-11-27T15:32:16.134 INFO:tasks.workunit.client.0.smithi008.stdout:ceph.dir.pin="1" 2023-11-27T15:32:16.134 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -x ceph.dir.pin dir 2023-11-27T15:32:16.137 INFO:tasks.workunit.client.0.smithi008.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-27T15:32:16.138 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-27T15:32:16.140 INFO:tasks.workunit.client.0.smithi008.stdout:ceph.dir.pin="-1" 2023-11-27T15:32:16.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -n ceph.dir.pin.distributed dir 2023-11-27T15:32:16.141 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:32:16.143 INFO:tasks.workunit.client.0.smithi008.stdout:setfattr: dir: Invalid argument 2023-11-27T15:32:16.143 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 dir 2023-11-27T15:32:16.145 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -x ceph.dir.pin.distributed dir 2023-11-27T15:32:16.149 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -n ceph.dir.pin.random dir 2023-11-27T15:32:16.149 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:32:16.151 INFO:tasks.workunit.client.0.smithi008.stdout:setfattr: dir: Invalid argument 2023-11-27T15:32:16.151 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -n ceph.dir.pin.random -v 0.01 dir 2023-11-27T15:32:16.154 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -x ceph.dir.pin.random dir 2023-11-27T15:32:16.157 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -n ceph.dir.subvolume dir 2023-11-27T15:32:16.157 INFO:tasks.workunit.client.0.smithi008.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:32:16.159 INFO:tasks.workunit.client.0.smithi008.stdout:setfattr: dir: Invalid argument 2023-11-27T15:32:16.159 INFO:tasks.workunit.client.0.smithi008.stderr:+ setfattr -n ceph.dir.subvolume -v 1 dir 2023-11-27T15:32:16.167 INFO:tasks.workunit.client.0.smithi008.stderr:setfattr: dir: Invalid argument 2023-11-27T15:32:16.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:32:16.168 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2023-11-27T15:32:16.168 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-11-27T15:32:17.622 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-27T15:32:17.689 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=622b1e18819f4d049812695aab7c60e3 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi008 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-27T15:32:17.692 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-11-27T15:32:17.732 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi008.front.sepia.ceph.com 2023-11-27T15:32:17.746 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-11-27T15:32:17.755 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:32:17.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:17 smithi060 ceph-mon[131518]: pgmap v553: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2023-11-27T15:32:17.847 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:17 smithi008 ceph-mon[128788]: pgmap v553: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2023-11-27T15:32:17.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:17 smithi029 ceph-mon[131166]: pgmap v553: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2023-11-27T15:32:17.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.988+0000 7f16d2fff700 1 -- 172.21.15.8:0/2108685164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc1014c0 msgr2=0x7f16cc101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:17.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.988+0000 7f16d2fff700 1 --2- 172.21.15.8:0/2108685164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc1014c0 0x7f16cc101910 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009990 tx=0x7f16b8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:32:17.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.988+0000 7f16d2fff700 1 -- 172.21.15.8:0/2108685164 shutdown_connections 2023-11-27T15:32:17.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.988+0000 7f16d2fff700 1 --2- 172.21.15.8:0/2108685164 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16cc1002d0 0x7f16cc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:17.992 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.988+0000 7f16d2fff700 1 --2- 172.21.15.8:0/2108685164 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16cc101e50 0x7f16cc108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:17.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.988+0000 7f16d2fff700 1 --2- 172.21.15.8:0/2108685164 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc1014c0 0x7f16cc101910 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:17.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.988+0000 7f16d2fff700 1 -- 172.21.15.8:0/2108685164 >> 172.21.15.8:0/2108685164 conn(0x7f16cc0fb8b0 msgr2=0x7f16cc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:17.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.988+0000 7f16d2fff700 1 -- 172.21.15.8:0/2108685164 shutdown_connections 2023-11-27T15:32:17.993 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.989+0000 7f16d2fff700 1 -- 172.21.15.8:0/2108685164 wait complete. 2023-11-27T15:32:17.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.990+0000 7f16d2fff700 1 Processor -- start 2023-11-27T15:32:17.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.990+0000 7f16d2fff700 1 -- start start 2023-11-27T15:32:17.995 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d2fff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16cc1002d0 0x7f16cc0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d2fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc1014c0 0x7f16cc072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d2fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16cc101e50 0x7f16cc073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d2fff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc073bc0 con 0x7f16cc1014c0 2023-11-27T15:32:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d2fff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc073d30 con 0x7f16cc101e50 2023-11-27T15:32:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d2fff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc1a0090 con 0x7f16cc1002d0 2023-11-27T15:32:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d0d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16cc1002d0 0x7f16cc0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d0d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16cc1002d0 0x7f16cc0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:41256/0 (socket says 172.21.15.8:41256) 2023-11-27T15:32:17.996 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d0d9b700 1 -- 172.21.15.8:0/469170521 learned_addr learned my addr 172.21.15.8:0/469170521 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16cbfff700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc1014c0 0x7f16cc072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d159c700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16cc101e50 0x7f16cc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16cbfff700 1 -- 172.21.15.8:0/469170521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16cc1002d0 msgr2=0x7f16cc0748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16cbfff700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16cc1002d0 0x7f16cc0748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16cbfff700 1 -- 172.21.15.8:0/469170521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16cc101e50 msgr2=0x7f16cc073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:17.997 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16cbfff700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16cc101e50 0x7f16cc073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16cbfff700 1 -- 172.21.15.8:0/469170521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b80095f0 con 0x7f16cc1014c0 2023-11-27T15:32:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d0d9b700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16cc1002d0 0x7f16cc0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:32:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.991+0000 7f16d159c700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16cc101e50 0x7f16cc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:32:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.992+0000 7f16cbfff700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc1014c0 0x7f16cc072ee0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098e0 tx=0x7f16b8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.992+0000 7f16c9ffb700 1 -- 172.21.15.8:0/469170521 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b801d070 con 0x7f16cc1014c0 2023-11-27T15:32:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.992+0000 7f16c9ffb700 1 -- 172.21.15.8:0/469170521 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16b8021410 con 0x7f16cc1014c0 2023-11-27T15:32:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.992+0000 7f16d2fff700 1 -- 172.21.15.8:0/469170521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc10bc80 con 0x7f16cc1014c0 2023-11-27T15:32:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.992+0000 7f16c9ffb700 1 -- 172.21.15.8:0/469170521 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8017650 con 0x7f16cc1014c0 2023-11-27T15:32:17.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.992+0000 7f16d2fff700 1 -- 172.21.15.8:0/469170521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc10baf0 con 0x7f16cc1014c0 2023-11-27T15:32:17.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.993+0000 7f16d2fff700 1 -- 172.21.15.8:0/469170521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16cc1a0630 con 0x7f16cc1014c0 2023-11-27T15:32:17.999 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.995+0000 7f16c9ffb700 1 -- 172.21.15.8:0/469170521 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f16b8021580 con 0x7f16cc1014c0 2023-11-27T15:32:18.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.997+0000 7f16c9ffb700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f16bc0a1150 0x7f16bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:18.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.997+0000 7f16c9ffb700 1 -- 172.21.15.8:0/469170521 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f16b80cdfe0 con 0x7f16cc1014c0 2023-11-27T15:32:18.000 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.997+0000 7f16d0d9b700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f16bc0a1150 0x7f16bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:18.001 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:17.997+0000 7f16d0d9b700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f16bc0a1150 0x7f16bc0a3600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f16c0005950 tx=0x7f16c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:18.004 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.001+0000 7f16c9ffb700 1 -- 172.21.15.8:0/469170521 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f16b8097450 con 0x7f16cc1014c0 2023-11-27T15:32:18.218 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.214+0000 7f16d2fff700 1 -- 172.21.15.8:0/469170521 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16cc107160 con 0x7f16cc1014c0 2023-11-27T15:32:18.219 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.216+0000 7f16c9ffb700 1 -- 172.21.15.8:0/469170521 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24392 (secure 0 0 0) 0x7f16b809aa70 con 0x7f16cc1014c0 2023-11-27T15:32:18.220 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:32:18.220 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":94,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:30:30.970908+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:32:18.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.218+0000 7f16d2fff700 1 -- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f16bc0a1150 msgr2=0x7f16bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:18.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.218+0000 7f16d2fff700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f16bc0a1150 0x7f16bc0a3600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f16c0005950 tx=0x7f16c0009260 comp rx=0 tx=0).stop 2023-11-27T15:32:18.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.218+0000 7f16d2fff700 1 -- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc1014c0 msgr2=0x7f16cc072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:18.222 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.218+0000 7f16d2fff700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc1014c0 0x7f16cc072ee0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098e0 tx=0x7f16b8005740 comp rx=0 tx=0).stop 2023-11-27T15:32:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.219+0000 7f16d2fff700 1 -- 172.21.15.8:0/469170521 shutdown_connections 2023-11-27T15:32:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.219+0000 7f16d2fff700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f16bc0a1150 0x7f16bc0a3600 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.219+0000 7f16d2fff700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f16cc1002d0 0x7f16cc0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.219+0000 7f16d2fff700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f16cc101e50 0x7f16cc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.219+0000 7f16d2fff700 1 --2- 172.21.15.8:0/469170521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f16cc1014c0 0x7f16cc072ee0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.219+0000 7f16d2fff700 1 -- 172.21.15.8:0/469170521 >> 172.21.15.8:0/469170521 conn(0x7f16cc0fb8b0 msgr2=0x7f16cc0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.219+0000 7f16d2fff700 1 -- 172.21.15.8:0/469170521 shutdown_connections 2023-11-27T15:32:18.224 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.219+0000 7f16d2fff700 1 -- 172.21.15.8:0/469170521 wait complete. 2023-11-27T15:32:18.241 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:23:25.166148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:23:26.584539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:23:30.130965+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:32:18.242 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:32:18.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.446+0000 7fc90ebff700 1 -- 172.21.15.8:0/804205898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc908101480 msgr2=0x7fc9081018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:18.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.446+0000 7fc90ebff700 1 --2- 172.21.15.8:0/804205898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc908101480 0x7fc9081018d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009990 tx=0x7fc8f4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.450 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.446+0000 7fc90ebff700 1 -- 172.21.15.8:0/804205898 shutdown_connections 2023-11-27T15:32:18.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.446+0000 7fc90ebff700 1 --2- 172.21.15.8:0/804205898 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc908100290 0x7fc9081006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.446+0000 7fc90ebff700 1 --2- 172.21.15.8:0/804205898 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908101e10 0x7fc9081086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.446+0000 7fc90ebff700 1 --2- 172.21.15.8:0/804205898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc908101480 0x7fc9081018d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.446+0000 7fc90ebff700 1 -- 172.21.15.8:0/804205898 >> 172.21.15.8:0/804205898 conn(0x7fc9080fb870 msgr2=0x7fc9080fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:18.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.447+0000 7fc90ebff700 1 -- 172.21.15.8:0/804205898 shutdown_connections 2023-11-27T15:32:18.451 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.447+0000 7fc90ebff700 1 -- 172.21.15.8:0/804205898 wait complete. 2023-11-27T15:32:18.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.448+0000 7fc90ebff700 1 Processor -- start 2023-11-27T15:32:18.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.448+0000 7fc90ebff700 1 -- start start 2023-11-27T15:32:18.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90ebff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc908100290 0x7fc9081981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:18.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90ebff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908101480 0x7fc908198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:18.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90ebff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc908101e10 0x7fc90819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:18.454 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90ebff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819fd60 con 0x7fc908100290 2023-11-27T15:32:18.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90ebff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819fea0 con 0x7fc908101480 2023-11-27T15:32:18.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90ebff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9081a0030 con 0x7fc908101e10 2023-11-27T15:32:18.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90c99b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc908100290 0x7fc9081981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:18.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90c99b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc908100290 0x7fc9081981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43790/0 (socket says 172.21.15.8:43790) 2023-11-27T15:32:18.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc907fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908101480 0x7fc908198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:18.455 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90c99b700 1 -- 172.21.15.8:0/2706603291 learned_addr learned my addr 172.21.15.8:0/2706603291 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:18.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90d19c700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc908101e10 0x7fc90819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:18.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90c99b700 1 -- 172.21.15.8:0/2706603291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc908101e10 msgr2=0x7fc90819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:18.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90c99b700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc908101e10 0x7fc90819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.449+0000 7fc90c99b700 1 -- 172.21.15.8:0/2706603291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908101480 msgr2=0x7fc908198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:18.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.450+0000 7fc90c99b700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908101480 0x7fc908198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.450+0000 7fc90c99b700 1 -- 172.21.15.8:0/2706603291 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f40095f0 con 0x7fc908100290 2023-11-27T15:32:18.456 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.450+0000 7fc90c99b700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc908100290 0x7fc9081981e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00e970 tx=0x7fc8fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.450+0000 7fc905ffb700 1 -- 172.21.15.8:0/2706603291 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc009790 con 0x7fc908100290 2023-11-27T15:32:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.450+0000 7fc905ffb700 1 -- 172.21.15.8:0/2706603291 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8fc00eec0 con 0x7fc908100290 2023-11-27T15:32:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.450+0000 7fc90ebff700 1 -- 172.21.15.8:0/2706603291 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90810bc50 con 0x7fc908100290 2023-11-27T15:32:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.450+0000 7fc905ffb700 1 -- 172.21.15.8:0/2706603291 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc003cf0 con 0x7fc908100290 2023-11-27T15:32:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.450+0000 7fc90ebff700 1 -- 172.21.15.8:0/2706603291 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90810bac0 con 0x7fc908100290 2023-11-27T15:32:18.458 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.451+0000 7fc90ebff700 1 -- 172.21.15.8:0/2706603291 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9081a0730 con 0x7fc908100290 2023-11-27T15:32:18.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.453+0000 7fc905ffb700 1 -- 172.21.15.8:0/2706603291 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc8fc00ccd0 con 0x7fc908100290 2023-11-27T15:32:18.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.454+0000 7fc905ffb700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8f80a1150 0x7fc8f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:18.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.455+0000 7fc905ffb700 1 -- 172.21.15.8:0/2706603291 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc8fc014070 con 0x7fc908100290 2023-11-27T15:32:18.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.455+0000 7fc907fff700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8f80a1150 0x7fc8f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:18.459 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.455+0000 7fc907fff700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8f80a1150 0x7fc8f80a3600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400b2e0 tx=0x7fc8f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:18.462 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.458+0000 7fc905ffb700 1 -- 172.21.15.8:0/2706603291 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc8fc0962b0 con 0x7fc908100290 2023-11-27T15:32:18.674 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.671+0000 7fc90ebff700 1 -- 172.21.15.8:0/2706603291 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9081a0ce0 con 0x7fc908100290 2023-11-27T15:32:18.675 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.671+0000 7fc905ffb700 1 -- 172.21.15.8:0/2706603291 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8fc0998d0 con 0x7fc908100290 2023-11-27T15:32:18.675 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:32:18.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.673+0000 7fc90ebff700 1 -- 172.21.15.8:0/2706603291 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8f80a1150 msgr2=0x7fc8f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:18.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8f80a1150 0x7fc8f80a3600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400b2e0 tx=0x7fc8f4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 -- 172.21.15.8:0/2706603291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc908100290 msgr2=0x7fc9081981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:18.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc908100290 0x7fc9081981e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00e970 tx=0x7fc8fc00ec80 comp rx=0 tx=0).stop 2023-11-27T15:32:18.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 -- 172.21.15.8:0/2706603291 shutdown_connections 2023-11-27T15:32:18.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc8f80a1150 0x7fc8f80a3600 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc908101e10 0x7fc90819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc908100290 0x7fc9081981e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 --2- 172.21.15.8:0/2706603291 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc908101480 0x7fc908198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 -- 172.21.15.8:0/2706603291 >> 172.21.15.8:0/2706603291 conn(0x7fc9080fb870 msgr2=0x7fc9080fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:18.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 -- 172.21.15.8:0/2706603291 shutdown_connections 2023-11-27T15:32:18.680 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.674+0000 7fc90ebff700 1 -- 172.21.15.8:0/2706603291 wait complete. 2023-11-27T15:32:18.696 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:32:18.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:18 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/469170521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:32:18.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:18 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/469170521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:32:18.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:18 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/469170521' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:32:18.904 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.901+0000 7fba7a3ff700 1 -- 172.21.15.8:0/3766845566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba74101480 msgr2=0x7fba741018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:18.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.901+0000 7fba7a3ff700 1 --2- 172.21.15.8:0/3766845566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba74101480 0x7fba741018d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fba680098f0 tx=0x7fba68009c00 comp rx=0 tx=0).stop 2023-11-27T15:32:18.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.901+0000 7fba7a3ff700 1 -- 172.21.15.8:0/3766845566 shutdown_connections 2023-11-27T15:32:18.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.901+0000 7fba7a3ff700 1 --2- 172.21.15.8:0/3766845566 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba74100290 0x7fba741006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.901+0000 7fba7a3ff700 1 --2- 172.21.15.8:0/3766845566 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fba74101e10 0x7fba741086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.901+0000 7fba7a3ff700 1 --2- 172.21.15.8:0/3766845566 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba74101480 0x7fba741018d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.905 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.901+0000 7fba7a3ff700 1 -- 172.21.15.8:0/3766845566 >> 172.21.15.8:0/3766845566 conn(0x7fba740fb870 msgr2=0x7fba740fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:18.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.901+0000 7fba7a3ff700 1 -- 172.21.15.8:0/3766845566 shutdown_connections 2023-11-27T15:32:18.906 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.901+0000 7fba7a3ff700 1 -- 172.21.15.8:0/3766845566 wait complete. 2023-11-27T15:32:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.902+0000 7fba7a3ff700 1 Processor -- start 2023-11-27T15:32:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.902+0000 7fba7a3ff700 1 -- start start 2023-11-27T15:32:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.902+0000 7fba7a3ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba74100290 0x7fba74196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.902+0000 7fba7a3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba74101480 0x7fba741965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7a3ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fba74101e10 0x7fba7419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:18.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7a3ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7419dbe0 con 0x7fba74101e10 2023-11-27T15:32:18.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7a3ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7419dd20 con 0x7fba74100290 2023-11-27T15:32:18.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7a3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7419deb0 con 0x7fba74101480 2023-11-27T15:32:18.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba73fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba74100290 0x7fba74196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:18.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba73fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba74100290 0x7fba74196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:58728/0 (socket says 172.21.15.8:58728) 2023-11-27T15:32:18.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba73fff700 1 -- 172.21.15.8:0/1050330901 learned_addr learned my addr 172.21.15.8:0/1050330901 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:18.909 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7899c700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fba74101e10 0x7fba7419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:18.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba737fe700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba74101480 0x7fba741965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:18.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7899c700 1 -- 172.21.15.8:0/1050330901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba74101480 msgr2=0x7fba741965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:18.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7899c700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba74101480 0x7fba741965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7899c700 1 -- 172.21.15.8:0/1050330901 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba74100290 msgr2=0x7fba74196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:18.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7899c700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba74100290 0x7fba74196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:18.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7899c700 1 -- 172.21.15.8:0/1050330901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba68009520 con 0x7fba74101e10 2023-11-27T15:32:18.910 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba73fff700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba74100290 0x7fba74196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:32:18.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba737fe700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba74101480 0x7fba741965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:32:18.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.903+0000 7fba7899c700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fba74101e10 0x7fba7419d610 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00efc0 tx=0x7fba6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:18.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.904+0000 7fba717fa700 1 -- 172.21.15.8:0/1050330901 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c018070 con 0x7fba74101e10 2023-11-27T15:32:18.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.904+0000 7fba717fa700 1 -- 172.21.15.8:0/1050330901 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba6c00f040 con 0x7fba74101e10 2023-11-27T15:32:18.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.904+0000 7fba7a3ff700 1 -- 172.21.15.8:0/1050330901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7410bc50 con 0x7fba74101e10 2023-11-27T15:32:18.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.904+0000 7fba717fa700 1 -- 172.21.15.8:0/1050330901 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00ccb0 con 0x7fba74101e10 2023-11-27T15:32:18.911 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.904+0000 7fba7a3ff700 1 -- 172.21.15.8:0/1050330901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7410bac0 con 0x7fba74101e10 2023-11-27T15:32:18.912 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.905+0000 7fba7a3ff700 1 -- 172.21.15.8:0/1050330901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba741072b0 con 0x7fba74101e10 2023-11-27T15:32:18.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.907+0000 7fba717fa700 1 -- 172.21.15.8:0/1050330901 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fba6c003a00 con 0x7fba74101e10 2023-11-27T15:32:18.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.908+0000 7fba717fa700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fba5c0a1150 0x7fba5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:18.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.908+0000 7fba717fa700 1 -- 172.21.15.8:0/1050330901 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fba6c0cc6b0 con 0x7fba74101e10 2023-11-27T15:32:18.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.909+0000 7fba73fff700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fba5c0a1150 0x7fba5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:18.913 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.909+0000 7fba73fff700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fba5c0a1150 0x7fba5c0a3600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fba64005950 tx=0x7fba64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:18.915 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:18.912+0000 7fba717fa700 1 -- 172.21.15.8:0/1050330901 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fba6c095b00 con 0x7fba74101e10 2023-11-27T15:32:19.129 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.126+0000 7fba7a3ff700 1 -- 172.21.15.8:0/1050330901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba740619a0 con 0x7fba74101e10 2023-11-27T15:32:19.130 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.126+0000 7fba717fa700 1 -- 172.21.15.8:0/1050330901 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba6c04f9f0 con 0x7fba74101e10 2023-11-27T15:32:19.130 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:32:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 -- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fba5c0a1150 msgr2=0x7fba5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fba5c0a1150 0x7fba5c0a3600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fba64005950 tx=0x7fba64009260 comp rx=0 tx=0).stop 2023-11-27T15:32:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 -- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fba74101e10 msgr2=0x7fba7419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:19.133 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fba74101e10 0x7fba7419d610 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00efc0 tx=0x7fba6c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 -- 172.21.15.8:0/1050330901 shutdown_connections 2023-11-27T15:32:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fba74101e10 0x7fba7419d610 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fba74100290 0x7fba74196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fba5c0a1150 0x7fba5c0a3600 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 --2- 172.21.15.8:0/1050330901 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fba74101480 0x7fba741965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.134 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 -- 172.21.15.8:0/1050330901 >> 172.21.15.8:0/1050330901 conn(0x7fba740fb870 msgr2=0x7fba740fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:19.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 -- 172.21.15.8:0/1050330901 shutdown_connections 2023-11-27T15:32:19.135 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.129+0000 7fba7a3ff700 1 -- 172.21.15.8:0/1050330901 wait complete. 2023-11-27T15:32:19.151 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:32:19.360 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.356+0000 7f618ebff700 1 -- 172.21.15.8:0/1747158377 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6188101490 msgr2=0x7f61881018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:19.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.356+0000 7f618ebff700 1 --2- 172.21.15.8:0/1747158377 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6188101490 0x7f61881018e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f61740098f0 tx=0x7f6174009c00 comp rx=0 tx=0).stop 2023-11-27T15:32:19.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.356+0000 7f618ebff700 1 -- 172.21.15.8:0/1747158377 shutdown_connections 2023-11-27T15:32:19.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.356+0000 7f618ebff700 1 --2- 172.21.15.8:0/1747158377 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188101e20 0x7f61881086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.356+0000 7f618ebff700 1 --2- 172.21.15.8:0/1747158377 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f61881002a0 0x7f61881006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.361 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.356+0000 7f618ebff700 1 --2- 172.21.15.8:0/1747158377 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6188101490 0x7f61881018e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.356+0000 7f618ebff700 1 -- 172.21.15.8:0/1747158377 >> 172.21.15.8:0/1747158377 conn(0x7f61880fb880 msgr2=0x7f61880fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:19.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.357+0000 7f618ebff700 1 -- 172.21.15.8:0/1747158377 shutdown_connections 2023-11-27T15:32:19.362 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.357+0000 7f618ebff700 1 -- 172.21.15.8:0/1747158377 wait complete. 2023-11-27T15:32:19.364 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618ebff700 1 Processor -- start 2023-11-27T15:32:19.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618ebff700 1 -- start start 2023-11-27T15:32:19.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618ebff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f61881002a0 0x7f61881982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:19.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618ebff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188101490 0x7f61881987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:19.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618ebff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6188101e20 0x7f618819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:19.365 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618ebff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618819fe30 con 0x7f61881002a0 2023-11-27T15:32:19.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618ebff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618819ff70 con 0x7f6188101e20 2023-11-27T15:32:19.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618ebff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61881a0100 con 0x7f6188101490 2023-11-27T15:32:19.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618c99b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f61881002a0 0x7f61881982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:19.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618c99b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f61881002a0 0x7f61881982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43828/0 (socket says 172.21.15.8:43828) 2023-11-27T15:32:19.366 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618c99b700 1 -- 172.21.15.8:0/192412563 learned_addr learned my addr 172.21.15.8:0/192412563 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:19.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.358+0000 7f618d19c700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6188101e20 0x7f618819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:19.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f6187fff700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188101490 0x7f61881987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:19.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f618c99b700 1 -- 172.21.15.8:0/192412563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188101490 msgr2=0x7f61881987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:19.367 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f618c99b700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188101490 0x7f61881987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f618c99b700 1 -- 172.21.15.8:0/192412563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6188101e20 msgr2=0x7f618819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:19.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f618c99b700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6188101e20 0x7f618819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f618c99b700 1 -- 172.21.15.8:0/192412563 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c0095f0 con 0x7f61881002a0 2023-11-27T15:32:19.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f618c99b700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f61881002a0 0x7f61881982b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f617c00ebf0 tx=0x7f617c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:19.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f6185ffb700 1 -- 172.21.15.8:0/192412563 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c009a10 con 0x7f61881002a0 2023-11-27T15:32:19.368 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f618ebff700 1 -- 172.21.15.8:0/192412563 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6174009520 con 0x7f61881002a0 2023-11-27T15:32:19.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f6185ffb700 1 -- 172.21.15.8:0/192412563 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f617c00cd40 con 0x7f61881002a0 2023-11-27T15:32:19.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f6185ffb700 1 -- 172.21.15.8:0/192412563 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c018820 con 0x7f61881002a0 2023-11-27T15:32:19.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.359+0000 7f618ebff700 1 -- 172.21.15.8:0/192412563 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618810bc50 con 0x7f61881002a0 2023-11-27T15:32:19.369 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.361+0000 7f618ebff700 1 -- 172.21.15.8:0/192412563 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618810bde0 con 0x7f61881002a0 2023-11-27T15:32:19.370 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.363+0000 7f6185ffb700 1 -- 172.21.15.8:0/192412563 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f617c018980 con 0x7f61881002a0 2023-11-27T15:32:19.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.364+0000 7f6185ffb700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f61780a1150 0x7f61780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:19.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.364+0000 7f6185ffb700 1 -- 172.21.15.8:0/192412563 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f617c014070 con 0x7f61881002a0 2023-11-27T15:32:19.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.364+0000 7f6187fff700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f61780a1150 0x7f61780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:19.371 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.365+0000 7f6187fff700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f61780a1150 0x7f61780a3600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f617400f9e0 tx=0x7f6174019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:19.372 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.368+0000 7f6185ffb700 1 -- 172.21.15.8:0/192412563 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f617c096700 con 0x7f61881002a0 2023-11-27T15:32:19.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.579+0000 7f618ebff700 1 -- 172.21.15.8:0/192412563 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6188067a50 con 0x7f61881002a0 2023-11-27T15:32:19.583 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.580+0000 7f6185ffb700 1 -- 172.21.15.8:0/192412563 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f617c099d20 con 0x7f61881002a0 2023-11-27T15:32:19.584 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-11-27T15:32:19.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.582+0000 7f618ebff700 1 -- 172.21.15.8:0/192412563 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f61780a1150 msgr2=0x7f61780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:19.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.582+0000 7f618ebff700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f61780a1150 0x7f61780a3600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f617400f9e0 tx=0x7f6174019040 comp rx=0 tx=0).stop 2023-11-27T15:32:19.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.582+0000 7f618ebff700 1 -- 172.21.15.8:0/192412563 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f61881002a0 msgr2=0x7f61881982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:19.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.582+0000 7f618ebff700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f61881002a0 0x7f61881982b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f617c00ebf0 tx=0x7f617c00ef00 comp rx=0 tx=0).stop 2023-11-27T15:32:19.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.582+0000 7f618ebff700 1 -- 172.21.15.8:0/192412563 shutdown_connections 2023-11-27T15:32:19.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.582+0000 7f618ebff700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6188101e20 0x7f618819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.582+0000 7f618ebff700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f61881002a0 0x7f61881982b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.582+0000 7f618ebff700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f61780a1150 0x7f61780a3600 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.582+0000 7f618ebff700 1 --2- 172.21.15.8:0/192412563 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6188101490 0x7f61881987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.582+0000 7f618ebff700 1 -- 172.21.15.8:0/192412563 >> 172.21.15.8:0/192412563 conn(0x7f61880fb880 msgr2=0x7f6188106e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:19.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.583+0000 7f618ebff700 1 -- 172.21.15.8:0/192412563 shutdown_connections 2023-11-27T15:32:19.589 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.583+0000 7f618ebff700 1 -- 172.21.15.8:0/192412563 wait complete. 2023-11-27T15:32:19.605 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:32:19.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.812+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3732351716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9adc1002a0 msgr2=0x7f9adc1006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:19.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.812+0000 7f9ae17ff700 1 --2- 172.21.15.8:0/3732351716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9adc1002a0 0x7f9adc1006b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009990 tx=0x7f9acc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.816 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.812+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3732351716 shutdown_connections 2023-11-27T15:32:19.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.812+0000 7f9ae17ff700 1 --2- 172.21.15.8:0/3732351716 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9adc101e20 0x7f9adc1086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.812+0000 7f9ae17ff700 1 --2- 172.21.15.8:0/3732351716 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9adc1002a0 0x7f9adc1006b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.812+0000 7f9ae17ff700 1 --2- 172.21.15.8:0/3732351716 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9adc101490 0x7f9adc1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.812+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3732351716 >> 172.21.15.8:0/3732351716 conn(0x7f9adc0fb880 msgr2=0x7f9adc0fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:19.817 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.812+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3732351716 shutdown_connections 2023-11-27T15:32:19.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.812+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3732351716 wait complete. 2023-11-27T15:32:19.818 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.813+0000 7f9ae17ff700 1 Processor -- start 2023-11-27T15:32:19.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.813+0000 7f9ae17ff700 1 -- start start 2023-11-27T15:32:19.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.813+0000 7f9ae17ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9adc1002a0 0x7f9adc198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:19.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.813+0000 7f9ae17ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9adc101490 0x7f9adc1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:19.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.813+0000 7f9ae17ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9adc101e20 0x7f9adc19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:19.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.813+0000 7f9ae17ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19fe00 con 0x7f9adc101490 2023-11-27T15:32:19.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.813+0000 7f9ae17ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19ff40 con 0x7f9adc101e20 2023-11-27T15:32:19.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.813+0000 7f9ae17ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1a00d0 con 0x7f9adc1002a0 2023-11-27T15:32:19.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9adaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9adc1002a0 0x7f9adc198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:19.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9ada7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9adc101490 0x7f9adc1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:19.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9adaffd700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9adc1002a0 0x7f9adc198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.60:3300/0 says I am v2:172.21.15.8:41334/0 (socket says 172.21.15.8:41334) 2023-11-27T15:32:19.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9adaffd700 1 -- 172.21.15.8:0/3849108104 learned_addr learned my addr 172.21.15.8:0/3849108104 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:19.822 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9adb7fe700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9adc101e20 0x7f9adc19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:19.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9ada7fc700 1 -- 172.21.15.8:0/3849108104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9adc1002a0 msgr2=0x7f9adc198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:19.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9ada7fc700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9adc1002a0 0x7f9adc198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9ada7fc700 1 -- 172.21.15.8:0/3849108104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9adc101e20 msgr2=0x7f9adc19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:19.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9ada7fc700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9adc101e20 0x7f9adc19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:19.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9ada7fc700 1 -- 172.21.15.8:0/3849108104 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0095f0 con 0x7f9adc101490 2023-11-27T15:32:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.814+0000 7f9ada7fc700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9adc101490 0x7f9adc1987c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000e9f0 tx=0x7f9ad000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.815+0000 7f9ac3fff700 1 -- 172.21.15.8:0/3849108104 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000c9c0 con 0x7f9adc101490 2023-11-27T15:32:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.815+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3849108104 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc10bc70 con 0x7f9adc101490 2023-11-27T15:32:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.815+0000 7f9ac3fff700 1 -- 172.21.15.8:0/3849108104 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ad0010430 con 0x7f9adc101490 2023-11-27T15:32:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.815+0000 7f9ac3fff700 1 -- 172.21.15.8:0/3849108104 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0018680 con 0x7f9adc101490 2023-11-27T15:32:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.815+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3849108104 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc10bae0 con 0x7f9adc101490 2023-11-27T15:32:19.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.816+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3849108104 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc192900 con 0x7f9adc101490 2023-11-27T15:32:19.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:19 smithi060 ceph-mon[131518]: pgmap v554: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-11-27T15:32:19.825 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:19 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2706603291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:32:19.826 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:19 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1050330901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:32:19.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.818+0000 7f9ac3fff700 1 -- 172.21.15.8:0/3849108104 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9ad0010860 con 0x7f9adc101490 2023-11-27T15:32:19.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.819+0000 7f9ac3fff700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ac40a11d0 0x7f9ac40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:19.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.819+0000 7f9ac3fff700 1 -- 172.21.15.8:0/3849108104 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f9ad0014070 con 0x7f9adc101490 2023-11-27T15:32:19.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.819+0000 7f9adaffd700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ac40a11d0 0x7f9ac40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:19.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.820+0000 7f9adaffd700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ac40a11d0 0x7f9ac40a3680 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00b2e0 tx=0x7f9acc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:19.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:19.823+0000 7f9ac3fff700 1 -- 172.21.15.8:0/3849108104 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9ad0051730 con 0x7f9adc101490 2023-11-27T15:32:19.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:19 smithi008 ceph-mon[128788]: pgmap v554: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-11-27T15:32:19.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:19 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2706603291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:32:19.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:19 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1050330901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:32:19.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:19 smithi029 ceph-mon[131166]: pgmap v554: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-11-27T15:32:19.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:19 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2706603291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:32:19.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:19 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1050330901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:32:20.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.032+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3849108104 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9adc061c00 con 0x7f9adc101490 2023-11-27T15:32:20.036 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.032+0000 7f9ac3fff700 1 -- 172.21.15.8:0/3849108104 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9ad0096180 con 0x7f9adc101490 2023-11-27T15:32:20.036 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:32:20.038 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.034+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3849108104 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ac40a11d0 msgr2=0x7f9ac40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:20.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.034+0000 7f9ae17ff700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ac40a11d0 0x7f9ac40a3680 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00b2e0 tx=0x7f9acc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.034+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3849108104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9adc101490 msgr2=0x7f9adc1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:20.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.034+0000 7f9ae17ff700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9adc101490 0x7f9adc1987c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000e9f0 tx=0x7f9ad000ed00 comp rx=0 tx=0).stop 2023-11-27T15:32:20.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.034+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3849108104 shutdown_connections 2023-11-27T15:32:20.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.034+0000 7f9ae17ff700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f9ac40a11d0 0x7f9ac40a3680 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.034+0000 7f9ae17ff700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f9adc1002a0 0x7f9adc198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.034+0000 7f9ae17ff700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f9adc101e20 0x7f9adc19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.034+0000 7f9ae17ff700 1 --2- 172.21.15.8:0/3849108104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9adc101490 0x7f9adc1987c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.034+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3849108104 >> 172.21.15.8:0/3849108104 conn(0x7f9adc0fb880 msgr2=0x7f9adc106e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:20.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.035+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3849108104 shutdown_connections 2023-11-27T15:32:20.040 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.035+0000 7f9ae17ff700 1 -- 172.21.15.8:0/3849108104 wait complete. 2023-11-27T15:32:20.057 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-27T15:32:20.263 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.260+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3312577195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc714101e50 msgr2=0x7fc714108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:20.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.260+0000 7fc71c7ff700 1 --2- 172.21.15.8:0/3312577195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc714101e50 0x7fc714108710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00b0a0 tx=0x7fc70c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.260+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3312577195 shutdown_connections 2023-11-27T15:32:20.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.260+0000 7fc71c7ff700 1 --2- 172.21.15.8:0/3312577195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc714101e50 0x7fc714108710 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.260+0000 7fc71c7ff700 1 --2- 172.21.15.8:0/3312577195 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7141002d0 0x7fc7141006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.264 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.260+0000 7fc71c7ff700 1 --2- 172.21.15.8:0/3312577195 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7141014c0 0x7fc714101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.260+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3312577195 >> 172.21.15.8:0/3312577195 conn(0x7fc7140fb8b0 msgr2=0x7fc7140fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:20.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.260+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3312577195 shutdown_connections 2023-11-27T15:32:20.265 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.260+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3312577195 wait complete. 2023-11-27T15:32:20.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.261+0000 7fc71c7ff700 1 Processor -- start 2023-11-27T15:32:20.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.261+0000 7fc71c7ff700 1 -- start start 2023-11-27T15:32:20.268 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.261+0000 7fc71c7ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7141002d0 0x7fc7140748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:20.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.261+0000 7fc71c7ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7141014c0 0x7fc714072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:20.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.261+0000 7fc71c7ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc714101e50 0x7fc714073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:20.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.261+0000 7fc71c7ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714073bc0 con 0x7fc714101e50 2023-11-27T15:32:20.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.261+0000 7fc71c7ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc714073d30 con 0x7fc7141014c0 2023-11-27T15:32:20.269 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.261+0000 7fc71c7ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7141a01b0 con 0x7fc7141002d0 2023-11-27T15:32:20.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc71ad9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc714101e50 0x7fc714073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:20.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc71ad9c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc714101e50 0x7fc714073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55292/0 (socket says 172.21.15.8:55292) 2023-11-27T15:32:20.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc71ad9c700 1 -- 172.21.15.8:0/3557587010 learned_addr learned my addr 172.21.15.8:0/3557587010 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:32:20.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc719d9a700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7141014c0 0x7fc714072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:20.270 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc71a59b700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7141002d0 0x7fc7140748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc71ad9c700 1 -- 172.21.15.8:0/3557587010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7141002d0 msgr2=0x7fc7140748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc71ad9c700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7141002d0 0x7fc7140748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc71ad9c700 1 -- 172.21.15.8:0/3557587010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7141014c0 msgr2=0x7fc714072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc71ad9c700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7141014c0 0x7fc714072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc71ad9c700 1 -- 172.21.15.8:0/3557587010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c009d00 con 0x7fc714101e50 2023-11-27T15:32:20.271 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.262+0000 7fc71ad9c700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc714101e50 0x7fc714073420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d010 tx=0x7fc70c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.263+0000 7fc70b7fe700 1 -- 172.21.15.8:0/3557587010 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c00e040 con 0x7fc714101e50 2023-11-27T15:32:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.263+0000 7fc70b7fe700 1 -- 172.21.15.8:0/3557587010 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc70c003e10 con 0x7fc714101e50 2023-11-27T15:32:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.263+0000 7fc70b7fe700 1 -- 172.21.15.8:0/3557587010 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0052a0 con 0x7fc714101e50 2023-11-27T15:32:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.263+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3557587010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71410bca0 con 0x7fc714101e50 2023-11-27T15:32:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.263+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3557587010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71410bb10 con 0x7fc714101e50 2023-11-27T15:32:20.272 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.264+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3557587010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7141077d0 con 0x7fc714101e50 2023-11-27T15:32:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.266+0000 7fc70b7fe700 1 -- 172.21.15.8:0/3557587010 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc70c01d480 con 0x7fc714101e50 2023-11-27T15:32:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.268+0000 7fc70b7fe700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc7000a11d0 0x7fc7000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:32:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.268+0000 7fc70b7fe700 1 -- 172.21.15.8:0/3557587010 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fc70c0cdd50 con 0x7fc714101e50 2023-11-27T15:32:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.268+0000 7fc71a59b700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc7000a11d0 0x7fc7000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:32:20.274 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.269+0000 7fc71a59b700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc7000a11d0 0x7fc7000a3680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc714101320 tx=0x7fc710009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:32:20.275 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.272+0000 7fc70b7fe700 1 -- 172.21.15.8:0/3557587010 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc70c0971a0 con 0x7fc714101e50 2023-11-27T15:32:20.545 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.541+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3557587010 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fc71402cfe0 con 0x7fc714101e50 2023-11-27T15:32:20.555 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.552+0000 7fc70b7fe700 1 -- 172.21.15.8:0/3557587010 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733236 (secure 0 0 0) 0x7fc70c09a7c0 con 0x7fc714101e50 2023-11-27T15:32:20.567 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:32:20.568 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:32:20.568 INFO:teuthology.orchestra.run.smithi008.stdout: "epoch": 18, 2023-11-27T15:32:20.568 INFO:teuthology.orchestra.run.smithi008.stdout: "active_gid": 14152, 2023-11-27T15:32:20.568 INFO:teuthology.orchestra.run.smithi008.stdout: "active_name": "x", 2023-11-27T15:32:20.569 INFO:teuthology.orchestra.run.smithi008.stdout: "active_addrs": { 2023-11-27T15:32:20.569 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2023-11-27T15:32:20.569 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.569 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2023-11-27T15:32:20.570 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:6800", 2023-11-27T15:32:20.570 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 1756612424 2023-11-27T15:32:20.570 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.570 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.571 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v1", 2023-11-27T15:32:20.571 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:6801", 2023-11-27T15:32:20.571 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 1756612424 2023-11-27T15:32:20.571 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.572 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2023-11-27T15:32:20.572 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.572 INFO:teuthology.orchestra.run.smithi008.stdout: "active_addr": "172.21.15.8:6801/1756612424", 2023-11-27T15:32:20.572 INFO:teuthology.orchestra.run.smithi008.stdout: "active_change": "2023-11-27T15:16:38.243059+0000", 2023-11-27T15:32:20.573 INFO:teuthology.orchestra.run.smithi008.stdout: "active_mgr_features": 4540138322906710015, 2023-11-27T15:32:20.573 INFO:teuthology.orchestra.run.smithi008.stdout: "available": true, 2023-11-27T15:32:20.573 INFO:teuthology.orchestra.run.smithi008.stdout: "standbys": [ 2023-11-27T15:32:20.573 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.573 INFO:teuthology.orchestra.run.smithi008.stdout: "gid": 24112, 2023-11-27T15:32:20.574 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "z", 2023-11-27T15:32:20.574 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr_features": 4540138322906710015, 2023-11-27T15:32:20.574 INFO:teuthology.orchestra.run.smithi008.stdout: "available_modules": [ 2023-11-27T15:32:20.574 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.575 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alerts", 2023-11-27T15:32:20.575 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.575 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.575 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.575 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:20.576 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:20.576 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.576 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.576 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.577 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:20.577 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.577 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.577 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.578 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How frequently to reexamine health status", 2023-11-27T15:32:20.578 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.578 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.578 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.579 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.579 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.579 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.579 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.580 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.580 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.580 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.580 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.581 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.581 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.581 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.581 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.581 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.582 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.582 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.582 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.582 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.583 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.583 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.583 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.583 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.584 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.584 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.584 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.584 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.585 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.585 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.585 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.585 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.586 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.586 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.586 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.586 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.586 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.587 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.587 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.587 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.587 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.588 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.588 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.588 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.588 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.589 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.589 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.589 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.589 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.590 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.590 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.590 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.590 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.591 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.591 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.591 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.592 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.592 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.592 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.592 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.592 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.593 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.593 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.593 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.593 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.594 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.594 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.594 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.594 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.595 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.595 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.595 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.595 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.595 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.596 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_destination": { 2023-11-27T15:32:20.596 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_destination", 2023-11-27T15:32:20.596 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.596 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.597 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.597 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.597 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.597 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.598 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.598 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email address to send alerts to", 2023-11-27T15:32:20.598 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.598 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.599 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.599 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.599 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_from_name": { 2023-11-27T15:32:20.599 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_from_name", 2023-11-27T15:32:20.599 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.600 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.600 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.600 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "Ceph", 2023-11-27T15:32:20.600 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.601 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.601 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.601 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email From: name", 2023-11-27T15:32:20.601 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.602 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.602 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.602 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.602 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_host": { 2023-11-27T15:32:20.602 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_host", 2023-11-27T15:32:20.603 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.603 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.603 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.603 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.604 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.604 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.604 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.604 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP server", 2023-11-27T15:32:20.604 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.605 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.605 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.605 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.605 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_password": { 2023-11-27T15:32:20.605 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_password", 2023-11-27T15:32:20.606 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.606 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.606 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.606 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.607 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.607 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.607 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.607 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Password to authenticate with", 2023-11-27T15:32:20.607 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.608 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.608 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.608 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.608 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_port": { 2023-11-27T15:32:20.609 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_port", 2023-11-27T15:32:20.609 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.609 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.609 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.609 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "465", 2023-11-27T15:32:20.610 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.610 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.610 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.610 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP port", 2023-11-27T15:32:20.611 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.611 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.611 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.611 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.611 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_sender": { 2023-11-27T15:32:20.612 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_sender", 2023-11-27T15:32:20.612 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.612 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.612 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.613 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.613 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.613 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.613 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.613 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP envelope sender", 2023-11-27T15:32:20.614 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.614 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.614 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.614 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.615 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_ssl": { 2023-11-27T15:32:20.615 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_ssl", 2023-11-27T15:32:20.615 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.615 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.616 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.616 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.616 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.616 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.616 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.617 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-27T15:32:20.617 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.617 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.617 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.617 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.618 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_user": { 2023-11-27T15:32:20.618 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_user", 2023-11-27T15:32:20.618 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.618 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.619 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.619 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.619 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.619 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.619 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.620 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "User to authenticate as", 2023-11-27T15:32:20.620 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.620 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.620 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.621 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.621 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.621 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.621 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.621 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "balancer", 2023-11-27T15:32:20.622 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.622 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.622 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.622 INFO:teuthology.orchestra.run.smithi008.stdout: "active": { 2023-11-27T15:32:20.623 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "active", 2023-11-27T15:32:20.623 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.623 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.623 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.623 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.624 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.624 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.624 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.624 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically balance PGs across cluster", 2023-11-27T15:32:20.625 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.625 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.625 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.625 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.625 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_time": { 2023-11-27T15:32:20.626 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_time", 2023-11-27T15:32:20.626 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.626 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.627 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.627 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0000", 2023-11-27T15:32:20.627 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.627 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.627 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.627 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "beginning time of day to automatically balance", 2023-11-27T15:32:20.627 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:32:20.628 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.628 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.628 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.628 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_weekday": { 2023-11-27T15:32:20.628 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_weekday", 2023-11-27T15:32:20.628 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:20.628 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.629 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.629 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:20.629 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2023-11-27T15:32:20.629 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2023-11-27T15:32:20.629 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.629 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-27T15:32:20.629 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:32:20.630 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.630 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.630 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.630 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_max_iterations": { 2023-11-27T15:32:20.630 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_max_iterations", 2023-11-27T15:32:20.630 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:20.630 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.631 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.631 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "25", 2023-11-27T15:32:20.631 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:20.631 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "250", 2023-11-27T15:32:20.631 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.631 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-27T15:32:20.631 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.631 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.632 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.632 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.632 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_metrics": { 2023-11-27T15:32:20.632 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_metrics", 2023-11-27T15:32:20.632 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.632 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.632 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.633 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "pgs,objects,bytes", 2023-11-27T15:32:20.633 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.633 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.633 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.633 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-27T15:32:20.633 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-27T15:32:20.633 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.633 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.634 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.634 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_step": { 2023-11-27T15:32:20.634 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_step", 2023-11-27T15:32:20.634 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:20.634 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.634 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.634 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.5", 2023-11-27T15:32:20.634 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0.001", 2023-11-27T15:32:20.635 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "0.999", 2023-11-27T15:32:20.635 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.635 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "aggressiveness of optimization", 2023-11-27T15:32:20.635 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-27T15:32:20.635 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.635 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.635 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.636 INFO:teuthology.orchestra.run.smithi008.stdout: "end_time": { 2023-11-27T15:32:20.636 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_time", 2023-11-27T15:32:20.636 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.636 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.636 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.636 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2359", 2023-11-27T15:32:20.636 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.636 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.637 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.637 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ending time of day to automatically balance", 2023-11-27T15:32:20.637 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:32:20.637 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.637 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.637 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.637 INFO:teuthology.orchestra.run.smithi008.stdout: "end_weekday": { 2023-11-27T15:32:20.638 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_weekday", 2023-11-27T15:32:20.638 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:20.638 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.638 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.638 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:20.638 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2023-11-27T15:32:20.638 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2023-11-27T15:32:20.638 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.639 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-27T15:32:20.639 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:32:20.639 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.639 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.639 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.639 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.639 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.640 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.640 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.640 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.640 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.640 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.640 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.640 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.640 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.641 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.641 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.641 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.641 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.641 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.641 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.641 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.641 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.642 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.642 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.642 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.642 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.642 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.642 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.642 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.643 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.643 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.643 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.643 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.643 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.643 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.643 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.644 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.644 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.644 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.644 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.644 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.644 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.644 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.644 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.645 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.645 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.645 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.645 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.645 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.645 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.645 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.645 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.646 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.646 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.646 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.646 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.646 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.646 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.646 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.647 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.647 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.647 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.647 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.647 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.647 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.647 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.647 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.648 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.648 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.648 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.648 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.648 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.648 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.648 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.649 INFO:teuthology.orchestra.run.smithi008.stdout: "min_score": { 2023-11-27T15:32:20.649 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_score", 2023-11-27T15:32:20.649 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:20.649 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.649 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.649 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:20.649 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.649 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.650 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.650 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-27T15:32:20.650 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.650 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.650 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.650 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.650 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2023-11-27T15:32:20.651 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2023-11-27T15:32:20.651 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.651 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.651 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.651 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "upmap", 2023-11-27T15:32:20.651 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.651 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.651 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.652 INFO:teuthology.orchestra.run.smithi008.stdout: "crush-compat", 2023-11-27T15:32:20.652 INFO:teuthology.orchestra.run.smithi008.stdout: "none", 2023-11-27T15:32:20.652 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap" 2023-11-27T15:32:20.652 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.652 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Balancer mode", 2023-11-27T15:32:20.652 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.652 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.652 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.653 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.653 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_ids": { 2023-11-27T15:32:20.653 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_ids", 2023-11-27T15:32:20.653 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.653 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.653 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.653 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.654 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.654 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.654 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.654 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-27T15:32:20.654 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.654 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.654 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.654 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.655 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:20.655 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:20.655 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.655 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.655 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.655 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:20.655 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-27T15:32:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.656 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_deviation": { 2023-11-27T15:32:20.657 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_deviation", 2023-11-27T15:32:20.657 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.657 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.657 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.657 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:20.657 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:20.657 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.658 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.658 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-27T15:32:20.658 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-27T15:32:20.658 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.658 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.658 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.658 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_optimizations": { 2023-11-27T15:32:20.659 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_optimizations", 2023-11-27T15:32:20.659 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:20.659 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.659 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.659 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:20.659 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.659 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.659 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.660 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-27T15:32:20.660 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.660 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.660 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.660 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.660 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.660 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.661 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.661 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cephadm", 2023-11-27T15:32:20.661 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.661 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.661 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.661 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_down_multiplier": { 2023-11-27T15:32:20.661 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_down_multiplier", 2023-11-27T15:32:20.661 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:20.662 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.662 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.662 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2023-11-27T15:32:20.662 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.662 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.662 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.662 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-27T15:32:20.662 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.663 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.663 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.663 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.663 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_refresh_rate": { 2023-11-27T15:32:20.663 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_refresh_rate", 2023-11-27T15:32:20.663 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.663 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.663 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.664 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "20", 2023-11-27T15:32:20.664 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.664 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.664 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.664 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-27T15:32:20.664 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.664 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.664 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.665 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.665 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_starting_port": { 2023-11-27T15:32:20.665 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_starting_port", 2023-11-27T15:32:20.665 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.665 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.665 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.665 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4721", 2023-11-27T15:32:20.665 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.666 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.666 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.666 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-27T15:32:20.666 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.666 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.666 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.666 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.667 INFO:teuthology.orchestra.run.smithi008.stdout: "alertmanager_web_password": { 2023-11-27T15:32:20.667 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alertmanager_web_password", 2023-11-27T15:32:20.667 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.667 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.667 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.667 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:20.667 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.667 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.668 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.668 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Alertmanager web password", 2023-11-27T15:32:20.668 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.668 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.668 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.668 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.668 INFO:teuthology.orchestra.run.smithi008.stdout: "alertmanager_web_user": { 2023-11-27T15:32:20.668 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alertmanager_web_user", 2023-11-27T15:32:20.669 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.669 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.669 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.669 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:20.669 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.669 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.669 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.669 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Alertmanager web user", 2023-11-27T15:32:20.670 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.670 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.670 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.670 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.670 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_ptrace": { 2023-11-27T15:32:20.670 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_ptrace", 2023-11-27T15:32:20.670 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.670 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.671 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.671 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.671 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.671 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.671 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.671 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-27T15:32:20.671 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-27T15:32:20.671 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.672 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.672 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.672 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_interval": { 2023-11-27T15:32:20.672 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_interval", 2023-11-27T15:32:20.672 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.672 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.672 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.672 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:20.673 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.673 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.673 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.673 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-27T15:32:20.673 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.673 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.673 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.673 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.674 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_memory_target_ratio": { 2023-11-27T15:32:20.674 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_memory_target_ratio", 2023-11-27T15:32:20.674 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:20.674 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.674 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.674 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.7", 2023-11-27T15:32:20.674 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.674 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.675 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.675 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-27T15:32:20.675 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.675 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.675 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.675 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.675 INFO:teuthology.orchestra.run.smithi008.stdout: "cgroups_split": { 2023-11-27T15:32:20.675 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cgroups_split", 2023-11-27T15:32:20.676 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.676 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.676 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.676 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.676 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.676 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.676 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.676 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-27T15:32:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: "config_checks_enabled": { 2023-11-27T15:32:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_checks_enabled", 2023-11-27T15:32:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.678 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.678 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.678 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.678 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.678 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-27T15:32:20.678 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.678 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.679 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.679 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.679 INFO:teuthology.orchestra.run.smithi008.stdout: "config_dashboard": { 2023-11-27T15:32:20.679 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_dashboard", 2023-11-27T15:32:20.679 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.679 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.679 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.679 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.680 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.680 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.680 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.680 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-27T15:32:20.680 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.680 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.680 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.680 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.681 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_alertmanager": { 2023-11-27T15:32:20.681 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_alertmanager", 2023-11-27T15:32:20.681 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.681 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.681 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.681 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-27T15:32:20.681 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.681 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.681 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.682 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:20.682 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.682 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.682 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.682 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.682 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_base": { 2023-11-27T15:32:20.683 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_base", 2023-11-27T15:32:20.683 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.683 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.683 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.683 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-27T15:32:20.683 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.683 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.683 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.683 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Container image name, without the tag", 2023-11-27T15:32:20.684 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.684 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.684 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.684 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.684 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_elasticsearch": { 2023-11-27T15:32:20.684 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_elasticsearch", 2023-11-27T15:32:20.684 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.684 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.685 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.685 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-27T15:32:20.685 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.685 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.685 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.685 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "elasticsearch container image", 2023-11-27T15:32:20.685 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.685 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.685 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.686 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.686 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_grafana": { 2023-11-27T15:32:20.686 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_grafana", 2023-11-27T15:32:20.686 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.686 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.686 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.686 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-27T15:32:20.686 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.687 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.687 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.687 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:20.687 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.687 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.687 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.687 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.688 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_haproxy": { 2023-11-27T15:32:20.688 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_haproxy", 2023-11-27T15:32:20.688 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.688 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.688 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.688 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-27T15:32:20.688 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.688 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.689 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.689 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "HAproxy container image", 2023-11-27T15:32:20.689 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.689 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.689 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.689 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.689 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_agent": { 2023-11-27T15:32:20.689 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_agent", 2023-11-27T15:32:20.690 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.690 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.690 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.690 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-27T15:32:20.690 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.690 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.690 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.690 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger agent container image", 2023-11-27T15:32:20.690 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.691 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.691 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.691 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.691 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_collector": { 2023-11-27T15:32:20.691 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_collector", 2023-11-27T15:32:20.691 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.691 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.691 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.692 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-27T15:32:20.692 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.692 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.692 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.692 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger collector container image", 2023-11-27T15:32:20.692 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.692 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.692 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.692 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.693 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_query": { 2023-11-27T15:32:20.693 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_query", 2023-11-27T15:32:20.693 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.693 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.693 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.693 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-27T15:32:20.693 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.693 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.694 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.694 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger query container image", 2023-11-27T15:32:20.694 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.694 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.694 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.694 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.694 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_keepalived": { 2023-11-27T15:32:20.694 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_keepalived", 2023-11-27T15:32:20.695 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.695 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.695 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.695 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-27T15:32:20.695 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.695 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.695 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.695 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Keepalived container image", 2023-11-27T15:32:20.695 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.696 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.696 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.696 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.696 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_loki": { 2023-11-27T15:32:20.696 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_loki", 2023-11-27T15:32:20.696 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.696 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.696 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.697 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-27T15:32:20.697 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.697 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.697 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.697 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Loki container image", 2023-11-27T15:32:20.697 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.697 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.697 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.697 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.698 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_node_exporter": { 2023-11-27T15:32:20.698 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_node_exporter", 2023-11-27T15:32:20.698 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.698 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.698 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.698 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-27T15:32:20.698 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.698 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.699 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.699 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:20.699 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.699 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.699 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.699 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.699 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_prometheus": { 2023-11-27T15:32:20.699 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_prometheus", 2023-11-27T15:32:20.700 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.700 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.700 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.700 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-27T15:32:20.700 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.700 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.700 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.700 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:20.700 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.701 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.701 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.701 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.701 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_promtail": { 2023-11-27T15:32:20.701 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_promtail", 2023-11-27T15:32:20.701 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.701 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.701 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.702 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-27T15:32:20.702 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.702 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.702 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.702 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Promtail container image", 2023-11-27T15:32:20.702 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.702 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.702 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.702 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.703 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_snmp_gateway": { 2023-11-27T15:32:20.703 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_snmp_gateway", 2023-11-27T15:32:20.703 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.703 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.703 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.703 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-27T15:32:20.703 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.703 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.704 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.704 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SNMP Gateway container image", 2023-11-27T15:32:20.704 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.704 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.704 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.704 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.704 INFO:teuthology.orchestra.run.smithi008.stdout: "container_init": { 2023-11-27T15:32:20.704 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_init", 2023-11-27T15:32:20.704 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.705 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.705 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.705 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.705 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.705 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.705 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.705 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Run podman/docker with `--init`", 2023-11-27T15:32:20.705 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.706 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.706 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.706 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.706 INFO:teuthology.orchestra.run.smithi008.stdout: "daemon_cache_timeout": { 2023-11-27T15:32:20.706 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "daemon_cache_timeout", 2023-11-27T15:32:20.706 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.706 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.706 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.707 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:20.707 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.707 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.707 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.707 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-27T15:32:20.707 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.707 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.707 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.707 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.708 INFO:teuthology.orchestra.run.smithi008.stdout: "default_cephadm_command_timeout": { 2023-11-27T15:32:20.708 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_cephadm_command_timeout", 2023-11-27T15:32:20.708 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.708 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.708 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.708 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "900", 2023-11-27T15:32:20.708 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.708 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.709 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.709 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-27T15:32:20.709 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.709 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.709 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.709 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.709 INFO:teuthology.orchestra.run.smithi008.stdout: "default_registry": { 2023-11-27T15:32:20.709 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_registry", 2023-11-27T15:32:20.710 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.710 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.710 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.710 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io", 2023-11-27T15:32:20.710 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.710 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.710 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.710 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-27T15:32:20.711 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.711 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.711 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.711 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.711 INFO:teuthology.orchestra.run.smithi008.stdout: "device_cache_timeout": { 2023-11-27T15:32:20.711 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_cache_timeout", 2023-11-27T15:32:20.711 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.711 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.711 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.712 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1800", 2023-11-27T15:32:20.712 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.712 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.712 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.712 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache device inventory", 2023-11-27T15:32:20.712 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.712 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.712 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.713 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.713 INFO:teuthology.orchestra.run.smithi008.stdout: "device_enhanced_scan": { 2023-11-27T15:32:20.713 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_enhanced_scan", 2023-11-27T15:32:20.713 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.713 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.713 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.713 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.714 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.714 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.714 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.714 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-27T15:32:20.714 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.714 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.714 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.714 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.715 INFO:teuthology.orchestra.run.smithi008.stdout: "facts_cache_timeout": { 2023-11-27T15:32:20.715 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "facts_cache_timeout", 2023-11-27T15:32:20.715 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.715 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.715 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.715 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:20.715 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.715 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.715 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.716 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache host facts data", 2023-11-27T15:32:20.716 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.716 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.716 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.716 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.716 INFO:teuthology.orchestra.run.smithi008.stdout: "host_check_interval": { 2023-11-27T15:32:20.716 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "host_check_interval", 2023-11-27T15:32:20.716 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.716 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.717 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.717 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:20.717 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.717 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.717 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.717 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to perform a host check", 2023-11-27T15:32:20.717 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.717 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.718 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.718 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.718 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.718 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.718 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.718 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.718 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.718 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.719 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.719 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.719 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.719 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.719 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.719 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.719 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.719 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.719 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.720 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.720 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.720 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.720 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.720 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.720 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.720 INFO:teuthology.orchestra.run.smithi008.stdout: "log_refresh_metadata": { 2023-11-27T15:32:20.720 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_refresh_metadata", 2023-11-27T15:32:20.721 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.721 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.721 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.721 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.721 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.721 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.721 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.721 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-27T15:32:20.721 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.722 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.722 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.722 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.722 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.722 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.722 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.722 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.722 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.723 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.723 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.723 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.723 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.723 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-27T15:32:20.723 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.723 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.723 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.723 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.724 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.724 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.724 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.724 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.724 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.724 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.724 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.724 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.725 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.725 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.725 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.725 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.725 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.725 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.725 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.725 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.726 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.726 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.726 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.726 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.726 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.726 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.726 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.726 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.726 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.727 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.727 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.727 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.727 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.727 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.727 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.727 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.727 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.727 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.728 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.728 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-27T15:32:20.728 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-27T15:32:20.728 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.728 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.728 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.728 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.729 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.729 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.729 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.729 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-27T15:32:20.729 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.729 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.729 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.729 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.730 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-27T15:32:20.730 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-27T15:32:20.730 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.730 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.730 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.730 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "*", 2023-11-27T15:32:20.730 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.730 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.731 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.731 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-27T15:32:20.731 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.731 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.731 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.731 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.731 INFO:teuthology.orchestra.run.smithi008.stdout: "max_count_per_host": { 2023-11-27T15:32:20.731 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_count_per_host", 2023-11-27T15:32:20.731 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.732 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.732 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.732 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:20.732 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.732 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.732 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.732 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of daemons per service per host", 2023-11-27T15:32:20.732 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.733 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.733 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.733 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.733 INFO:teuthology.orchestra.run.smithi008.stdout: "max_osd_draining_count": { 2023-11-27T15:32:20.733 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_osd_draining_count", 2023-11-27T15:32:20.733 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.733 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.733 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.733 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:20.734 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.734 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.734 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.734 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-27T15:32:20.734 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.734 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.734 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.734 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.735 INFO:teuthology.orchestra.run.smithi008.stdout: "migration_current": { 2023-11-27T15:32:20.735 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "migration_current", 2023-11-27T15:32:20.735 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.735 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.735 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.735 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.735 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.735 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.736 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.736 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "internal - do not modify", 2023-11-27T15:32:20.736 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.736 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.736 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.736 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.736 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2023-11-27T15:32:20.736 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2023-11-27T15:32:20.737 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.737 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.737 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.737 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "root", 2023-11-27T15:32:20.737 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.737 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.737 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.738 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm-package", 2023-11-27T15:32:20.738 INFO:teuthology.orchestra.run.smithi008.stdout: "root" 2023-11-27T15:32:20.738 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.738 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "mode for remote execution of cephadm", 2023-11-27T15:32:20.738 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.738 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.738 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.739 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.739 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_alerts_path": { 2023-11-27T15:32:20.739 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_alerts_path", 2023-11-27T15:32:20.739 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.739 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.739 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.739 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-27T15:32:20.739 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.739 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.740 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.740 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-27T15:32:20.740 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.740 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.740 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.740 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.740 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_web_password": { 2023-11-27T15:32:20.740 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_web_password", 2023-11-27T15:32:20.741 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.741 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.741 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.741 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:20.741 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.741 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.741 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.741 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus web password", 2023-11-27T15:32:20.741 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.742 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.742 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.742 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.742 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_web_user": { 2023-11-27T15:32:20.742 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_web_user", 2023-11-27T15:32:20.742 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.742 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.742 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.743 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:20.743 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.743 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.743 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.743 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus web user", 2023-11-27T15:32:20.743 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.743 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.743 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.743 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.744 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_insecure": { 2023-11-27T15:32:20.744 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_insecure", 2023-11-27T15:32:20.744 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.744 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.744 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.744 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.744 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.744 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.745 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.745 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-27T15:32:20.745 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.745 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.745 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.745 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.745 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_password": { 2023-11-27T15:32:20.745 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_password", 2023-11-27T15:32:20.746 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.746 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.746 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.746 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.746 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.746 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.746 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.746 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-27T15:32:20.747 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.747 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.747 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.747 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.747 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_url": { 2023-11-27T15:32:20.747 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_url", 2023-11-27T15:32:20.747 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.747 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.748 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.748 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.748 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.748 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.748 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.748 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-27T15:32:20.748 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.748 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.749 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.749 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.749 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_username": { 2023-11-27T15:32:20.749 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_username", 2023-11-27T15:32:20.749 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.749 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.749 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.749 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.749 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.750 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.750 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.750 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-27T15:32:20.750 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.750 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.750 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.750 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.750 INFO:teuthology.orchestra.run.smithi008.stdout: "secure_monitoring_stack": { 2023-11-27T15:32:20.751 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "secure_monitoring_stack", 2023-11-27T15:32:20.751 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.751 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.751 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.751 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.751 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.751 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.751 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.752 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-27T15:32:20.752 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.752 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.752 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.752 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.752 INFO:teuthology.orchestra.run.smithi008.stdout: "service_discovery_port": { 2023-11-27T15:32:20.752 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "service_discovery_port", 2023-11-27T15:32:20.752 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.752 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.753 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.753 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8765", 2023-11-27T15:32:20.753 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.753 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.753 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.753 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "cephadm service discovery port", 2023-11-27T15:32:20.753 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.754 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.754 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.754 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.754 INFO:teuthology.orchestra.run.smithi008.stdout: "ssh_config_file": { 2023-11-27T15:32:20.754 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssh_config_file", 2023-11-27T15:32:20.754 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.755 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.755 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.755 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.755 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.755 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.755 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.755 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-27T15:32:20.755 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.755 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.756 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.756 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.756 INFO:teuthology.orchestra.run.smithi008.stdout: "use_agent": { 2023-11-27T15:32:20.756 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_agent", 2023-11-27T15:32:20.756 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.756 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.756 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.757 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.757 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.757 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.757 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.757 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-27T15:32:20.757 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.757 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.757 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.758 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.758 INFO:teuthology.orchestra.run.smithi008.stdout: "use_repo_digest": { 2023-11-27T15:32:20.758 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_repo_digest", 2023-11-27T15:32:20.758 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.758 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.758 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.758 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.758 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.759 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.759 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.759 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-27T15:32:20.759 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.759 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.759 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.759 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.760 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_failed_host_check": { 2023-11-27T15:32:20.760 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_failed_host_check", 2023-11-27T15:32:20.760 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.760 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.760 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.760 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.760 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.761 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.761 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.761 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if the host check fails", 2023-11-27T15:32:20.761 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.761 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.761 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.761 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.761 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_daemons": { 2023-11-27T15:32:20.762 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_daemons", 2023-11-27T15:32:20.762 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.762 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.762 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.762 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.762 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.762 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.762 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.763 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-27T15:32:20.763 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.763 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.763 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.763 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.763 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_hosts": { 2023-11-27T15:32:20.763 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_hosts", 2023-11-27T15:32:20.763 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.764 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.764 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.764 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.764 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.764 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.764 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.764 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-27T15:32:20.764 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.765 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.765 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.765 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.765 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.765 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.765 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.765 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crash", 2023-11-27T15:32:20.765 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.766 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.766 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.766 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.766 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.766 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.766 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.766 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.766 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.766 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.767 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.767 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.767 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.767 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.767 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.767 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.767 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.767 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.768 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.768 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.768 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.768 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.768 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.768 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.768 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.768 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.769 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.769 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.769 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.769 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.769 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.769 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.769 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.769 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.770 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.770 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.770 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.770 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.770 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.770 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.770 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.771 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.771 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.771 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.771 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.771 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.771 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.771 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.771 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.772 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.772 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.772 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.772 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.772 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.772 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.772 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.772 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.772 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.773 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.773 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.773 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.773 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.773 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.773 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.773 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.773 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.774 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.774 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.774 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.774 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.774 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.774 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.774 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.774 INFO:teuthology.orchestra.run.smithi008.stdout: "retain_interval": { 2023-11-27T15:32:20.775 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retain_interval", 2023-11-27T15:32:20.775 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.775 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.775 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.775 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "31536000", 2023-11-27T15:32:20.775 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.775 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.775 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.776 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-27T15:32:20.776 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.776 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.776 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.776 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.776 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_recent_interval": { 2023-11-27T15:32:20.776 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_recent_interval", 2023-11-27T15:32:20.776 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.776 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.777 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.777 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1209600", 2023-11-27T15:32:20.777 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.777 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.777 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.777 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-27T15:32:20.777 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.777 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.778 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.778 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.778 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.778 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.778 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.778 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dashboard", 2023-11-27T15:32:20.778 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.778 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.779 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.779 INFO:teuthology.orchestra.run.smithi008.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-27T15:32:20.779 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-27T15:32:20.779 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.779 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.779 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.779 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:20.779 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.780 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.780 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.780 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.780 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.780 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.780 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.780 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.780 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_HOST": { 2023-11-27T15:32:20.780 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-27T15:32:20.781 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.781 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.781 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.781 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.781 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.781 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.781 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.781 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.782 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.782 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.782 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.782 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.782 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-27T15:32:20.782 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-27T15:32:20.782 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.782 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.782 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.783 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.783 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.783 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.783 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.783 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.783 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.783 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.783 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.784 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.784 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_ENABLED": { 2023-11-27T15:32:20.784 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_ENABLED", 2023-11-27T15:32:20.784 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.784 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.784 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.784 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.784 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.784 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.785 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.785 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.785 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.785 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.785 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.785 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.785 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-27T15:32:20.785 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-27T15:32:20.786 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.786 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.786 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.786 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.786 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.786 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.786 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.786 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.787 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.787 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.787 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.787 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.787 INFO:teuthology.orchestra.run.smithi008.stdout: "ENABLE_BROWSABLE_API": { 2023-11-27T15:32:20.787 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-27T15:32:20.787 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.788 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.788 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.788 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.788 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.788 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.788 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.788 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.789 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.789 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.789 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.789 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.789 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-27T15:32:20.789 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-27T15:32:20.789 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.789 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.790 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.790 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.790 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.790 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.790 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.790 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.790 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.791 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.791 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.791 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.791 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-27T15:32:20.791 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-27T15:32:20.791 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.791 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.791 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.792 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.792 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.792 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.792 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.792 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.792 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.792 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.792 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.793 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.793 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-27T15:32:20.793 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-27T15:32:20.793 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.793 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.793 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.793 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.793 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.794 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.794 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.794 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.794 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.794 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.794 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.794 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.794 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-27T15:32:20.795 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-27T15:32:20.795 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.795 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.795 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.795 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.795 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.795 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.795 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.796 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.796 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.796 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.796 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.796 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.796 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-27T15:32:20.796 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-27T15:32:20.796 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.797 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.797 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.797 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.797 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.797 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.797 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.797 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.798 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.798 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.798 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.798 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.798 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-27T15:32:20.798 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-27T15:32:20.798 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.798 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.799 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.799 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.799 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.799 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.799 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.799 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.799 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.799 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.800 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.800 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.800 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-27T15:32:20.800 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-27T15:32:20.800 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.800 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.800 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.800 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.801 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.801 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.801 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.801 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.801 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.801 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.801 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.801 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.801 INFO:teuthology.orchestra.run.smithi008.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-27T15:32:20.802 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-27T15:32:20.802 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.802 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.802 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.802 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.802 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.802 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.802 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.803 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.803 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.803 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.803 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.803 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.803 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_PASSWORD": { 2023-11-27T15:32:20.803 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-27T15:32:20.803 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.804 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.804 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.804 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:20.804 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.804 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.804 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.804 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.804 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.804 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.805 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.805 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.805 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-27T15:32:20.805 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-27T15:32:20.805 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.805 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.805 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.805 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.806 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.806 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.806 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.806 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.806 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.806 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.806 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.806 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.807 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_URL": { 2023-11-27T15:32:20.807 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_URL", 2023-11-27T15:32:20.807 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.807 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.807 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.807 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.807 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.807 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.808 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.808 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.808 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.808 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.808 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.808 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.808 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_USERNAME": { 2023-11-27T15:32:20.808 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-27T15:32:20.808 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.809 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.809 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.809 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:20.809 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.809 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.809 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.809 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.809 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.810 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.810 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.810 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.810 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-27T15:32:20.810 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-27T15:32:20.810 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.810 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.810 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.810 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.811 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.811 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.811 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.811 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.811 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.811 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.811 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.811 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.812 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-27T15:32:20.812 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-27T15:32:20.812 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.812 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.812 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.812 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.812 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.812 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.812 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.813 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.813 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.813 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.813 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.813 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.813 INFO:teuthology.orchestra.run.smithi008.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-27T15:32:20.813 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-27T15:32:20.813 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.814 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.814 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.814 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.814 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.814 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.814 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.814 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.814 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.815 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.815 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.815 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.815 INFO:teuthology.orchestra.run.smithi008.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-27T15:32:20.815 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-27T15:32:20.815 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.815 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.815 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.815 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.816 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.816 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.816 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.816 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.816 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.816 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.816 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.816 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.816 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_HOST": { 2023-11-27T15:32:20.817 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-27T15:32:20.817 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.817 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.817 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.817 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.817 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.817 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.817 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.817 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.818 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.818 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.818 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.818 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.818 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-27T15:32:20.818 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-27T15:32:20.818 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.819 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.819 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.819 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.819 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.819 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.819 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.819 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.819 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.819 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.820 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.820 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.820 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-27T15:32:20.820 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-27T15:32:20.820 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.820 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.820 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.820 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.821 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.821 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.821 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.821 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.821 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.821 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.821 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.821 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.821 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-27T15:32:20.822 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-27T15:32:20.822 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.822 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.822 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.822 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.822 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.822 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.822 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.823 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.823 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.823 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.823 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.823 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.823 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-27T15:32:20.823 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-27T15:32:20.824 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.824 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.824 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.824 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.824 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.824 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.824 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.824 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.825 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.825 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.825 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.825 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.825 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-27T15:32:20.825 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-27T15:32:20.825 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.825 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.825 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.826 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.826 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.826 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.826 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.826 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.826 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.826 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.826 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.827 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.827 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-27T15:32:20.827 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-27T15:32:20.827 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.827 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.827 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.827 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.827 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.827 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.828 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.828 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.828 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.828 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.828 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.828 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.828 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-27T15:32:20.828 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-27T15:32:20.829 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.829 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.829 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.829 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.829 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.829 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.829 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.829 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.830 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.830 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.830 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.830 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.830 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-27T15:32:20.830 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-27T15:32:20.830 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.830 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.830 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.831 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.831 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.831 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.831 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.831 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.831 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.831 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.831 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.832 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.832 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_ENABLED": { 2023-11-27T15:32:20.832 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-27T15:32:20.832 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.832 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.832 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.832 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.832 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.832 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.833 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.833 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.833 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.833 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.833 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.833 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.833 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-27T15:32:20.833 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-27T15:32:20.834 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.834 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.834 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.834 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-27T15:32:20.834 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.834 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.834 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.834 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.835 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.835 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.835 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.835 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.835 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-27T15:32:20.835 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-27T15:32:20.835 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.835 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.835 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.836 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:20.836 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.836 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.836 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.836 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.836 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.836 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.836 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.837 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.837 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-27T15:32:20.837 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-27T15:32:20.837 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.837 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.837 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.837 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8", 2023-11-27T15:32:20.837 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.837 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.838 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.838 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.838 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.838 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.838 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.838 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.838 INFO:teuthology.orchestra.run.smithi008.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-27T15:32:20.838 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-27T15:32:20.839 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.839 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.839 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.839 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "45", 2023-11-27T15:32:20.839 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.839 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.839 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.839 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.839 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.840 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.840 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.840 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.840 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ACCESS_KEY": { 2023-11-27T15:32:20.840 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-27T15:32:20.840 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.840 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.840 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.840 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.841 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.841 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.841 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.841 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.841 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.841 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.841 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.841 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.841 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-27T15:32:20.842 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-27T15:32:20.842 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.842 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.842 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.842 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:20.842 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.842 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.842 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.842 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.843 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.843 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.843 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.843 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.843 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SECRET_KEY": { 2023-11-27T15:32:20.843 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-27T15:32:20.843 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.843 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.843 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.844 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.844 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.844 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.844 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.844 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.844 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.844 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.844 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.844 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.844 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SSL_VERIFY": { 2023-11-27T15:32:20.845 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-27T15:32:20.845 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.845 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.845 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.845 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.845 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.845 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.845 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.845 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.846 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.846 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.846 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.846 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.846 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-27T15:32:20.846 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-27T15:32:20.846 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.846 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.846 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.847 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:20.847 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.847 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.847 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.847 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.847 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.847 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.847 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.847 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.848 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-27T15:32:20.848 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-27T15:32:20.848 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.848 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.848 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.848 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:20.848 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.848 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.848 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.849 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.849 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.849 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.849 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.849 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.849 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-27T15:32:20.849 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-27T15:32:20.849 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.849 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.850 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.850 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:20.850 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.850 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.850 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.850 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.850 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.850 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.850 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.851 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.851 INFO:teuthology.orchestra.run.smithi008.stdout: "cross_origin_url": { 2023-11-27T15:32:20.851 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cross_origin_url", 2023-11-27T15:32:20.851 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.851 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.851 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.851 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.851 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.851 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.852 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.852 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.852 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.852 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.852 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.852 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.852 INFO:teuthology.orchestra.run.smithi008.stdout: "crt_file": { 2023-11-27T15:32:20.852 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crt_file", 2023-11-27T15:32:20.852 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.853 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.853 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.853 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.853 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.853 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.853 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.853 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.853 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.853 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.854 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.854 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.854 INFO:teuthology.orchestra.run.smithi008.stdout: "debug": { 2023-11-27T15:32:20.854 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "debug", 2023-11-27T15:32:20.854 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.854 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.854 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.854 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.854 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.855 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.855 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.855 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable/disable debug options", 2023-11-27T15:32:20.855 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.855 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.855 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.855 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.855 INFO:teuthology.orchestra.run.smithi008.stdout: "jwt_token_ttl": { 2023-11-27T15:32:20.855 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "jwt_token_ttl", 2023-11-27T15:32:20.856 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.856 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.856 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.856 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "28800", 2023-11-27T15:32:20.856 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.856 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.856 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.856 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.857 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.857 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.857 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.857 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.857 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2023-11-27T15:32:20.857 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2023-11-27T15:32:20.857 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.857 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.857 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.858 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.858 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.858 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.858 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.858 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.858 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.858 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.859 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.859 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.859 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.859 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.859 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.859 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.859 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.859 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.859 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.860 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.860 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.860 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.860 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.860 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.860 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.860 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.860 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.861 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.861 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.861 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.861 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.861 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.861 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.861 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.861 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.862 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.862 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.862 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.862 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.862 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.862 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.862 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.862 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.863 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.863 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.863 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.863 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.863 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.863 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.863 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.863 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.864 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.864 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.864 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.864 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.864 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.864 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.864 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.864 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.865 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.865 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.865 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.865 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.865 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.865 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.865 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.865 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.865 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.866 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.866 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.866 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.866 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.866 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.866 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.866 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.866 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.867 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.867 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.867 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.867 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.867 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.867 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.867 INFO:teuthology.orchestra.run.smithi008.stdout: "motd": { 2023-11-27T15:32:20.867 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "motd", 2023-11-27T15:32:20.868 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.868 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.868 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.868 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.868 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.868 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.868 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.868 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "The message of the day", 2023-11-27T15:32:20.868 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.869 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.869 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.869 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.869 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect_resolve_ip_addr": { 2023-11-27T15:32:20.869 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "redirect_resolve_ip_addr", 2023-11-27T15:32:20.869 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.869 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.869 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.870 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.870 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.870 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.870 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.870 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.870 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.870 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.870 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.871 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.871 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2023-11-27T15:32:20.871 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2023-11-27T15:32:20.871 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.871 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.871 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.871 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2023-11-27T15:32:20.871 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.871 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.872 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.872 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.872 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.872 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.872 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.872 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.872 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2023-11-27T15:32:20.872 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2023-11-27T15:32:20.873 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.873 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.873 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.873 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8080", 2023-11-27T15:32:20.873 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.873 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.873 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.873 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.873 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.874 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.874 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.874 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.874 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2023-11-27T15:32:20.874 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2023-11-27T15:32:20.874 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.874 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.874 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.875 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.875 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.875 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.875 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.875 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.875 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.875 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.875 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.875 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.876 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl_server_port": { 2023-11-27T15:32:20.876 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl_server_port", 2023-11-27T15:32:20.876 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.876 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.876 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.876 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8443", 2023-11-27T15:32:20.876 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.876 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.877 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.877 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.877 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.877 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.877 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.877 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.877 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2023-11-27T15:32:20.877 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2023-11-27T15:32:20.878 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.878 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.878 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.878 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "redirect", 2023-11-27T15:32:20.878 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.878 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.878 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.878 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.879 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect" 2023-11-27T15:32:20.879 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.879 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.879 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.879 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.879 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.879 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.879 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2023-11-27T15:32:20.880 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2023-11-27T15:32:20.880 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.880 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.880 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.880 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2023-11-27T15:32:20.880 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2023-11-27T15:32:20.880 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2023-11-27T15:32:20.880 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.881 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.881 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.881 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.881 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.881 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.881 INFO:teuthology.orchestra.run.smithi008.stdout: "url_prefix": { 2023-11-27T15:32:20.881 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url_prefix", 2023-11-27T15:32:20.881 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.881 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.882 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.882 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.882 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.882 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.882 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.882 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.882 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.882 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.883 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.883 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.883 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.883 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.883 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.883 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "devicehealth", 2023-11-27T15:32:20.883 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.883 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.884 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.884 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_monitoring": { 2023-11-27T15:32:20.884 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_monitoring", 2023-11-27T15:32:20.884 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.884 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.884 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.884 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.884 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.885 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.885 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.885 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "monitor device health metrics", 2023-11-27T15:32:20.885 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.885 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.885 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.885 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.885 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.885 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.886 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.886 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.886 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.886 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.886 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.886 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.886 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.886 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.887 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.887 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.887 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.887 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.887 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.887 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.887 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.887 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.888 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.888 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.888 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.888 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.888 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.888 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.888 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.888 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.889 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.889 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.889 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.889 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.889 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.889 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.889 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.889 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.890 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.890 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.890 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.890 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.890 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.890 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.890 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.890 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.891 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.891 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.891 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.891 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.891 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.891 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.891 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.891 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.892 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.892 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.892 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.892 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.892 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.892 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.892 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.892 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.893 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.893 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.893 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.893 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.893 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.893 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.893 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.893 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.893 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.894 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.894 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.894 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.894 INFO:teuthology.orchestra.run.smithi008.stdout: "mark_out_threshold": { 2023-11-27T15:32:20.894 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mark_out_threshold", 2023-11-27T15:32:20.894 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.894 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.895 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.895 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2419200", 2023-11-27T15:32:20.895 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.895 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.895 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.895 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-27T15:32:20.895 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.895 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.895 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.896 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.896 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_name": { 2023-11-27T15:32:20.896 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_name", 2023-11-27T15:32:20.896 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.896 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.896 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.896 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "device_health_metrics", 2023-11-27T15:32:20.896 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.897 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.897 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.897 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-27T15:32:20.897 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.897 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.897 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.897 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.897 INFO:teuthology.orchestra.run.smithi008.stdout: "retention_period": { 2023-11-27T15:32:20.898 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retention_period", 2023-11-27T15:32:20.898 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.898 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.898 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.898 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15552000", 2023-11-27T15:32:20.898 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.898 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.899 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.899 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain device health metrics", 2023-11-27T15:32:20.899 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.899 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.899 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.899 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.899 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_frequency": { 2023-11-27T15:32:20.899 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_frequency", 2023-11-27T15:32:20.900 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.900 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.900 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.900 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2023-11-27T15:32:20.900 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.900 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.900 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.900 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-27T15:32:20.900 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.901 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.901 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.901 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.901 INFO:teuthology.orchestra.run.smithi008.stdout: "self_heal": { 2023-11-27T15:32:20.901 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "self_heal", 2023-11-27T15:32:20.901 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.901 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.901 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.902 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:20.902 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.902 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.902 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.902 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-27T15:32:20.902 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.902 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.902 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.902 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.903 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:20.903 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:20.903 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.903 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.903 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.903 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:20.903 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.903 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.904 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.904 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and check device health", 2023-11-27T15:32:20.904 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.904 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.904 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.904 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.904 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_threshold": { 2023-11-27T15:32:20.904 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_threshold", 2023-11-27T15:32:20.904 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.905 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.905 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.905 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7257600", 2023-11-27T15:32:20.905 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.905 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.905 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.905 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-27T15:32:20.905 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.906 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.906 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.906 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.906 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.906 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.906 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.906 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "diskprediction_local", 2023-11-27T15:32:20.906 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.906 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.907 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.907 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.907 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.907 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.907 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.907 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.907 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.907 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.908 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.908 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.908 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.908 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.908 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.908 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.908 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.909 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.909 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.909 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.909 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.909 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.909 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.909 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.909 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.910 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.910 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.910 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.910 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.910 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.910 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.910 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.910 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.910 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.911 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.911 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.911 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.911 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.911 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.911 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.911 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.911 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.912 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.912 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.912 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.912 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.912 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.912 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.912 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.912 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.913 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.913 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.913 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.913 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.913 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.913 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.913 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.913 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.913 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.914 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.914 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.914 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.914 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.914 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.914 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.914 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.914 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.915 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.915 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.915 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.915 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.915 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.915 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.915 INFO:teuthology.orchestra.run.smithi008.stdout: "predict_interval": { 2023-11-27T15:32:20.915 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predict_interval", 2023-11-27T15:32:20.916 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.916 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.916 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.916 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2023-11-27T15:32:20.916 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.916 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.916 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.916 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.917 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.917 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.917 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.917 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.917 INFO:teuthology.orchestra.run.smithi008.stdout: "predictor_model": { 2023-11-27T15:32:20.917 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predictor_model", 2023-11-27T15:32:20.917 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.917 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.917 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.918 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "prophetstor", 2023-11-27T15:32:20.918 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.918 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.918 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.918 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.918 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.918 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.919 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.919 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.919 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:20.919 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:20.919 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.919 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.919 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.919 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:20.920 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.920 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.920 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.920 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.920 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.920 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.920 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.920 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.920 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.921 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.921 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.921 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "influx", 2023-11-27T15:32:20.921 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": false, 2023-11-27T15:32:20.921 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "influxdb python module not found", 2023-11-27T15:32:20.921 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.921 INFO:teuthology.orchestra.run.smithi008.stdout: "batch_size": { 2023-11-27T15:32:20.921 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "batch_size", 2023-11-27T15:32:20.922 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.922 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.922 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.922 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5000", 2023-11-27T15:32:20.922 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.922 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.922 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.922 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-27T15:32:20.923 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.923 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.923 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.923 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.923 INFO:teuthology.orchestra.run.smithi008.stdout: "database": { 2023-11-27T15:32:20.923 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "database", 2023-11-27T15:32:20.923 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.923 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.923 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.923 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "ceph", 2023-11-27T15:32:20.924 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.924 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.924 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.924 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-27T15:32:20.924 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.924 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.924 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.924 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.924 INFO:teuthology.orchestra.run.smithi008.stdout: "hostname": { 2023-11-27T15:32:20.925 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "hostname", 2023-11-27T15:32:20.925 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.925 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.925 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.925 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.925 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.925 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.925 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.925 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server hostname", 2023-11-27T15:32:20.926 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.926 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.926 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.926 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.926 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:20.926 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:20.926 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:20.926 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.926 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.927 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "30", 2023-11-27T15:32:20.927 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "5", 2023-11-27T15:32:20.927 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.927 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.927 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-27T15:32:20.927 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.927 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.927 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.927 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.928 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.928 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.928 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.928 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.928 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.928 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.928 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.928 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.928 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.929 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.929 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.929 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.929 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.929 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.929 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.929 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.929 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.930 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.930 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.930 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.930 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.930 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.930 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.930 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.930 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.930 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.931 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.931 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.931 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.931 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.931 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.931 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.931 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.931 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.931 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.931 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.932 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.932 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.932 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.932 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.932 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.932 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.932 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.932 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.933 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.933 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.933 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.933 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.933 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.933 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.933 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.933 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.934 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.934 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.934 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.934 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.934 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.934 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.934 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.934 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.935 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.935 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.935 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.935 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.935 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.935 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.935 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.935 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.936 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.936 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.936 INFO:teuthology.orchestra.run.smithi008.stdout: "password": { 2023-11-27T15:32:20.936 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "password", 2023-11-27T15:32:20.936 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.936 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.936 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.936 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.936 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.937 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.937 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.937 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "password of InfluxDB server user", 2023-11-27T15:32:20.937 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.937 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.937 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.937 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.937 INFO:teuthology.orchestra.run.smithi008.stdout: "port": { 2023-11-27T15:32:20.938 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "port", 2023-11-27T15:32:20.938 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.938 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.938 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.938 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8086", 2023-11-27T15:32:20.938 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.938 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.938 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.939 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server port", 2023-11-27T15:32:20.939 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.939 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.939 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.939 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.939 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2023-11-27T15:32:20.939 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2023-11-27T15:32:20.939 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.940 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.940 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.940 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "false", 2023-11-27T15:32:20.940 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.940 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.940 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.940 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:32:20.940 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.941 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.941 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.941 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.941 INFO:teuthology.orchestra.run.smithi008.stdout: "threads": { 2023-11-27T15:32:20.941 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threads", 2023-11-27T15:32:20.941 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.941 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.941 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.941 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:20.942 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:20.942 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "32", 2023-11-27T15:32:20.942 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.942 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-27T15:32:20.942 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.942 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.942 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.942 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.943 INFO:teuthology.orchestra.run.smithi008.stdout: "username": { 2023-11-27T15:32:20.943 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "username", 2023-11-27T15:32:20.943 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.943 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.943 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.943 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.943 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.943 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.943 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.944 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "username of InfluxDB server user", 2023-11-27T15:32:20.944 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.944 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.944 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.944 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.944 INFO:teuthology.orchestra.run.smithi008.stdout: "verify_ssl": { 2023-11-27T15:32:20.944 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "verify_ssl", 2023-11-27T15:32:20.944 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.945 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.945 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.945 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "true", 2023-11-27T15:32:20.945 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.945 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.945 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.945 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:32:20.945 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.945 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.946 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.946 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.946 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.946 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.946 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.946 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "insights", 2023-11-27T15:32:20.946 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.946 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.947 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.947 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.947 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.947 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.947 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.947 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.947 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.947 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.947 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.948 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.948 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.948 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.948 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.948 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.948 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.948 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.948 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.949 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.949 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.949 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.949 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.949 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.949 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.949 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.949 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.950 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.950 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.950 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.950 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.950 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.950 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.950 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.950 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.951 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.951 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.951 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.951 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.951 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.951 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.951 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.952 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.952 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.952 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.952 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.952 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.952 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.952 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.952 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.953 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.953 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.953 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.953 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.953 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.953 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.953 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.953 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.953 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.954 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.954 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.954 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.954 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.954 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.954 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.954 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.955 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.955 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.955 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.955 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.955 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.955 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.955 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.955 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.955 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.956 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.956 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "iostat", 2023-11-27T15:32:20.956 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.956 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.956 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.956 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.956 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.956 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.957 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.957 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.957 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.957 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.957 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.957 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.957 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.957 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.958 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.958 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.958 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.958 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.958 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.958 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.958 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.958 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.959 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.959 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.959 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.959 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.959 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.959 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.959 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.959 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.960 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.960 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.960 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.960 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.960 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.960 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.960 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.960 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.960 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.961 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.961 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.961 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.961 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.961 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.961 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.961 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.961 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.962 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.962 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.962 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.962 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.962 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.962 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.962 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.962 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.963 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.963 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.963 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.963 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.963 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.963 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.963 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.963 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.963 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.964 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.964 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.964 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.964 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.964 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.964 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.964 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.964 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.965 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.965 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.965 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.965 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.965 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "k8sevents", 2023-11-27T15:32:20.965 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.965 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.965 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.965 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_event_retention_days": { 2023-11-27T15:32:20.966 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ceph_event_retention_days", 2023-11-27T15:32:20.966 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.966 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.966 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.966 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7", 2023-11-27T15:32:20.966 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.967 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.967 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.967 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-27T15:32:20.967 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.967 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.967 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.967 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.967 INFO:teuthology.orchestra.run.smithi008.stdout: "config_check_secs": { 2023-11-27T15:32:20.968 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_check_secs", 2023-11-27T15:32:20.968 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.968 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.968 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:20.968 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:20.968 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "10", 2023-11-27T15:32:20.968 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.968 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.969 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-27T15:32:20.969 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.969 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.969 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.969 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.969 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.969 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.969 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.970 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.970 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.970 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.970 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.970 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.970 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.970 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.970 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.970 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.971 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.971 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.971 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.971 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.971 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.971 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.971 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.971 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.972 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.972 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.972 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.972 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.972 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.972 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.972 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.972 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.972 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.973 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.973 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.973 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.973 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.973 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.973 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.973 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.973 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.974 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.974 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.974 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.974 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.974 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.974 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.974 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.974 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.974 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.975 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.975 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.975 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.975 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.975 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.975 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.975 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.975 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.976 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.976 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.976 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.976 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.976 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.976 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.976 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.976 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.977 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.977 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.977 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.977 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.977 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.977 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.977 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.977 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.977 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.978 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.978 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.978 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "localpool", 2023-11-27T15:32:20.978 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.978 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.978 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.978 INFO:teuthology.orchestra.run.smithi008.stdout: "failure_domain": { 2023-11-27T15:32:20.978 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "failure_domain", 2023-11-27T15:32:20.979 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.979 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.979 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.979 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "host", 2023-11-27T15:32:20.979 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.979 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.979 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.979 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "failure domain for any created local pool", 2023-11-27T15:32:20.979 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-27T15:32:20.980 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.980 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.980 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.980 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.980 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.980 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.980 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.980 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.980 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.981 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.981 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.981 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.981 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.981 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.981 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.981 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.981 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.982 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.982 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.982 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.982 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.982 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.982 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.982 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.982 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:20.982 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:20.983 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.983 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.983 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.983 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.983 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.983 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.983 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.983 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.983 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.984 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.984 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.984 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.984 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:20.984 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:20.984 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.984 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.984 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.985 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:20.985 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.985 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.985 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.985 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.985 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.985 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.985 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.985 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.986 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.986 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.986 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.986 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.986 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.986 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.986 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.986 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:20.986 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:20.987 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:20.987 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.987 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.987 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:20.987 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.987 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.987 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.987 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.988 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.988 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.988 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.988 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.988 INFO:teuthology.orchestra.run.smithi008.stdout: "min_size": { 2023-11-27T15:32:20.988 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_size", 2023-11-27T15:32:20.988 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.989 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.989 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.989 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.989 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.989 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.989 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.989 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default min_size for any created local pool", 2023-11-27T15:32:20.989 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-27T15:32:20.989 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.990 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.990 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.990 INFO:teuthology.orchestra.run.smithi008.stdout: "num_rep": { 2023-11-27T15:32:20.990 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "num_rep", 2023-11-27T15:32:20.990 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.990 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.990 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.990 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3", 2023-11-27T15:32:20.991 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.991 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.991 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.991 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default replica count for any created local pool", 2023-11-27T15:32:20.991 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.991 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.991 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.991 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.992 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_num": { 2023-11-27T15:32:20.992 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_num", 2023-11-27T15:32:20.992 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:20.992 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.992 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.992 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "128", 2023-11-27T15:32:20.992 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.992 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.993 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.993 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default pg_num for any created local pool", 2023-11-27T15:32:20.993 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.993 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.993 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.993 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.993 INFO:teuthology.orchestra.run.smithi008.stdout: "prefix": { 2023-11-27T15:32:20.993 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prefix", 2023-11-27T15:32:20.993 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.994 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.994 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.994 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.994 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.994 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.994 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.994 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name prefix for any created local pool", 2023-11-27T15:32:20.994 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:20.995 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.995 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.995 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.995 INFO:teuthology.orchestra.run.smithi008.stdout: "subtree": { 2023-11-27T15:32:20.995 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "subtree", 2023-11-27T15:32:20.995 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.995 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.995 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.995 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "rack", 2023-11-27T15:32:20.996 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.996 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.996 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:20.996 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-27T15:32:20.996 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-27T15:32:20.996 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:20.996 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:20.996 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.996 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:20.997 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:20.997 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:20.997 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mds_autoscaler", 2023-11-27T15:32:20.997 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:20.997 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:20.997 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:20.997 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:20.997 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:20.998 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:20.998 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:20.998 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:20.998 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:20.998 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:20.998 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:20.998 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:20.999 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:20.999 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:20.999 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:20.999 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:20.999 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:20.999 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:20.999 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:20.999 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:20.999 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.000 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.000 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.000 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.000 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.000 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.000 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.000 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.000 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.001 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.001 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.001 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.001 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.001 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.001 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.001 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.001 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.001 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.002 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.002 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.002 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.002 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.002 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.002 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.002 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.002 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.003 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.003 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.003 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.003 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.003 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.003 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.003 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.003 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.003 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.004 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.004 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.004 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.004 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.004 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.004 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.004 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.004 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.005 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.005 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.005 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.005 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.005 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.005 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.005 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.005 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.006 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.006 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.006 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.006 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.006 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.006 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirroring", 2023-11-27T15:32:21.006 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.006 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.007 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.007 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.007 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.007 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.007 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.007 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.007 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.007 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.008 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.008 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.008 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.008 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.008 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.008 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.008 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.009 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.009 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.009 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.009 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.009 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.009 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.009 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.009 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.010 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.010 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.010 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.010 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.010 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.010 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.010 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.011 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.011 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.011 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.011 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.011 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.011 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.011 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.011 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.012 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.012 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.012 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.012 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.012 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.012 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.012 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.012 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.013 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.013 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.013 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.013 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.013 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.013 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.013 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.013 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.014 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.014 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.014 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.014 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.014 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.014 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.014 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.015 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.015 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.015 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.015 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.015 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.015 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.015 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.015 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.016 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.016 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.016 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.016 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.016 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.016 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "nfs", 2023-11-27T15:32:21.016 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.017 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.017 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.017 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.017 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.017 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.017 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.017 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.017 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.018 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.018 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.018 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.018 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.018 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.018 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.019 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.019 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.019 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.019 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.019 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.019 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.019 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.019 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.020 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.020 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.020 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.020 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.020 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.020 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.020 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.021 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.021 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.021 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.021 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.021 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.021 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.021 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.021 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.021 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.022 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.022 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.022 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.022 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.022 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.022 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.022 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.022 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.023 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.023 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.023 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.023 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.023 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.023 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.023 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.023 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.024 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.024 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.024 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.024 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.024 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.024 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.024 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.024 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.025 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.025 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.025 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.025 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.025 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.025 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.025 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.025 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.026 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.026 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.026 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.026 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.026 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.026 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2023-11-27T15:32:21.026 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.026 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.027 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.027 INFO:teuthology.orchestra.run.smithi008.stdout: "fail_fs": { 2023-11-27T15:32:21.027 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "fail_fs", 2023-11-27T15:32:21.027 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.027 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.027 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.027 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.027 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.028 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.028 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.028 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-27T15:32:21.028 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.028 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.028 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.028 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.028 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.029 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.029 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.029 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.029 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.029 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.029 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.029 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.029 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.030 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.030 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.030 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.030 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.030 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.030 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.030 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.030 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.031 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.031 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.031 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.031 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.031 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.031 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.031 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.031 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.032 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.032 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.032 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.032 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.032 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.032 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.032 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.032 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.033 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.033 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.033 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.033 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.033 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.033 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.033 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.033 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.034 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.034 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.034 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.034 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.034 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.034 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.034 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.034 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.034 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.035 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.035 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.035 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.035 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.035 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.035 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.035 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.035 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.035 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.036 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.036 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.036 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.036 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.036 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.036 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.036 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.036 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.037 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.037 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.037 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.037 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator": { 2023-11-27T15:32:21.037 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2023-11-27T15:32:21.037 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.037 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.037 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.038 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.038 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.038 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.038 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.038 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm", 2023-11-27T15:32:21.038 INFO:teuthology.orchestra.run.smithi008.stdout: "rook", 2023-11-27T15:32:21.038 INFO:teuthology.orchestra.run.smithi008.stdout: "test_orchestrator" 2023-11-27T15:32:21.038 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.039 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Orchestrator backend", 2023-11-27T15:32:21.039 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.039 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.039 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.039 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.039 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.039 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.039 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.040 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_perf_query", 2023-11-27T15:32:21.040 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.040 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.040 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.040 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.040 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.040 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.040 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.041 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.041 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.041 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.041 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.041 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.041 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.041 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.041 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.041 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.042 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.042 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.042 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.042 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.042 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.042 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.042 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.042 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.043 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.043 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.043 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.043 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.043 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.043 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.043 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.043 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.044 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.044 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.044 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.044 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.044 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.044 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.044 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.044 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.044 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.045 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.045 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.045 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.045 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.045 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.045 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.045 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.045 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.046 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.046 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.046 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.046 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.046 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.046 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.046 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.046 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.047 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.047 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.047 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.047 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.047 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.047 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.047 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.048 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.048 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.048 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.048 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.048 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.048 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.048 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.048 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.049 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.049 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.049 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.049 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.049 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_support", 2023-11-27T15:32:21.049 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.049 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.049 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.050 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.050 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.050 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.050 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.050 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.050 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.050 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.051 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.051 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.051 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.051 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.051 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.051 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.051 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.051 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.052 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.052 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.052 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.052 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.052 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.052 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.052 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.052 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.053 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.053 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.053 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.053 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.053 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.053 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.053 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.053 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.054 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.054 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.054 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.054 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.054 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.054 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.054 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.054 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.055 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.055 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.055 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.055 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.055 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.055 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.055 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.055 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.056 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.056 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.056 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.056 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.056 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.056 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.056 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.056 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.057 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.057 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.057 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.057 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.057 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.057 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.057 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.057 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.058 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.058 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.058 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.058 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.058 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.058 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.058 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.058 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.059 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.059 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.059 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_autoscaler", 2023-11-27T15:32:21.059 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.059 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.059 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.059 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.059 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.060 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.060 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.060 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.060 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.060 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.060 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.060 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.060 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.061 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.061 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.061 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.061 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.061 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.061 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.061 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.061 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.062 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.062 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.062 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.062 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.062 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.062 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.062 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.062 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.063 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.063 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.063 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.063 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.063 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.063 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.063 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.063 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.063 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.064 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.064 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.064 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.064 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.064 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.064 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.064 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.064 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.065 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.065 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.065 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.065 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.065 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.065 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.065 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.065 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.066 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.066 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.066 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.066 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.066 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.066 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.066 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.066 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.067 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.067 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.067 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.067 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.067 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.067 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.067 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.067 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.068 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.068 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.068 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.068 INFO:teuthology.orchestra.run.smithi008.stdout: "noautoscale": { 2023-11-27T15:32:21.068 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "noautoscale", 2023-11-27T15:32:21.068 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.068 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.069 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.069 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.069 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.069 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.069 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.069 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "global autoscale flag", 2023-11-27T15:32:21.069 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-27T15:32:21.069 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.069 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.070 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.070 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:21.070 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:21.070 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.070 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.070 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.070 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:21.070 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.071 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.071 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.071 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.071 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.071 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.071 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.071 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.071 INFO:teuthology.orchestra.run.smithi008.stdout: "threshold": { 2023-11-27T15:32:21.072 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threshold", 2023-11-27T15:32:21.072 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:21.072 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.072 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.072 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2023-11-27T15:32:21.072 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1.0", 2023-11-27T15:32:21.072 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.072 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.073 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "scaling threshold", 2023-11-27T15:32:21.073 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-27T15:32:21.073 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.073 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.073 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.073 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.073 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.073 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.074 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "progress", 2023-11-27T15:32:21.074 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.074 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.074 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.074 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_pg_recovery_event": { 2023-11-27T15:32:21.074 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_pg_recovery_event", 2023-11-27T15:32:21.074 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.075 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.075 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.075 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.075 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.075 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.075 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.075 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-27T15:32:21.075 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.075 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.076 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.076 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.076 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2023-11-27T15:32:21.076 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2023-11-27T15:32:21.076 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.076 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.076 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.076 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.077 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.077 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.077 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.077 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.077 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.077 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.077 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.077 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.078 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.078 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.078 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.078 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.078 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.078 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.078 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.078 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.079 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.079 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.079 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.079 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.079 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.079 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.079 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.079 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.080 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.080 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.080 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.080 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.080 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.080 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.080 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.080 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.080 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.081 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.081 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.081 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.081 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.081 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.081 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.081 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.081 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.081 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.082 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.082 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.082 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.082 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.082 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.082 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.082 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.083 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.083 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.083 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.083 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.083 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.083 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.083 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.083 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.084 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.084 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.084 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.084 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.084 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.084 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.084 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.084 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.085 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.085 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.085 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.085 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.085 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.085 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.085 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.085 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.086 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.086 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.086 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.086 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.086 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.086 INFO:teuthology.orchestra.run.smithi008.stdout: "max_completed_events": { 2023-11-27T15:32:21.086 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_completed_events", 2023-11-27T15:32:21.087 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.087 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.087 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.087 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "50", 2023-11-27T15:32:21.087 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.087 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.087 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.087 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "number of past completed events to remember", 2023-11-27T15:32:21.088 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.088 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.088 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.088 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.088 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:21.088 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:21.088 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.089 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.089 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.089 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:21.089 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.089 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.089 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.089 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long the module is going to sleep", 2023-11-27T15:32:21.089 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.090 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.090 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.090 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.090 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.090 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.090 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.090 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus", 2023-11-27T15:32:21.090 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.091 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.091 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.091 INFO:teuthology.orchestra.run.smithi008.stdout: "cache": { 2023-11-27T15:32:21.091 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cache", 2023-11-27T15:32:21.091 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.091 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.091 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.091 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.092 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.092 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.092 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.092 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.092 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.092 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.092 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.092 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.093 INFO:teuthology.orchestra.run.smithi008.stdout: "exclude_perf_counters": { 2023-11-27T15:32:21.093 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "exclude_perf_counters", 2023-11-27T15:32:21.093 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.093 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.093 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.093 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.093 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.093 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.094 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.094 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-27T15:32:21.094 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-27T15:32:21.094 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.094 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.094 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.094 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.095 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.095 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.095 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.095 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.095 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.095 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.095 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.095 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.096 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.096 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.096 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.096 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.096 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.096 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.096 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.096 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.097 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.097 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.097 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.097 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.097 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.097 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.097 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.097 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.098 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.098 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.098 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.098 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.098 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.098 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.098 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.099 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.099 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.099 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.099 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.099 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.099 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.099 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.099 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.100 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.100 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.100 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.100 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.100 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.100 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.100 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.100 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.101 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.101 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.101 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.101 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.101 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.101 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.101 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.101 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.102 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.102 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.102 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.102 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.102 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.102 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.102 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.102 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.103 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.103 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.103 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.103 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.103 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.103 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.103 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools": { 2023-11-27T15:32:21.103 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools", 2023-11-27T15:32:21.103 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.104 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.104 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.104 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.104 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.104 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.104 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.104 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.104 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.105 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.105 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.105 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.105 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-27T15:32:21.105 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-27T15:32:21.105 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.105 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.105 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.105 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300", 2023-11-27T15:32:21.106 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.106 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.106 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.106 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.106 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.106 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.106 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.106 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.107 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_interval": { 2023-11-27T15:32:21.107 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_interval", 2023-11-27T15:32:21.107 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:21.107 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.107 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.107 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15.0", 2023-11-27T15:32:21.107 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.107 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.108 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.108 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.108 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.108 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.108 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.108 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.108 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2023-11-27T15:32:21.108 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2023-11-27T15:32:21.108 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.109 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.109 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.109 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2023-11-27T15:32:21.109 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.109 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.109 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.109 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-27T15:32:21.109 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.110 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.110 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.110 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.110 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2023-11-27T15:32:21.110 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2023-11-27T15:32:21.110 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.110 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.110 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.111 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "9283", 2023-11-27T15:32:21.111 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.111 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.111 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.111 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-27T15:32:21.111 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.111 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.111 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.111 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.112 INFO:teuthology.orchestra.run.smithi008.stdout: "stale_cache_strategy": { 2023-11-27T15:32:21.112 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stale_cache_strategy", 2023-11-27T15:32:21.112 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.112 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.112 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.112 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "log", 2023-11-27T15:32:21.112 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.112 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.112 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.113 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.113 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.113 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.113 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.113 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.113 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2023-11-27T15:32:21.113 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2023-11-27T15:32:21.113 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.113 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.113 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.114 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "default", 2023-11-27T15:32:21.114 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.114 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.114 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.114 INFO:teuthology.orchestra.run.smithi008.stdout: "default", 2023-11-27T15:32:21.114 INFO:teuthology.orchestra.run.smithi008.stdout: "error" 2023-11-27T15:32:21.114 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.114 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.114 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.115 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.115 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.115 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.115 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2023-11-27T15:32:21.115 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2023-11-27T15:32:21.115 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.115 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.115 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.115 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2023-11-27T15:32:21.116 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2023-11-27T15:32:21.116 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2023-11-27T15:32:21.116 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.116 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.116 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.116 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.116 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.116 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.117 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.117 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.117 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.117 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_support", 2023-11-27T15:32:21.117 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.117 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.117 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.117 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.117 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.118 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.118 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.118 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.118 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.118 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.118 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.118 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.118 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.119 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.119 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.119 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.119 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.119 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.119 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.119 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.119 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.119 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.120 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.120 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.120 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.120 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.120 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.120 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.120 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.120 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.120 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.120 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.121 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.121 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.121 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.121 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.121 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.121 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.121 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.121 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.121 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.122 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.122 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.122 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.122 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.122 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.122 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.122 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.122 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.122 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.123 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.123 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.123 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.123 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.123 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.123 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.123 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.123 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.123 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.124 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.124 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.124 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.124 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.124 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.124 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.124 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.124 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.124 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.124 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.125 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.125 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.125 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.125 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.125 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_snap_create": { 2023-11-27T15:32:21.125 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_snap_create", 2023-11-27T15:32:21.125 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.125 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.125 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.126 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:21.126 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.126 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.126 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.126 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.126 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.126 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.126 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.127 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.127 INFO:teuthology.orchestra.run.smithi008.stdout: "mirror_snapshot_schedule": { 2023-11-27T15:32:21.127 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirror_snapshot_schedule", 2023-11-27T15:32:21.127 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.127 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.127 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.127 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.127 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.127 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.127 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.128 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.128 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.128 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.128 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.128 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.128 INFO:teuthology.orchestra.run.smithi008.stdout: "trash_purge_schedule": { 2023-11-27T15:32:21.128 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "trash_purge_schedule", 2023-11-27T15:32:21.128 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.129 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.129 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.129 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.129 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.129 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.129 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.129 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.129 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.129 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.130 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.130 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.130 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.130 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.130 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.130 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "restful", 2023-11-27T15:32:21.130 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.130 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.130 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.131 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_auth": { 2023-11-27T15:32:21.131 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_auth", 2023-11-27T15:32:21.131 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.131 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.131 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.131 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.131 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.131 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.132 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.132 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.132 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.132 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.132 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.132 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.132 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2023-11-27T15:32:21.132 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2023-11-27T15:32:21.132 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.133 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.133 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.133 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.133 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.133 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.133 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.133 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.133 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.133 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.134 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.134 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.134 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.134 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.134 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.134 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.134 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.134 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.134 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.135 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.135 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.135 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.135 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.135 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.135 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.135 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.135 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.135 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.135 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.136 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.136 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.136 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.136 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.136 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.136 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.136 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.136 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.136 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.137 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.137 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.137 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.137 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.137 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.137 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.137 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.137 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.137 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.138 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.138 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.138 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.138 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.138 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.139 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.139 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.139 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.139 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.139 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.139 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.139 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.139 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.139 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.140 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.140 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.140 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.140 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.140 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.140 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.140 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.140 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.140 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.141 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.141 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.141 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.141 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.141 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.141 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.141 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.141 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.141 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.142 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.142 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.142 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.142 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2023-11-27T15:32:21.142 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2023-11-27T15:32:21.142 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.142 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.142 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.142 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.143 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.143 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.143 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.143 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.143 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.143 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.143 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.143 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.143 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2023-11-27T15:32:21.143 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2023-11-27T15:32:21.144 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.144 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.144 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.144 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.144 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.144 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.144 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.144 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.144 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.145 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.145 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.145 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.145 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.145 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.145 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.145 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rgw", 2023-11-27T15:32:21.145 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.145 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.146 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.146 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.146 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.146 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.146 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.146 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.146 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.146 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.147 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.147 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.147 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.147 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.147 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.147 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.147 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.147 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.147 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.148 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.148 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.148 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.148 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.148 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.148 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.148 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.148 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.149 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.149 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.149 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.149 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.149 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.149 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.149 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.149 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.149 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.150 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.150 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.150 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.150 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.150 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.150 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.150 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.150 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.150 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.151 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.151 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.151 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.151 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.151 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.151 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.151 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.151 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.152 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.152 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.152 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.152 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.152 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.152 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.152 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.152 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.152 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.153 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.153 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.153 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.153 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.153 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.153 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.153 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.153 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.153 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.154 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.154 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.154 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.154 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.154 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.154 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rook", 2023-11-27T15:32:21.154 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.154 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.154 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.155 INFO:teuthology.orchestra.run.smithi008.stdout: "drive_group_interval": { 2023-11-27T15:32:21.155 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "drive_group_interval", 2023-11-27T15:32:21.155 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:21.155 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.155 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.155 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300.0", 2023-11-27T15:32:21.155 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.155 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.156 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.156 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-27T15:32:21.156 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.156 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.156 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.156 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.156 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.156 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.156 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.156 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.157 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.157 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.157 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.157 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.157 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.157 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.157 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.157 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.157 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.158 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.158 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.158 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.158 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.158 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.158 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.158 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.158 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.159 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.159 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.159 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.159 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.159 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.159 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.159 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.159 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.159 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.160 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.160 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.160 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.160 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.160 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.160 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.160 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.160 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.160 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.161 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.161 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.161 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.161 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.161 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.161 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.161 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.161 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.161 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.162 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.162 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.162 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.162 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.162 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.162 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.162 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.162 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "storage_class": { 2023-11-27T15:32:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "storage_class", 2023-11-27T15:32:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.165 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "local", 2023-11-27T15:32:21.165 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.165 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.165 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.165 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-27T15:32:21.165 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.165 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.165 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.165 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.166 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.166 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.166 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.166 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "selftest", 2023-11-27T15:32:21.166 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.166 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.166 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.166 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.166 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.167 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.167 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.167 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.167 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.167 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.167 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.167 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.167 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.168 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.168 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.168 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.168 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.168 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.168 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.168 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.168 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.168 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.169 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.169 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.169 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.169 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.169 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.169 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.169 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.169 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.169 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.170 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.170 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.170 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.170 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.170 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.170 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.170 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.170 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.170 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.171 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.171 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.171 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.171 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.171 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.171 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.171 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.171 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.172 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.172 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.172 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.172 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.172 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.172 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.172 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.172 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.172 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.173 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.173 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.173 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.173 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.173 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.173 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.173 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.173 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.173 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.173 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.174 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.174 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.174 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.174 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.174 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.174 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.174 INFO:teuthology.orchestra.run.smithi008.stdout: "roption1": { 2023-11-27T15:32:21.174 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption1", 2023-11-27T15:32:21.175 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.175 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.175 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.175 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.175 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.175 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.175 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.175 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.176 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.176 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.176 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.176 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.176 INFO:teuthology.orchestra.run.smithi008.stdout: "roption2": { 2023-11-27T15:32:21.176 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption2", 2023-11-27T15:32:21.176 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.176 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.177 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.177 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "xyz", 2023-11-27T15:32:21.177 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.177 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.177 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.177 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.177 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.177 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.178 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.178 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.178 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption1": { 2023-11-27T15:32:21.178 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption1", 2023-11-27T15:32:21.178 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.178 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.178 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.179 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.179 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.179 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.179 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.179 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.179 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.179 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.179 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.179 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.180 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption2": { 2023-11-27T15:32:21.180 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption2", 2023-11-27T15:32:21.180 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.180 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.180 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.180 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.180 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.180 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.180 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.181 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.181 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.181 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.181 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.181 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.181 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption3": { 2023-11-27T15:32:21.181 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption3", 2023-11-27T15:32:21.181 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:21.181 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.182 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.182 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.182 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.182 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.182 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.182 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.182 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.182 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.182 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.183 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.183 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption4": { 2023-11-27T15:32:21.183 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption4", 2023-11-27T15:32:21.183 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.183 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.183 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.183 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.183 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.184 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.184 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.184 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.184 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.184 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.184 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.184 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.184 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption5": { 2023-11-27T15:32:21.184 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption5", 2023-11-27T15:32:21.185 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.185 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.185 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.185 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.185 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.185 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.185 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.185 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.185 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.185 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.186 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.186 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.186 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption6": { 2023-11-27T15:32:21.186 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption6", 2023-11-27T15:32:21.186 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.186 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.186 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.186 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.186 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.187 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.187 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.187 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.187 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.187 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.187 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.187 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.187 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption7": { 2023-11-27T15:32:21.188 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption7", 2023-11-27T15:32:21.188 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.188 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.188 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.188 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.188 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:21.188 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "42", 2023-11-27T15:32:21.188 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.188 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.189 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.189 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.189 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.189 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.189 INFO:teuthology.orchestra.run.smithi008.stdout: "testkey": { 2023-11-27T15:32:21.189 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testkey", 2023-11-27T15:32:21.189 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.189 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.189 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.190 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.190 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.190 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.190 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.190 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.190 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.190 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.190 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.191 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.191 INFO:teuthology.orchestra.run.smithi008.stdout: "testlkey": { 2023-11-27T15:32:21.191 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testlkey", 2023-11-27T15:32:21.191 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.191 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.191 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.191 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.191 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.191 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.192 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.192 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.192 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.192 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.192 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.192 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.192 INFO:teuthology.orchestra.run.smithi008.stdout: "testnewline": { 2023-11-27T15:32:21.192 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testnewline", 2023-11-27T15:32:21.193 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.193 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.193 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.193 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.193 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.193 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.193 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.193 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.193 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.194 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.194 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.194 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.194 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.194 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.194 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.194 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snap_schedule", 2023-11-27T15:32:21.194 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.195 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.195 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.195 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_m_granularity": { 2023-11-27T15:32:21.195 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_m_granularity", 2023-11-27T15:32:21.195 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.195 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.195 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.195 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.196 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.196 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.196 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.196 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow minute scheduled snapshots", 2023-11-27T15:32:21.196 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.196 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.196 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.196 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.196 INFO:teuthology.orchestra.run.smithi008.stdout: "dump_on_update": { 2023-11-27T15:32:21.197 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dump_on_update", 2023-11-27T15:32:21.197 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.197 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.197 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.197 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.197 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.197 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.197 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.198 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "dump database to debug log on update", 2023-11-27T15:32:21.198 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.198 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.198 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.198 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.198 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.198 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.198 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.199 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.199 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.199 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.199 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.199 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.199 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.199 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.199 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.199 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.200 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.200 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.200 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.200 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.200 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.200 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.200 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.200 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.200 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.201 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.201 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.201 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.201 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.201 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.201 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.201 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.201 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.202 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.202 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.202 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.202 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.202 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.202 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.202 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.202 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.202 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.203 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.203 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.203 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.203 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.203 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.203 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.203 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.203 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.203 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.204 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.204 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.204 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.204 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.204 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.204 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.204 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.204 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.204 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.205 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.205 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.205 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.205 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.205 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.205 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.205 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.205 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.206 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.206 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.206 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.206 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.206 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.206 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.206 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.206 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.206 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.207 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stats", 2023-11-27T15:32:21.207 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.207 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.207 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.207 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.207 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.207 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.207 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.207 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.208 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.208 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.208 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.208 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.208 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.208 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.208 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.208 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.209 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.209 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.209 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.209 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.209 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.209 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.209 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.209 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.209 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.210 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.210 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.210 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.210 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.210 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.210 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.210 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.210 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.210 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.211 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.211 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.211 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.211 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.211 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.211 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.211 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.211 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.211 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.212 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.212 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.212 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.212 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.212 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.212 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.212 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.212 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.213 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.213 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.213 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.213 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.213 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.213 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.213 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.213 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.213 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.214 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.214 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.214 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.214 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.214 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.214 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.214 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.214 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.215 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.215 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.215 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.215 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.215 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.215 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.215 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.215 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.215 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "status", 2023-11-27T15:32:21.216 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.216 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.216 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.216 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.216 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.216 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.216 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.216 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.216 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.217 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.217 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.217 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.217 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.217 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.217 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.217 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.217 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.218 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.218 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.218 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.218 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.218 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.218 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.218 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.219 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.219 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.219 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.219 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.219 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.219 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.219 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.219 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.219 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.220 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.220 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.220 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.220 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.220 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.220 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.220 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.220 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.220 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.220 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.221 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.221 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.221 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.221 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.221 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.221 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.221 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.221 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.221 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.222 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.222 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.222 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.222 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.222 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.222 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.222 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.222 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.222 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.223 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.223 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.223 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.223 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.223 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.223 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.223 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.223 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.223 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.224 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.224 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.224 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.224 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.224 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.224 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.224 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telegraf", 2023-11-27T15:32:21.224 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.224 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.225 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.225 INFO:teuthology.orchestra.run.smithi008.stdout: "address": { 2023-11-27T15:32:21.225 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "address", 2023-11-27T15:32:21.225 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.225 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.225 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.225 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-27T15:32:21.225 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.225 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.225 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.226 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.226 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.226 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.226 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.226 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.226 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:21.226 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:21.226 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.226 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.227 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.227 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15", 2023-11-27T15:32:21.227 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.227 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.227 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.227 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.227 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.227 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.227 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.228 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.228 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.228 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.228 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.228 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.228 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.228 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.228 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.229 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.229 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.229 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.229 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.229 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.229 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.229 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.229 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.230 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.230 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.230 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.230 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.230 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.230 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.230 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.230 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.231 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.231 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.231 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.231 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.231 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.231 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.231 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.231 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.231 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.231 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.232 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.232 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.232 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.232 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.232 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.232 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.232 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.232 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.232 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.233 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.233 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.233 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.233 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.233 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.233 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.233 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.233 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.233 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.234 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.234 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.234 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.234 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.234 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.234 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.234 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.234 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.234 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.235 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.235 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.235 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.235 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.235 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.235 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.235 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.235 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.235 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.235 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telemetry", 2023-11-27T15:32:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_basic": { 2023-11-27T15:32:21.236 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_basic", 2023-11-27T15:32:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.237 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.238 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-27T15:32:21.238 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.238 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.238 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.238 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.238 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_crash": { 2023-11-27T15:32:21.238 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_crash", 2023-11-27T15:32:21.238 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.239 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.239 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.239 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.239 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.239 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.239 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.239 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-27T15:32:21.239 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.239 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.240 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.240 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.240 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_device": { 2023-11-27T15:32:21.240 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_device", 2023-11-27T15:32:21.240 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.240 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.240 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.240 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.241 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.241 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.253 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.253 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-27T15:32:21.253 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.253 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.253 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.253 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.253 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_ident": { 2023-11-27T15:32:21.253 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_ident", 2023-11-27T15:32:21.254 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.254 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.254 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.254 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.254 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.254 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.254 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.254 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-27T15:32:21.255 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.255 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.255 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.255 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.255 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_perf": { 2023-11-27T15:32:21.255 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_perf", 2023-11-27T15:32:21.255 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.255 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.256 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.256 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.256 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.256 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.256 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.256 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-27T15:32:21.256 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.256 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.256 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.257 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.257 INFO:teuthology.orchestra.run.smithi008.stdout: "contact": { 2023-11-27T15:32:21.257 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "contact", 2023-11-27T15:32:21.257 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.257 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.257 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.257 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.257 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.257 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.258 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.258 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.258 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.258 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.258 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.258 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.258 INFO:teuthology.orchestra.run.smithi008.stdout: "description": { 2023-11-27T15:32:21.258 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "description", 2023-11-27T15:32:21.258 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.259 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.259 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.259 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.259 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.259 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.259 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.259 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.259 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.259 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.259 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.260 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.260 INFO:teuthology.orchestra.run.smithi008.stdout: "device_url": { 2023-11-27T15:32:21.260 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_url", 2023-11-27T15:32:21.260 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.260 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.260 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.260 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-27T15:32:21.260 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.260 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.261 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.261 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.261 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.261 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.261 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.261 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.261 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2023-11-27T15:32:21.261 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2023-11-27T15:32:21.261 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.261 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.262 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.262 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.262 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.262 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.262 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.262 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.262 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.262 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.262 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.263 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.263 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:21.263 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:21.263 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.263 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.263 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.263 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "24", 2023-11-27T15:32:21.263 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "8", 2023-11-27T15:32:21.264 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.264 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.264 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.264 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.264 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.264 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.264 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.264 INFO:teuthology.orchestra.run.smithi008.stdout: "last_opt_revision": { 2023-11-27T15:32:21.264 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "last_opt_revision", 2023-11-27T15:32:21.264 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.265 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.265 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.265 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1", 2023-11-27T15:32:21.265 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.265 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.265 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.265 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.265 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.265 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.266 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.266 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.266 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard": { 2023-11-27T15:32:21.266 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard", 2023-11-27T15:32:21.266 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.266 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.266 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.266 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.266 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.266 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.267 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.267 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.267 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.267 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.267 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.267 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.267 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard_description": { 2023-11-27T15:32:21.267 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard_description", 2023-11-27T15:32:21.267 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.268 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.268 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.268 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.268 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.268 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.268 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.268 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.268 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.268 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.269 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.269 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.269 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.269 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.269 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.269 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.269 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.269 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.269 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.269 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.270 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.270 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.270 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.270 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.270 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.270 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.270 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.270 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.270 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.271 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.271 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.271 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.271 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.271 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.271 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.271 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.271 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.271 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.271 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.272 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.272 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.272 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.272 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.272 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.272 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.272 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.272 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.273 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.273 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.273 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.273 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.273 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.273 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.273 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.273 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.273 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.274 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.275 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.276 INFO:teuthology.orchestra.run.smithi008.stdout: "organization": { 2023-11-27T15:32:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "organization", 2023-11-27T15:32:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.277 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: "proxy": { 2023-11-27T15:32:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "proxy", 2023-11-27T15:32:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.278 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.279 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.279 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.279 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.279 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.280 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.280 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.280 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.280 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.280 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.280 INFO:teuthology.orchestra.run.smithi008.stdout: "url": { 2023-11-27T15:32:21.280 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url", 2023-11-27T15:32:21.280 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.280 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.280 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.281 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-27T15:32:21.281 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.281 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.281 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.281 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.281 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.281 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.281 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.281 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.282 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.282 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.282 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.282 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "test_orchestrator", 2023-11-27T15:32:21.282 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.282 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.282 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.282 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.282 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.283 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.283 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.283 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.283 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.283 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.283 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.283 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.283 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.283 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.284 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.284 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.284 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.284 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.284 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.284 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.284 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.284 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.284 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.284 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.285 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.285 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.285 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.285 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.285 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.285 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.285 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.285 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.285 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.285 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.286 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.286 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.286 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.286 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.286 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.286 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.286 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.286 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.286 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.287 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.287 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.287 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.287 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.287 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.287 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.287 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.287 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.287 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.287 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.288 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.288 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.288 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.288 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.288 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.288 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.288 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.288 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.288 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.289 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.289 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.289 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.289 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.289 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.289 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.289 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.289 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.289 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.290 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.290 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.290 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.290 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.290 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.290 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "volumes", 2023-11-27T15:32:21.290 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.290 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.290 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.290 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.291 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.291 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.291 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.291 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.291 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.291 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.291 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.291 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.291 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.292 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.292 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.292 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.292 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.292 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.292 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.292 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.292 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.292 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.292 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.293 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.293 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.293 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.293 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.293 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.293 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.293 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.293 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.293 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.294 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.294 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.294 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.294 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.294 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.294 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.294 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.294 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.294 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.294 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.295 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.295 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.295 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.295 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.295 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.295 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.295 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.295 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.295 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.296 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.296 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.296 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.296 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.296 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.296 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.296 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.296 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.296 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.296 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.297 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.297 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.297 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.297 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.297 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.297 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.297 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.297 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.297 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.298 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.298 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.298 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.298 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_clones": { 2023-11-27T15:32:21.298 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_clones", 2023-11-27T15:32:21.298 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.298 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.298 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.298 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4", 2023-11-27T15:32:21.299 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.299 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.299 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.435 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-27T15:32:21.435 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.435 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.435 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.435 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.436 INFO:teuthology.orchestra.run.smithi008.stdout: "snapshot_clone_delay": { 2023-11-27T15:32:21.436 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snapshot_clone_delay", 2023-11-27T15:32:21.436 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.436 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.436 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.436 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:21.436 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.436 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.436 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.437 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-27T15:32:21.437 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.437 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.437 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.437 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.437 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.437 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.437 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.437 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix", 2023-11-27T15:32:21.437 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.438 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.438 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.438 INFO:teuthology.orchestra.run.smithi008.stdout: "discovery_interval": { 2023-11-27T15:32:21.438 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "discovery_interval", 2023-11-27T15:32:21.438 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:21.438 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.438 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.438 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "100", 2023-11-27T15:32:21.439 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.439 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.439 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.439 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.439 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.439 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.439 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.439 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.439 INFO:teuthology.orchestra.run.smithi008.stdout: "identifier": { 2023-11-27T15:32:21.440 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "identifier", 2023-11-27T15:32:21.440 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.440 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.440 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.440 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.440 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.440 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.440 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.440 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.441 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.441 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.441 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.441 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.441 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:21.441 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:21.441 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.441 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.441 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.441 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:21.442 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.442 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.442 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.442 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.442 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.442 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.442 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.442 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.442 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.443 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.443 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.443 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.443 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.443 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.443 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.443 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.443 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.443 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.444 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.444 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.444 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.444 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.444 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.444 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.444 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.444 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.444 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.444 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.445 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.445 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.445 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.445 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.445 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.445 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.445 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.445 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.445 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.446 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.446 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.446 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.446 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.446 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.446 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.446 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.446 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.446 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.446 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.447 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.447 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.447 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.447 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.447 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.447 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.447 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.447 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.447 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.448 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.448 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.448 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.448 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.448 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.448 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.448 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.448 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.448 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.449 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.449 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.449 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.449 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.449 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.449 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.449 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.449 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.449 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.450 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.450 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.450 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.450 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.450 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_host": { 2023-11-27T15:32:21.450 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_host", 2023-11-27T15:32:21.450 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.450 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.450 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.451 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.451 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.451 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.451 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.451 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.451 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.451 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.451 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.451 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.452 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_port": { 2023-11-27T15:32:21.452 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_port", 2023-11-27T15:32:21.452 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.452 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.452 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.452 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10051", 2023-11-27T15:32:21.452 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.452 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.452 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.453 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.453 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.453 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.453 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.453 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.453 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_sender": { 2023-11-27T15:32:21.453 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_sender", 2023-11-27T15:32:21.453 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.453 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.453 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.454 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-27T15:32:21.454 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.454 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.454 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.454 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.454 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.454 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.454 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.454 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.455 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.455 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.455 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2023-11-27T15:32:21.455 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.455 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.455 INFO:teuthology.orchestra.run.smithi008.stdout: "gid": 24113, 2023-11-27T15:32:21.455 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "y", 2023-11-27T15:32:21.455 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr_features": 4540138322906710015, 2023-11-27T15:32:21.455 INFO:teuthology.orchestra.run.smithi008.stdout: "available_modules": [ 2023-11-27T15:32:21.456 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.456 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alerts", 2023-11-27T15:32:21.456 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.456 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.456 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.456 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:21.456 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:21.456 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.456 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.457 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.457 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:21.457 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.457 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.457 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.457 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How frequently to reexamine health status", 2023-11-27T15:32:21.457 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.457 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.458 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.458 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.458 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.458 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.458 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.458 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.458 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.458 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.458 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.459 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.459 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.459 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.459 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.459 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.459 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.459 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.459 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.459 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.460 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.460 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.460 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.460 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.460 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.460 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.460 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.460 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.460 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.461 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.461 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.461 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.461 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.461 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.461 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.461 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.461 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.461 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.461 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.462 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.462 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.462 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.462 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.462 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.462 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.462 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.462 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.462 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.463 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.463 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.463 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.463 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.463 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.463 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.463 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.463 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.463 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.464 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.464 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.464 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.464 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.464 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.464 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.464 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.464 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.464 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.465 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.465 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.465 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.465 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.465 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.465 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.465 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.465 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.465 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_destination": { 2023-11-27T15:32:21.466 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_destination", 2023-11-27T15:32:21.466 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.466 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.466 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.466 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.466 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.466 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.466 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.466 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email address to send alerts to", 2023-11-27T15:32:21.467 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.467 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.467 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.467 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.467 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_from_name": { 2023-11-27T15:32:21.467 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_from_name", 2023-11-27T15:32:21.467 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.467 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.468 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.468 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "Ceph", 2023-11-27T15:32:21.468 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.468 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.468 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.468 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email From: name", 2023-11-27T15:32:21.468 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.468 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.468 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.468 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.469 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_host": { 2023-11-27T15:32:21.469 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_host", 2023-11-27T15:32:21.469 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.469 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.469 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.469 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.469 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.469 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.469 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.470 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP server", 2023-11-27T15:32:21.470 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.470 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.470 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.470 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.470 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_password": { 2023-11-27T15:32:21.470 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_password", 2023-11-27T15:32:21.470 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.470 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.471 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.471 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.471 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.471 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.471 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.471 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Password to authenticate with", 2023-11-27T15:32:21.471 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.471 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.471 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.472 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.472 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_port": { 2023-11-27T15:32:21.472 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_port", 2023-11-27T15:32:21.472 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.472 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.472 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.472 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "465", 2023-11-27T15:32:21.472 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.472 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.473 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.473 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP port", 2023-11-27T15:32:21.473 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.473 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.473 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.473 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.473 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_sender": { 2023-11-27T15:32:21.473 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_sender", 2023-11-27T15:32:21.474 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.474 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.474 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.474 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.474 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.474 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.474 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.474 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP envelope sender", 2023-11-27T15:32:21.474 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.475 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.475 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.475 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.475 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_ssl": { 2023-11-27T15:32:21.475 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_ssl", 2023-11-27T15:32:21.475 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.475 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.475 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.476 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.476 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.476 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.476 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.476 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-27T15:32:21.476 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.476 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.476 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.476 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.476 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_user": { 2023-11-27T15:32:21.477 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_user", 2023-11-27T15:32:21.477 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.477 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.477 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.477 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.477 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.477 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.477 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.477 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "User to authenticate as", 2023-11-27T15:32:21.478 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.478 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.478 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.478 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.478 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.478 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.478 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.478 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "balancer", 2023-11-27T15:32:21.479 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.479 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.479 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.479 INFO:teuthology.orchestra.run.smithi008.stdout: "active": { 2023-11-27T15:32:21.479 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "active", 2023-11-27T15:32:21.479 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.479 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.479 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.479 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.480 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.480 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.480 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.480 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically balance PGs across cluster", 2023-11-27T15:32:21.480 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.480 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.480 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.480 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.481 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_time": { 2023-11-27T15:32:21.481 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_time", 2023-11-27T15:32:21.481 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.481 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.481 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.481 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0000", 2023-11-27T15:32:21.481 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.481 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.481 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.481 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "beginning time of day to automatically balance", 2023-11-27T15:32:21.482 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:32:21.482 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.482 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.482 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.482 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_weekday": { 2023-11-27T15:32:21.482 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_weekday", 2023-11-27T15:32:21.482 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:21.482 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.483 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.483 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:21.483 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2023-11-27T15:32:21.483 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2023-11-27T15:32:21.483 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.483 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-27T15:32:21.483 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:32:21.483 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.483 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.484 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.484 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_max_iterations": { 2023-11-27T15:32:21.484 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_max_iterations", 2023-11-27T15:32:21.484 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:21.484 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.484 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.484 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "25", 2023-11-27T15:32:21.484 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:21.484 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "250", 2023-11-27T15:32:21.485 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.485 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-27T15:32:21.485 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.485 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.485 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.485 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.485 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_metrics": { 2023-11-27T15:32:21.485 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_metrics", 2023-11-27T15:32:21.485 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.486 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.486 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.486 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "pgs,objects,bytes", 2023-11-27T15:32:21.486 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.486 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.486 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.487 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-27T15:32:21.487 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-27T15:32:21.487 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.487 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.487 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.487 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_step": { 2023-11-27T15:32:21.487 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_step", 2023-11-27T15:32:21.487 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:21.488 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.488 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.488 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.5", 2023-11-27T15:32:21.488 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0.001", 2023-11-27T15:32:21.488 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "0.999", 2023-11-27T15:32:21.488 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.488 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "aggressiveness of optimization", 2023-11-27T15:32:21.488 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-27T15:32:21.489 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.489 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.489 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.489 INFO:teuthology.orchestra.run.smithi008.stdout: "end_time": { 2023-11-27T15:32:21.489 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_time", 2023-11-27T15:32:21.489 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.489 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.489 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.489 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2359", 2023-11-27T15:32:21.490 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.490 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.490 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.490 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ending time of day to automatically balance", 2023-11-27T15:32:21.490 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:32:21.490 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.490 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.490 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.491 INFO:teuthology.orchestra.run.smithi008.stdout: "end_weekday": { 2023-11-27T15:32:21.491 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_weekday", 2023-11-27T15:32:21.491 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:21.491 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.491 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.491 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:21.491 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2023-11-27T15:32:21.491 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2023-11-27T15:32:21.491 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.492 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-27T15:32:21.492 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:32:21.492 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.492 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.492 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.492 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.492 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.492 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.493 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.493 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.493 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.493 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.493 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.493 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.493 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.493 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.494 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.494 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.494 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.494 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.494 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.494 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.494 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.494 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.494 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.495 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.495 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.495 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.495 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.495 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.495 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.495 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.495 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.496 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.496 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.496 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.496 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.496 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.496 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.496 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.496 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.496 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.497 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.497 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.497 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.497 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.497 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.497 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.497 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.497 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.497 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.498 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.498 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.498 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.498 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.498 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.498 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.498 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.498 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.499 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.499 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.499 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.499 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.499 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.499 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.499 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.499 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.499 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.500 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.500 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.500 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.500 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.500 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.500 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.500 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.500 INFO:teuthology.orchestra.run.smithi008.stdout: "min_score": { 2023-11-27T15:32:21.501 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_score", 2023-11-27T15:32:21.501 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:21.501 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.501 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.501 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:21.501 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.501 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.501 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.501 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-27T15:32:21.502 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.502 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.502 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.502 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.502 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2023-11-27T15:32:21.502 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2023-11-27T15:32:21.502 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.502 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.503 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.503 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "upmap", 2023-11-27T15:32:21.503 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.503 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.503 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.503 INFO:teuthology.orchestra.run.smithi008.stdout: "crush-compat", 2023-11-27T15:32:21.503 INFO:teuthology.orchestra.run.smithi008.stdout: "none", 2023-11-27T15:32:21.503 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap" 2023-11-27T15:32:21.504 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.504 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Balancer mode", 2023-11-27T15:32:21.504 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.504 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.504 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.504 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.504 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_ids": { 2023-11-27T15:32:21.504 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_ids", 2023-11-27T15:32:21.505 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.505 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.505 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.505 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.505 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.505 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.505 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.505 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-27T15:32:21.505 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.506 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.506 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.506 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.506 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:21.506 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:21.506 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.506 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.506 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.507 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:21.507 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.507 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.507 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.507 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-27T15:32:21.507 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.507 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.507 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.508 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.508 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_deviation": { 2023-11-27T15:32:21.508 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_deviation", 2023-11-27T15:32:21.508 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.508 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.508 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.508 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:21.508 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:21.508 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.508 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.509 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-27T15:32:21.509 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-27T15:32:21.509 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.509 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.509 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.509 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_optimizations": { 2023-11-27T15:32:21.509 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_optimizations", 2023-11-27T15:32:21.509 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:21.510 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.510 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.510 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:21.510 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.510 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.510 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.510 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-27T15:32:21.510 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.511 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.511 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.511 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.511 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.511 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.511 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.511 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cephadm", 2023-11-27T15:32:21.511 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.512 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.512 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.512 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_down_multiplier": { 2023-11-27T15:32:21.512 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_down_multiplier", 2023-11-27T15:32:21.512 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:21.512 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.512 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.512 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2023-11-27T15:32:21.512 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.513 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.513 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.513 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-27T15:32:21.513 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.513 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.513 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.513 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.513 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_refresh_rate": { 2023-11-27T15:32:21.513 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_refresh_rate", 2023-11-27T15:32:21.514 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.514 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.514 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.514 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "20", 2023-11-27T15:32:21.514 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.514 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.514 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.514 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-27T15:32:21.515 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.515 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.515 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.515 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.515 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_starting_port": { 2023-11-27T15:32:21.515 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_starting_port", 2023-11-27T15:32:21.515 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.515 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.515 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.516 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4721", 2023-11-27T15:32:21.516 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.516 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.516 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.516 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-27T15:32:21.516 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.516 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.516 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.517 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.517 INFO:teuthology.orchestra.run.smithi008.stdout: "alertmanager_web_password": { 2023-11-27T15:32:21.517 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alertmanager_web_password", 2023-11-27T15:32:21.517 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.517 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.517 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.517 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:21.517 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.518 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.518 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.518 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Alertmanager web password", 2023-11-27T15:32:21.518 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.518 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.518 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.518 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.519 INFO:teuthology.orchestra.run.smithi008.stdout: "alertmanager_web_user": { 2023-11-27T15:32:21.519 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alertmanager_web_user", 2023-11-27T15:32:21.519 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.519 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.519 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.519 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:21.519 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.519 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.519 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.520 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Alertmanager web user", 2023-11-27T15:32:21.520 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.520 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.520 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.520 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.520 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_ptrace": { 2023-11-27T15:32:21.520 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_ptrace", 2023-11-27T15:32:21.520 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.520 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.521 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.521 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.616 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.616 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.616 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.616 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-27T15:32:21.616 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-27T15:32:21.616 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.616 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.617 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.617 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_interval": { 2023-11-27T15:32:21.617 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_interval", 2023-11-27T15:32:21.617 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.617 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.617 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.617 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:21.617 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.617 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.618 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.618 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-27T15:32:21.618 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.618 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.618 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.618 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.618 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_memory_target_ratio": { 2023-11-27T15:32:21.618 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_memory_target_ratio", 2023-11-27T15:32:21.619 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:21.619 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.619 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.619 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.7", 2023-11-27T15:32:21.619 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.619 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.619 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.619 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-27T15:32:21.619 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.619 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.620 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.620 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.620 INFO:teuthology.orchestra.run.smithi008.stdout: "cgroups_split": { 2023-11-27T15:32:21.620 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cgroups_split", 2023-11-27T15:32:21.620 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.620 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.620 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.620 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.620 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.621 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.621 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.621 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-27T15:32:21.621 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.621 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.621 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.621 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.621 INFO:teuthology.orchestra.run.smithi008.stdout: "config_checks_enabled": { 2023-11-27T15:32:21.621 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_checks_enabled", 2023-11-27T15:32:21.621 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.622 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.622 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.622 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.622 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.622 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.622 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.622 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-27T15:32:21.622 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.622 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.623 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.623 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.623 INFO:teuthology.orchestra.run.smithi008.stdout: "config_dashboard": { 2023-11-27T15:32:21.623 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_dashboard", 2023-11-27T15:32:21.623 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.623 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.623 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.623 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.623 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.624 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.624 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.624 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-27T15:32:21.624 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.624 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.624 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.624 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.624 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_alertmanager": { 2023-11-27T15:32:21.624 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_alertmanager", 2023-11-27T15:32:21.625 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.625 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.625 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.625 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-27T15:32:21.625 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.625 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.625 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.625 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:21.625 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.625 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.626 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.626 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.626 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_base": { 2023-11-27T15:32:21.626 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_base", 2023-11-27T15:32:21.626 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.626 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.626 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.626 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-27T15:32:21.626 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.627 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.627 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.627 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Container image name, without the tag", 2023-11-27T15:32:21.627 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.627 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.627 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.627 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.627 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_elasticsearch": { 2023-11-27T15:32:21.627 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_elasticsearch", 2023-11-27T15:32:21.627 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.628 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.628 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.628 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-27T15:32:21.628 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.628 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.628 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.628 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "elasticsearch container image", 2023-11-27T15:32:21.628 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.628 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.629 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.629 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.629 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_grafana": { 2023-11-27T15:32:21.629 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_grafana", 2023-11-27T15:32:21.629 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.629 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.629 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.629 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-27T15:32:21.629 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.629 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.630 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.630 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:21.630 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.630 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.630 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.630 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.630 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_haproxy": { 2023-11-27T15:32:21.630 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_haproxy", 2023-11-27T15:32:21.630 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.631 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.631 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.631 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-27T15:32:21.631 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.631 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.631 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.631 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "HAproxy container image", 2023-11-27T15:32:21.631 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.631 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.631 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.632 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.632 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_agent": { 2023-11-27T15:32:21.632 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_agent", 2023-11-27T15:32:21.632 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.632 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.632 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.632 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-27T15:32:21.632 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.633 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.633 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.633 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger agent container image", 2023-11-27T15:32:21.633 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.633 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.633 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.633 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.633 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_collector": { 2023-11-27T15:32:21.633 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_collector", 2023-11-27T15:32:21.634 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.634 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.634 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.634 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-27T15:32:21.634 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.634 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.634 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.634 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger collector container image", 2023-11-27T15:32:21.634 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.634 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.635 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.635 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.635 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_query": { 2023-11-27T15:32:21.635 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_query", 2023-11-27T15:32:21.635 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.635 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.635 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.635 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-27T15:32:21.635 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.636 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.636 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.636 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger query container image", 2023-11-27T15:32:21.636 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.636 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.636 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.636 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.636 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_keepalived": { 2023-11-27T15:32:21.636 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_keepalived", 2023-11-27T15:32:21.637 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.637 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.637 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.637 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-27T15:32:21.637 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.637 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.637 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.637 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Keepalived container image", 2023-11-27T15:32:21.637 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.638 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.638 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.638 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.638 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_loki": { 2023-11-27T15:32:21.638 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_loki", 2023-11-27T15:32:21.638 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.638 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.638 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.638 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-27T15:32:21.638 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.639 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.639 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.639 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Loki container image", 2023-11-27T15:32:21.639 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.639 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.639 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.639 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.639 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_node_exporter": { 2023-11-27T15:32:21.639 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_node_exporter", 2023-11-27T15:32:21.640 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.640 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.640 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.640 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-27T15:32:21.640 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.640 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.640 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.640 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:21.640 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.641 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.641 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.641 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.641 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_prometheus": { 2023-11-27T15:32:21.641 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_prometheus", 2023-11-27T15:32:21.641 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.641 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.641 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.641 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-27T15:32:21.641 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.642 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.642 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.642 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:21.642 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.642 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.642 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.642 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.642 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_promtail": { 2023-11-27T15:32:21.642 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_promtail", 2023-11-27T15:32:21.643 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.643 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.643 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.643 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-27T15:32:21.643 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.643 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.643 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.643 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Promtail container image", 2023-11-27T15:32:21.643 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.644 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.644 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.644 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.644 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_snmp_gateway": { 2023-11-27T15:32:21.644 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_snmp_gateway", 2023-11-27T15:32:21.644 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.644 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.644 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.644 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-27T15:32:21.644 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.645 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.645 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.645 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SNMP Gateway container image", 2023-11-27T15:32:21.645 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.645 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.645 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.645 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.645 INFO:teuthology.orchestra.run.smithi008.stdout: "container_init": { 2023-11-27T15:32:21.645 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_init", 2023-11-27T15:32:21.645 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.646 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.646 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.646 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.646 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.646 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.646 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.646 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Run podman/docker with `--init`", 2023-11-27T15:32:21.646 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.646 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.647 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.647 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.647 INFO:teuthology.orchestra.run.smithi008.stdout: "daemon_cache_timeout": { 2023-11-27T15:32:21.647 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "daemon_cache_timeout", 2023-11-27T15:32:21.647 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.647 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.647 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.647 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:21.647 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.647 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.648 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.648 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-27T15:32:21.648 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.648 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.648 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.648 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.648 INFO:teuthology.orchestra.run.smithi008.stdout: "default_cephadm_command_timeout": { 2023-11-27T15:32:21.648 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_cephadm_command_timeout", 2023-11-27T15:32:21.648 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.649 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.649 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.649 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "900", 2023-11-27T15:32:21.649 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.649 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.649 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.649 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-27T15:32:21.649 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.649 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.650 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.650 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.650 INFO:teuthology.orchestra.run.smithi008.stdout: "default_registry": { 2023-11-27T15:32:21.650 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_registry", 2023-11-27T15:32:21.650 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.650 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.650 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.650 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io", 2023-11-27T15:32:21.650 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.650 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.651 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.651 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-27T15:32:21.651 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.651 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.651 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.651 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.651 INFO:teuthology.orchestra.run.smithi008.stdout: "device_cache_timeout": { 2023-11-27T15:32:21.651 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_cache_timeout", 2023-11-27T15:32:21.651 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.652 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.652 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.652 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1800", 2023-11-27T15:32:21.652 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.652 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.652 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.652 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache device inventory", 2023-11-27T15:32:21.652 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.652 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.653 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.653 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.653 INFO:teuthology.orchestra.run.smithi008.stdout: "device_enhanced_scan": { 2023-11-27T15:32:21.653 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_enhanced_scan", 2023-11-27T15:32:21.653 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.653 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.653 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.653 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.653 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.653 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.654 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.654 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-27T15:32:21.654 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.654 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.654 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.654 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.654 INFO:teuthology.orchestra.run.smithi008.stdout: "facts_cache_timeout": { 2023-11-27T15:32:21.654 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "facts_cache_timeout", 2023-11-27T15:32:21.654 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.655 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.655 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.655 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:21.655 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.655 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.655 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.655 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache host facts data", 2023-11-27T15:32:21.655 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.655 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.656 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.656 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.656 INFO:teuthology.orchestra.run.smithi008.stdout: "host_check_interval": { 2023-11-27T15:32:21.656 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "host_check_interval", 2023-11-27T15:32:21.656 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.656 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.656 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.656 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:21.656 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.656 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.657 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.657 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to perform a host check", 2023-11-27T15:32:21.657 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.657 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.657 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.657 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.657 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.657 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.657 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.658 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.658 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.658 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.658 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.658 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.658 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.658 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.658 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.658 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.659 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.659 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.659 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.659 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.659 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.659 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.659 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.659 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.659 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.660 INFO:teuthology.orchestra.run.smithi008.stdout: "log_refresh_metadata": { 2023-11-27T15:32:21.660 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_refresh_metadata", 2023-11-27T15:32:21.660 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.660 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.660 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.660 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.660 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.660 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.660 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.660 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-27T15:32:21.661 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.661 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.661 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.661 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.661 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.661 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.661 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.661 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.661 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.661 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.662 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.662 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.662 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.662 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-27T15:32:21.662 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.662 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.662 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.662 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.663 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.663 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.663 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.663 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.663 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.663 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.663 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.663 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.663 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.664 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.664 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.664 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.664 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.664 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.664 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.664 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.664 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.664 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.665 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.665 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.665 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.665 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.665 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.665 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.665 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.665 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.666 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.666 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.666 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.666 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.666 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.666 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.666 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.667 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.667 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.667 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-27T15:32:21.667 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-27T15:32:21.667 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.667 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.667 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.667 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.668 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.668 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.668 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.668 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-27T15:32:21.668 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.668 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.668 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.668 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.669 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-27T15:32:21.669 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-27T15:32:21.669 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.669 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.669 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.669 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "*", 2023-11-27T15:32:21.669 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.669 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.669 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.670 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-27T15:32:21.670 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.670 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.670 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.670 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.670 INFO:teuthology.orchestra.run.smithi008.stdout: "max_count_per_host": { 2023-11-27T15:32:21.670 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_count_per_host", 2023-11-27T15:32:21.670 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.671 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.671 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.671 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:21.671 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.671 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.671 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.671 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of daemons per service per host", 2023-11-27T15:32:21.671 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.672 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.672 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.672 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.672 INFO:teuthology.orchestra.run.smithi008.stdout: "max_osd_draining_count": { 2023-11-27T15:32:21.672 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_osd_draining_count", 2023-11-27T15:32:21.672 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.672 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.672 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.672 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:21.673 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.673 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.673 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.673 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-27T15:32:21.673 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.673 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.673 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.673 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.673 INFO:teuthology.orchestra.run.smithi008.stdout: "migration_current": { 2023-11-27T15:32:21.674 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "migration_current", 2023-11-27T15:32:21.674 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.674 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.674 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.674 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.674 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.674 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.674 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.675 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "internal - do not modify", 2023-11-27T15:32:21.675 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.675 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.675 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.675 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.675 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2023-11-27T15:32:21.676 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2023-11-27T15:32:21.676 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.676 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.676 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.676 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "root", 2023-11-27T15:32:21.676 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.676 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.677 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.677 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm-package", 2023-11-27T15:32:21.677 INFO:teuthology.orchestra.run.smithi008.stdout: "root" 2023-11-27T15:32:21.677 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.677 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "mode for remote execution of cephadm", 2023-11-27T15:32:21.677 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.677 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.677 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.677 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.678 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_alerts_path": { 2023-11-27T15:32:21.678 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_alerts_path", 2023-11-27T15:32:21.678 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.678 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.678 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.678 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-27T15:32:21.678 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.678 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.679 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.679 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-27T15:32:21.679 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.679 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.679 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.679 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.679 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_web_password": { 2023-11-27T15:32:21.679 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_web_password", 2023-11-27T15:32:21.680 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.680 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.680 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.680 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:21.680 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.680 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.680 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.680 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus web password", 2023-11-27T15:32:21.680 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.681 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.681 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.681 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.681 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_web_user": { 2023-11-27T15:32:21.681 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_web_user", 2023-11-27T15:32:21.681 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.681 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.681 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.681 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:21.682 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.682 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.682 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.682 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus web user", 2023-11-27T15:32:21.682 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.682 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.682 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.682 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.683 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_insecure": { 2023-11-27T15:32:21.683 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_insecure", 2023-11-27T15:32:21.683 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.683 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.683 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.683 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.683 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.683 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.684 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.684 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-27T15:32:21.684 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.684 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.684 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.684 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.684 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_password": { 2023-11-27T15:32:21.684 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_password", 2023-11-27T15:32:21.685 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.685 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.685 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.685 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.685 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.685 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.685 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.685 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-27T15:32:21.686 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.686 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.686 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.686 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.686 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_url": { 2023-11-27T15:32:21.686 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_url", 2023-11-27T15:32:21.686 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.686 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.687 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.687 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.687 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.687 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.687 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.687 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-27T15:32:21.687 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.687 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.687 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.688 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.688 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_username": { 2023-11-27T15:32:21.688 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_username", 2023-11-27T15:32:21.688 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.688 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.688 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.688 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.688 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.689 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.689 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.689 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-27T15:32:21.689 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.689 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.689 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.689 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.689 INFO:teuthology.orchestra.run.smithi008.stdout: "secure_monitoring_stack": { 2023-11-27T15:32:21.689 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "secure_monitoring_stack", 2023-11-27T15:32:21.690 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.690 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.690 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.690 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.690 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.690 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.690 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.691 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-27T15:32:21.691 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.691 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.691 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.691 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.691 INFO:teuthology.orchestra.run.smithi008.stdout: "service_discovery_port": { 2023-11-27T15:32:21.691 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "service_discovery_port", 2023-11-27T15:32:21.691 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.691 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.692 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.692 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8765", 2023-11-27T15:32:21.692 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.692 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.692 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.692 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "cephadm service discovery port", 2023-11-27T15:32:21.692 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.692 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.692 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.693 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.693 INFO:teuthology.orchestra.run.smithi008.stdout: "ssh_config_file": { 2023-11-27T15:32:21.693 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssh_config_file", 2023-11-27T15:32:21.693 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.693 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.693 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.693 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.694 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.694 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.694 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.694 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-27T15:32:21.694 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.694 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.694 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.694 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.694 INFO:teuthology.orchestra.run.smithi008.stdout: "use_agent": { 2023-11-27T15:32:21.695 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_agent", 2023-11-27T15:32:21.695 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.695 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.695 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.695 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.695 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.695 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.695 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.696 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-27T15:32:21.696 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.696 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.696 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.696 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.696 INFO:teuthology.orchestra.run.smithi008.stdout: "use_repo_digest": { 2023-11-27T15:32:21.696 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_repo_digest", 2023-11-27T15:32:21.696 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.697 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.697 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.697 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.697 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.697 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.697 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.697 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-27T15:32:21.697 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.698 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.698 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.698 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.698 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_failed_host_check": { 2023-11-27T15:32:21.698 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_failed_host_check", 2023-11-27T15:32:21.698 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.698 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.698 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.699 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.699 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.699 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.699 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.699 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if the host check fails", 2023-11-27T15:32:21.699 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.699 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.699 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.699 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.700 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_daemons": { 2023-11-27T15:32:21.700 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_daemons", 2023-11-27T15:32:21.700 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.700 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.700 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.700 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.700 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.700 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.700 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.701 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-27T15:32:21.701 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.701 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.701 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.701 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.701 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_hosts": { 2023-11-27T15:32:21.701 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_hosts", 2023-11-27T15:32:21.701 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.702 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.702 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.702 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.702 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.702 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.702 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.702 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-27T15:32:21.702 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.703 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.703 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.703 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.703 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.703 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.703 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.703 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crash", 2023-11-27T15:32:21.703 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.704 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.704 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.704 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.704 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.704 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.704 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.704 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.704 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.704 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.705 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.705 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.705 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.705 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.705 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.705 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.705 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.705 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.706 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.706 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.706 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.706 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.706 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.706 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.706 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.706 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.707 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.707 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.707 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.707 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.707 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.707 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.707 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.707 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.708 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.708 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.708 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.708 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.708 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.708 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.708 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.708 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.709 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.709 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.709 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.709 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.709 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.709 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.709 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.709 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.709 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.710 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.710 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.710 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.710 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.710 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.710 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.710 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.710 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.711 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.711 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.711 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.711 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.711 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.711 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.711 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.711 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.712 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.712 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.712 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.712 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.712 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.712 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.712 INFO:teuthology.orchestra.run.smithi008.stdout: "retain_interval": { 2023-11-27T15:32:21.712 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retain_interval", 2023-11-27T15:32:21.712 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.713 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.713 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.713 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "31536000", 2023-11-27T15:32:21.713 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.713 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.713 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.713 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-27T15:32:21.713 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.714 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.714 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.714 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.714 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_recent_interval": { 2023-11-27T15:32:21.714 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_recent_interval", 2023-11-27T15:32:21.714 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.714 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.714 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.715 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1209600", 2023-11-27T15:32:21.715 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.715 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.715 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.715 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-27T15:32:21.715 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.715 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.715 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.716 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.716 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.716 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.716 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.716 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dashboard", 2023-11-27T15:32:21.716 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.716 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.716 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.716 INFO:teuthology.orchestra.run.smithi008.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-27T15:32:21.717 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-27T15:32:21.717 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.717 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.717 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.717 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:21.717 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.717 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.717 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.718 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.718 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.718 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.718 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.718 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.718 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_HOST": { 2023-11-27T15:32:21.718 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-27T15:32:21.718 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.719 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.719 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.719 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.719 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.719 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.719 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.719 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.719 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.719 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.720 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.720 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.720 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-27T15:32:21.720 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-27T15:32:21.720 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.720 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.720 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.720 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.720 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.721 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.721 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.721 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.721 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.721 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.721 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.721 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.721 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_ENABLED": { 2023-11-27T15:32:21.722 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_ENABLED", 2023-11-27T15:32:21.722 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.722 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.722 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.722 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.722 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.722 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.722 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.723 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.723 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.723 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.723 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.723 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.723 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-27T15:32:21.723 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-27T15:32:21.723 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.723 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.724 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.724 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.724 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.724 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.724 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.724 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.724 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.724 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.724 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.725 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.725 INFO:teuthology.orchestra.run.smithi008.stdout: "ENABLE_BROWSABLE_API": { 2023-11-27T15:32:21.725 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-27T15:32:21.725 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.725 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.725 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.725 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.725 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.726 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.726 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.726 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.726 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.726 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.726 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.726 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.726 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-27T15:32:21.727 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-27T15:32:21.727 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.727 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.727 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.727 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.727 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.727 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.727 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.727 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.728 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.728 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.728 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.728 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.728 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-27T15:32:21.728 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-27T15:32:21.728 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.728 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.729 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.729 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.729 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.729 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.729 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.729 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.729 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.729 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.841 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.841 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.841 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-27T15:32:21.841 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-27T15:32:21.841 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.841 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.841 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.841 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.842 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.842 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.842 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.842 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.842 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.842 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.842 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.842 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.842 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-27T15:32:21.843 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-27T15:32:21.843 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.843 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.843 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.843 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.843 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.843 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.843 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.844 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.844 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.844 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.844 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.844 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.844 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-27T15:32:21.844 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-27T15:32:21.844 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.844 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.845 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.845 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.845 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.845 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.845 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.845 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.845 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.845 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.845 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.846 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.846 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-27T15:32:21.846 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-27T15:32:21.846 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.846 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.846 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.847 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.847 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.847 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.847 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.847 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.847 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.847 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.847 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.848 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.848 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-27T15:32:21.848 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-27T15:32:21.848 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.848 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.848 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.848 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.848 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.849 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.849 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.849 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.849 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.849 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.849 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.849 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.849 INFO:teuthology.orchestra.run.smithi008.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-27T15:32:21.849 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-27T15:32:21.850 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.850 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.850 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.850 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.850 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.850 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.850 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.850 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.851 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.851 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.851 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.851 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.851 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_PASSWORD": { 2023-11-27T15:32:21.851 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-27T15:32:21.851 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.851 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.852 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.852 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:21.852 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.852 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.852 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.852 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.852 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.852 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.853 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.853 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.853 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-27T15:32:21.853 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-27T15:32:21.853 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.853 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.853 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.853 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.854 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.854 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.854 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.854 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.854 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.854 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.854 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.854 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.855 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_URL": { 2023-11-27T15:32:21.855 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_URL", 2023-11-27T15:32:21.855 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.855 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.855 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.855 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.855 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.855 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.856 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.856 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.856 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.856 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.856 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.856 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.856 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_USERNAME": { 2023-11-27T15:32:21.856 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-27T15:32:21.857 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.857 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.857 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.857 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:21.857 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.857 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.857 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.857 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.857 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.858 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.858 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.858 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.858 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-27T15:32:21.858 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-27T15:32:21.858 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.858 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.858 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.859 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.859 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.859 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.859 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.859 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.859 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.859 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.859 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.859 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.860 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-27T15:32:21.860 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-27T15:32:21.860 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.860 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.860 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.860 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.860 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.860 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.861 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.861 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.861 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.861 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.861 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.861 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.861 INFO:teuthology.orchestra.run.smithi008.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-27T15:32:21.861 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-27T15:32:21.861 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.862 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.862 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.862 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.862 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.862 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.862 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.862 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.862 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.863 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.863 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.863 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.863 INFO:teuthology.orchestra.run.smithi008.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-27T15:32:21.863 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-27T15:32:21.863 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.863 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.863 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.864 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.864 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.864 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.864 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.864 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.864 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.864 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.864 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.864 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.865 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_HOST": { 2023-11-27T15:32:21.865 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-27T15:32:21.865 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.865 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.865 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.865 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.865 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.865 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.866 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.866 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.866 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.866 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.866 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.866 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.866 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-27T15:32:21.866 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-27T15:32:21.866 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.867 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.867 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.867 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.867 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.867 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.867 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.867 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.867 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.868 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.868 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.868 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.868 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-27T15:32:21.868 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-27T15:32:21.868 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.868 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.868 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.869 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.869 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.869 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.869 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.869 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.869 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.869 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.869 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-27T15:32:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-27T15:32:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-27T15:32:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-27T15:32:21.872 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.872 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.872 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.872 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.872 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.872 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.872 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.872 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.873 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.873 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.873 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.873 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.873 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-27T15:32:21.873 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-27T15:32:21.873 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.873 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.873 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.874 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.874 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.874 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.874 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.874 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.874 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.874 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.874 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.875 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.875 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-27T15:32:21.875 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-27T15:32:21.875 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.875 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.875 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.875 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.875 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.876 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.876 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.876 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.876 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.876 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.876 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.876 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.876 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-27T15:32:21.877 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-27T15:32:21.877 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.877 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.877 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.877 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.877 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.877 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.877 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.877 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.878 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.878 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.878 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.878 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.878 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-27T15:32:21.878 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-27T15:32:21.878 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.878 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.879 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.879 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.879 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.879 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.879 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.879 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.879 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.879 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.880 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.880 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.880 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_ENABLED": { 2023-11-27T15:32:21.880 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-27T15:32:21.880 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.880 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.880 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.880 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.880 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.881 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.881 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.881 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.881 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.881 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.881 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.881 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.881 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-27T15:32:21.882 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-27T15:32:21.882 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.882 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.882 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.882 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-27T15:32:21.882 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.882 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.882 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.883 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.883 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.883 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.883 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.883 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.883 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-27T15:32:21.883 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-27T15:32:21.883 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.883 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.884 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.884 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:21.884 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.884 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.884 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.884 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.884 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.884 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.885 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.885 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.885 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-27T15:32:21.885 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-27T15:32:21.885 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.885 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.885 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.885 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8", 2023-11-27T15:32:21.885 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.886 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.886 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.886 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.886 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.886 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.886 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.886 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.886 INFO:teuthology.orchestra.run.smithi008.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-27T15:32:21.887 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-27T15:32:21.887 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.887 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.887 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.887 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "45", 2023-11-27T15:32:21.887 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.887 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.887 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.888 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.888 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.888 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.888 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.888 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.888 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ACCESS_KEY": { 2023-11-27T15:32:21.888 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-27T15:32:21.888 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.888 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.889 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.889 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.889 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.889 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.889 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.889 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.889 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.889 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.890 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.890 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.890 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-27T15:32:21.890 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-27T15:32:21.890 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.890 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.890 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.890 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:21.891 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.891 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.891 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.891 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.891 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.891 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.891 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.891 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.891 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SECRET_KEY": { 2023-11-27T15:32:21.892 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-27T15:32:21.892 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.892 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.892 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.892 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.892 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.892 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.892 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.893 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.893 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.893 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.893 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.893 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.893 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SSL_VERIFY": { 2023-11-27T15:32:21.893 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-27T15:32:21.893 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.894 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.894 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.894 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.894 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.894 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.894 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.894 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.894 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.895 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.895 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.895 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.895 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-27T15:32:21.895 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-27T15:32:21.895 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.895 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.895 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.895 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:21.896 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.896 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.896 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.896 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.896 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.896 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.896 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.896 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.897 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-27T15:32:21.897 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-27T15:32:21.897 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.897 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.897 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.897 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:21.897 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.897 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.898 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.898 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.898 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.898 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.898 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.898 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.898 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-27T15:32:21.898 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-27T15:32:21.899 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.899 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.899 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.899 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:21.899 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.899 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.899 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.899 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.900 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.900 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.900 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.900 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.900 INFO:teuthology.orchestra.run.smithi008.stdout: "cross_origin_url": { 2023-11-27T15:32:21.900 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cross_origin_url", 2023-11-27T15:32:21.900 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.900 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.900 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.901 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.901 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.901 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.901 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.901 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.901 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.901 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.901 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.902 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.902 INFO:teuthology.orchestra.run.smithi008.stdout: "crt_file": { 2023-11-27T15:32:21.902 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crt_file", 2023-11-27T15:32:21.902 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.902 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.902 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.902 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.902 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.903 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.903 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.903 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.903 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.903 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.903 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.903 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.903 INFO:teuthology.orchestra.run.smithi008.stdout: "debug": { 2023-11-27T15:32:21.903 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "debug", 2023-11-27T15:32:21.904 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.904 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.904 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.904 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.904 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.904 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.904 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.904 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable/disable debug options", 2023-11-27T15:32:21.905 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.905 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.905 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.905 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.905 INFO:teuthology.orchestra.run.smithi008.stdout: "jwt_token_ttl": { 2023-11-27T15:32:21.905 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "jwt_token_ttl", 2023-11-27T15:32:21.905 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.905 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.906 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.906 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "28800", 2023-11-27T15:32:21.906 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.906 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.906 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.906 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.906 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.906 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.907 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.907 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.907 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2023-11-27T15:32:21.907 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2023-11-27T15:32:21.907 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.907 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.907 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.907 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.908 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.908 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.908 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.908 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.908 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.908 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.908 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.908 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.909 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.909 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.909 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.909 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.909 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.909 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.909 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.909 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.909 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.910 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.910 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.910 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.910 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.910 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.910 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.910 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.910 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.911 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.911 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.911 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.911 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.911 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.911 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.911 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.911 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.911 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.912 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.912 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.912 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.912 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.912 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.912 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.912 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.912 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.913 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.913 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.913 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.913 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.913 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.913 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.913 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.913 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.914 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.914 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.914 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.914 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.914 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.914 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.914 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.914 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.914 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.915 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.915 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.915 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.915 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.915 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.915 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.915 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.915 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.915 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.916 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.916 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.916 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.916 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.916 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.916 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.916 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.917 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.917 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.917 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.917 INFO:teuthology.orchestra.run.smithi008.stdout: "motd": { 2023-11-27T15:32:21.917 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "motd", 2023-11-27T15:32:21.917 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.917 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.917 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.917 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.918 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.918 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.918 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.918 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "The message of the day", 2023-11-27T15:32:21.918 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.918 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.918 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.918 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.919 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect_resolve_ip_addr": { 2023-11-27T15:32:21.919 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "redirect_resolve_ip_addr", 2023-11-27T15:32:21.919 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.919 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.919 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.919 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.919 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.919 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.920 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.920 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.920 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.920 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.920 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.920 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.920 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2023-11-27T15:32:21.920 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2023-11-27T15:32:21.921 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.921 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.921 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.921 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2023-11-27T15:32:21.921 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.921 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.921 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.921 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.921 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.922 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.922 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.922 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.922 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2023-11-27T15:32:21.922 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2023-11-27T15:32:21.922 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.922 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.922 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.923 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8080", 2023-11-27T15:32:21.923 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.923 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.923 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.923 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.923 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.923 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.923 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.923 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.924 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2023-11-27T15:32:21.924 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2023-11-27T15:32:21.924 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.924 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.924 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.924 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.924 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.924 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.925 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.925 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.925 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.925 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.925 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.925 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.925 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl_server_port": { 2023-11-27T15:32:21.925 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl_server_port", 2023-11-27T15:32:21.925 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.926 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.926 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.926 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8443", 2023-11-27T15:32:21.926 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.926 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.926 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.926 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.927 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.927 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.927 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.927 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.927 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2023-11-27T15:32:21.927 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2023-11-27T15:32:21.927 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.927 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.927 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.928 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "redirect", 2023-11-27T15:32:21.928 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.928 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.928 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.928 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.928 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect" 2023-11-27T15:32:21.928 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.928 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.929 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.929 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.929 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.929 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.929 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2023-11-27T15:32:21.929 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2023-11-27T15:32:21.929 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.929 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.930 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.930 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2023-11-27T15:32:21.930 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2023-11-27T15:32:21.930 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2023-11-27T15:32:21.930 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.930 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.930 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.930 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.930 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.931 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.931 INFO:teuthology.orchestra.run.smithi008.stdout: "url_prefix": { 2023-11-27T15:32:21.931 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url_prefix", 2023-11-27T15:32:21.931 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.931 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.931 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.931 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.931 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.932 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.932 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.932 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.932 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.932 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.932 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.932 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.932 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.933 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.933 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.933 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "devicehealth", 2023-11-27T15:32:21.933 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.933 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.933 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.933 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_monitoring": { 2023-11-27T15:32:21.933 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_monitoring", 2023-11-27T15:32:21.933 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.934 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.934 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.934 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.934 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.934 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.934 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.934 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "monitor device health metrics", 2023-11-27T15:32:21.934 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.935 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.935 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.935 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.935 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.935 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.935 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.935 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.935 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.936 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.936 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.936 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.936 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.936 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.936 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.936 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.936 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.937 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.938 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.938 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.938 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.938 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.938 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.938 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.938 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.939 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.939 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.939 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.939 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.939 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.939 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.939 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.939 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.940 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.940 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.940 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.940 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.940 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.940 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.940 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.940 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.940 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.941 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.941 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.941 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.941 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.941 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.941 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.941 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.941 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.942 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.942 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.942 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.942 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.942 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.942 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.942 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.942 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.943 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.943 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.943 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.943 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.943 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.943 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.943 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.943 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.944 INFO:teuthology.orchestra.run.smithi008.stdout: "mark_out_threshold": { 2023-11-27T15:32:21.944 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mark_out_threshold", 2023-11-27T15:32:21.944 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.944 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.944 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.944 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2419200", 2023-11-27T15:32:21.944 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.944 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.944 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.945 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-27T15:32:21.945 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.945 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.945 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.945 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.945 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_name": { 2023-11-27T15:32:21.945 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_name", 2023-11-27T15:32:21.945 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.946 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.946 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.946 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "device_health_metrics", 2023-11-27T15:32:21.946 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.946 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.946 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.946 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-27T15:32:21.946 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.947 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.947 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.947 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.947 INFO:teuthology.orchestra.run.smithi008.stdout: "retention_period": { 2023-11-27T15:32:21.947 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retention_period", 2023-11-27T15:32:21.947 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.947 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.947 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.947 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15552000", 2023-11-27T15:32:21.948 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.948 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.948 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.948 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain device health metrics", 2023-11-27T15:32:21.948 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.948 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.948 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.948 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.949 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_frequency": { 2023-11-27T15:32:21.949 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_frequency", 2023-11-27T15:32:21.949 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.949 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.949 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.949 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2023-11-27T15:32:21.949 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.949 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.949 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-27T15:32:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "self_heal": { 2023-11-27T15:32:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "self_heal", 2023-11-27T15:32:21.950 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.951 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.951 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.951 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:21.951 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.951 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.951 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.951 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-27T15:32:21.951 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.952 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.952 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.952 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.952 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:21.952 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:21.952 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.952 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.952 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.952 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:21.953 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.953 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.953 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.953 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and check device health", 2023-11-27T15:32:21.953 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.953 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.953 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.953 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.954 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_threshold": { 2023-11-27T15:32:21.954 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_threshold", 2023-11-27T15:32:21.954 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.954 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.954 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.954 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7257600", 2023-11-27T15:32:21.954 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.954 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.955 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.955 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-27T15:32:21.955 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.955 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.955 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.955 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.955 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.955 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.955 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.956 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "diskprediction_local", 2023-11-27T15:32:21.956 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:21.956 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:21.956 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.956 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.956 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.956 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.956 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.957 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.957 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.957 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.957 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.957 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.957 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.957 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.957 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.957 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.958 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.958 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.958 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.958 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.958 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.958 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.958 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.958 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.959 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.959 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.959 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.959 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.959 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.959 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.959 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.959 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.960 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.960 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.960 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.960 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.960 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.960 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.960 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.960 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.960 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.961 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.961 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.961 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.961 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.961 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.961 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.961 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.961 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.962 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.962 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.962 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.962 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.962 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.962 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.962 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.962 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.963 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.963 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.963 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.963 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.963 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.963 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.963 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.963 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.963 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.964 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.964 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.964 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.964 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.964 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.964 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.964 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.964 INFO:teuthology.orchestra.run.smithi008.stdout: "predict_interval": { 2023-11-27T15:32:21.965 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predict_interval", 2023-11-27T15:32:21.965 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.965 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.965 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.965 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2023-11-27T15:32:21.965 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.965 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.965 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.965 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.966 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.966 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.966 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.966 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.966 INFO:teuthology.orchestra.run.smithi008.stdout: "predictor_model": { 2023-11-27T15:32:21.966 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predictor_model", 2023-11-27T15:32:21.966 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.966 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.967 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.967 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "prophetstor", 2023-11-27T15:32:21.967 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.967 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.967 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.967 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.967 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.967 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.968 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.968 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.968 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:21.968 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:21.968 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.968 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.968 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.968 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:21.968 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.969 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.969 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.969 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.969 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.969 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.969 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.969 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.969 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:21.970 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.970 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:21.970 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "influx", 2023-11-27T15:32:21.970 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": false, 2023-11-27T15:32:21.970 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "influxdb python module not found", 2023-11-27T15:32:21.970 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:21.970 INFO:teuthology.orchestra.run.smithi008.stdout: "batch_size": { 2023-11-27T15:32:21.970 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "batch_size", 2023-11-27T15:32:21.970 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:21.971 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.971 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.971 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5000", 2023-11-27T15:32:21.971 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.971 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.971 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.971 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-27T15:32:21.971 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.972 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.972 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.972 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.972 INFO:teuthology.orchestra.run.smithi008.stdout: "database": { 2023-11-27T15:32:21.972 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "database", 2023-11-27T15:32:21.972 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.972 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.972 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.972 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "ceph", 2023-11-27T15:32:21.973 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.973 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.973 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.973 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-27T15:32:21.973 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.973 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.973 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.973 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.974 INFO:teuthology.orchestra.run.smithi008.stdout: "hostname": { 2023-11-27T15:32:21.974 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "hostname", 2023-11-27T15:32:21.974 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.974 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.974 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.974 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.974 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.974 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.975 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.975 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server hostname", 2023-11-27T15:32:21.975 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.975 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.975 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.975 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.975 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:21.975 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:21.975 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:21.976 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.976 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.976 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "30", 2023-11-27T15:32:21.976 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "5", 2023-11-27T15:32:21.976 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.976 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.976 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-27T15:32:21.976 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.977 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.977 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.977 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.977 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:21.977 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:21.977 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.977 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.977 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.977 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.978 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.978 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.978 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.978 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.978 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.978 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.978 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.978 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.979 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.979 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.979 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.979 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.979 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.979 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.979 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.979 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:21.980 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:21.980 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.980 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.980 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.980 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.980 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.980 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.980 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.980 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.981 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.981 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.981 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.981 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.981 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:21.981 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:21.981 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.981 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.982 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.982 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:21.982 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.982 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.982 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:21.982 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:21.982 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:21.982 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:21.982 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:21.983 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:21.983 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:21.983 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:21.983 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.983 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.983 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.983 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.983 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.984 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:21.984 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:21.984 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:21.984 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.984 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:21.984 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:21.984 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.984 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.985 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.985 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:21.985 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.985 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.985 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.985 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.985 INFO:teuthology.orchestra.run.smithi008.stdout: "password": { 2023-11-27T15:32:21.985 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "password", 2023-11-27T15:32:21.985 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:21.986 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:21.986 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:21.986 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:21.986 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:21.986 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:21.986 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:21.986 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "password of InfluxDB server user", 2023-11-27T15:32:21.986 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:21.987 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:21.987 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:21.987 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:21.987 INFO:teuthology.orchestra.run.smithi008.stdout: "port": { 2023-11-27T15:32:21.987 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "port", 2023-11-27T15:32:22.121 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.121 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.121 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.121 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8086", 2023-11-27T15:32:22.121 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.121 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.121 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.121 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server port", 2023-11-27T15:32:22.122 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.122 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.122 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.122 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.122 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2023-11-27T15:32:22.122 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2023-11-27T15:32:22.122 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.122 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.123 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.123 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "false", 2023-11-27T15:32:22.123 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.123 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.123 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.123 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:32:22.123 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.123 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.123 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.124 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.124 INFO:teuthology.orchestra.run.smithi008.stdout: "threads": { 2023-11-27T15:32:22.124 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threads", 2023-11-27T15:32:22.124 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.124 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.124 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.124 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:22.124 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:22.125 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "32", 2023-11-27T15:32:22.125 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.125 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-27T15:32:22.125 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.125 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.125 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.125 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.125 INFO:teuthology.orchestra.run.smithi008.stdout: "username": { 2023-11-27T15:32:22.125 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "username", 2023-11-27T15:32:22.126 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.126 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.126 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.126 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.126 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.126 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.126 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.126 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "username of InfluxDB server user", 2023-11-27T15:32:22.127 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.127 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.127 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.127 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.127 INFO:teuthology.orchestra.run.smithi008.stdout: "verify_ssl": { 2023-11-27T15:32:22.127 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "verify_ssl", 2023-11-27T15:32:22.127 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.127 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.128 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.128 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "true", 2023-11-27T15:32:22.128 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.128 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.128 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.128 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:32:22.128 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.128 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.128 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.129 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.129 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.129 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.129 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.129 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "insights", 2023-11-27T15:32:22.129 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.129 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.129 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.129 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.130 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.130 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.130 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.130 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.130 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.130 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.130 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.130 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.131 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.131 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.131 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.131 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.131 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.131 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.131 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.131 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.131 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.132 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.132 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.132 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.132 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.132 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.132 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.132 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.132 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.132 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.133 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.133 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.133 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.133 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.133 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.133 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.133 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.133 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.134 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.134 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.134 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.134 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.134 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.134 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.134 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.134 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.135 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.135 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.135 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.135 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.135 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.135 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.135 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.135 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.135 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.136 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.136 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.136 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.136 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.136 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.136 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.136 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.136 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.136 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.137 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.137 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.137 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.137 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.137 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.137 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.137 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.137 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.138 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.138 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.138 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.138 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.138 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "iostat", 2023-11-27T15:32:22.138 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.138 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.138 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.139 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.139 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.139 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.139 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.139 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.139 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.139 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.139 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.140 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.140 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.140 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.140 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.140 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.140 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.140 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.140 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.140 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.141 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.141 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.141 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.141 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.141 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.141 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.141 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.141 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.141 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.142 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.142 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.142 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.142 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.142 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.142 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.142 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.142 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.143 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.143 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.143 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.143 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.143 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.143 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.143 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.143 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.144 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.144 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.144 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.144 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.144 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.144 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.144 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.144 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.144 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.145 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.145 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.145 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.145 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.145 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.145 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.145 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.145 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.145 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.146 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.146 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.146 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.146 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.146 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.146 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.146 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.146 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.147 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.147 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.147 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.147 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.147 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.147 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "k8sevents", 2023-11-27T15:32:22.147 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.147 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.148 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.148 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_event_retention_days": { 2023-11-27T15:32:22.148 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ceph_event_retention_days", 2023-11-27T15:32:22.148 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.148 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.148 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.148 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7", 2023-11-27T15:32:22.148 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.149 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.149 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.149 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-27T15:32:22.149 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.149 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.149 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.149 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.149 INFO:teuthology.orchestra.run.smithi008.stdout: "config_check_secs": { 2023-11-27T15:32:22.149 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_check_secs", 2023-11-27T15:32:22.150 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.150 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.150 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.150 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:22.150 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "10", 2023-11-27T15:32:22.150 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.150 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.150 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-27T15:32:22.150 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.151 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.151 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.151 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.151 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.151 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.151 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.151 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.151 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.152 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.152 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.152 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.152 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.152 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.152 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.152 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.152 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.153 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.153 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.153 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.153 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.153 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.153 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.153 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.153 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.153 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.154 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.154 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.154 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.154 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.154 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.154 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.154 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.154 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.155 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.156 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.156 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.156 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.156 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.156 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.156 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.156 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.156 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.157 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.157 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.157 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.157 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.157 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.157 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.157 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.157 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.158 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.158 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.158 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.158 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.158 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.158 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.158 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.158 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.158 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.159 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.159 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.159 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.159 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.159 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.159 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.159 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.159 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.160 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.160 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "localpool", 2023-11-27T15:32:22.160 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.160 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.160 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.160 INFO:teuthology.orchestra.run.smithi008.stdout: "failure_domain": { 2023-11-27T15:32:22.160 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "failure_domain", 2023-11-27T15:32:22.160 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.160 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.161 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.161 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "host", 2023-11-27T15:32:22.161 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.161 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.161 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.161 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "failure domain for any created local pool", 2023-11-27T15:32:22.161 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-27T15:32:22.161 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.162 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.162 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.162 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.162 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.162 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.162 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.162 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.162 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.162 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.163 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.163 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.163 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.163 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.163 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.163 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.163 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.163 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.164 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.164 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.164 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.164 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.164 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.164 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.164 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.164 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.164 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.165 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.165 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.165 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.165 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.165 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.165 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.165 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.165 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.165 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.166 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.166 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.166 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.166 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.166 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.166 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.166 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.166 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.167 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.167 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.167 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.167 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.167 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.167 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.167 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.167 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.168 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.168 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.168 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.168 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.168 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.168 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.168 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.168 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.168 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.169 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.169 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.169 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.169 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.169 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.169 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.169 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.169 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.169 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.170 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.170 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.170 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.170 INFO:teuthology.orchestra.run.smithi008.stdout: "min_size": { 2023-11-27T15:32:22.170 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_size", 2023-11-27T15:32:22.170 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.170 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.170 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.170 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.171 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.171 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.171 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.171 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default min_size for any created local pool", 2023-11-27T15:32:22.171 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-27T15:32:22.171 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.171 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.171 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.172 INFO:teuthology.orchestra.run.smithi008.stdout: "num_rep": { 2023-11-27T15:32:22.172 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "num_rep", 2023-11-27T15:32:22.172 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.172 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.172 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.172 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3", 2023-11-27T15:32:22.172 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.172 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.172 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.173 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default replica count for any created local pool", 2023-11-27T15:32:22.173 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.173 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.173 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.173 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.173 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_num": { 2023-11-27T15:32:22.173 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_num", 2023-11-27T15:32:22.173 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.173 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.174 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.174 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "128", 2023-11-27T15:32:22.174 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.174 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.174 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.174 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default pg_num for any created local pool", 2023-11-27T15:32:22.174 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.174 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.175 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.175 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.175 INFO:teuthology.orchestra.run.smithi008.stdout: "prefix": { 2023-11-27T15:32:22.175 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prefix", 2023-11-27T15:32:22.175 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.175 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.175 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.175 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.176 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.176 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.176 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.176 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name prefix for any created local pool", 2023-11-27T15:32:22.176 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.176 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.176 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.176 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.176 INFO:teuthology.orchestra.run.smithi008.stdout: "subtree": { 2023-11-27T15:32:22.177 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "subtree", 2023-11-27T15:32:22.177 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.177 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.177 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.177 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "rack", 2023-11-27T15:32:22.177 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.177 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.177 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.178 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-27T15:32:22.178 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-27T15:32:22.178 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.178 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.178 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.178 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.178 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.178 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.179 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mds_autoscaler", 2023-11-27T15:32:22.179 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.179 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.179 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.179 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.179 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.179 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.179 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.179 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.180 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.180 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.180 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.180 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.180 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.180 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.180 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.180 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.180 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.181 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.181 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.181 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.181 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.181 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.181 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.181 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.181 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.181 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.182 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.182 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.182 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.182 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.182 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.182 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.182 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.182 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.183 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.183 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.183 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.183 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.183 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.183 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.183 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.183 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.183 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.184 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.184 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.184 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.184 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.184 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.184 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.184 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.184 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.185 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.185 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.185 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.185 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.185 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.185 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.185 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.185 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.185 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.186 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.186 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.186 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.186 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.186 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.186 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.186 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.186 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.187 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.187 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.187 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.187 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.187 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.187 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.188 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.188 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.188 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirroring", 2023-11-27T15:32:22.188 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.188 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.188 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.188 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.188 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.189 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.189 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.189 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.189 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.189 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.189 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.189 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.189 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.189 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.190 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.190 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.190 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.190 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.190 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.190 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.190 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.190 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.191 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.191 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.191 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.191 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.191 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.191 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.191 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.191 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.191 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.192 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.192 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.192 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.192 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.192 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.192 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.192 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.192 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.193 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.193 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.193 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.193 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.193 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.193 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.193 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.193 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.193 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.194 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.194 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.194 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.194 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.194 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.194 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.194 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.194 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.194 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.195 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.195 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.195 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.195 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.195 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.195 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.195 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.195 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.196 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.196 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.196 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.196 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.196 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.196 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.196 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.196 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.196 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.197 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.197 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.197 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "nfs", 2023-11-27T15:32:22.197 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.197 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.197 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.197 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.198 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.198 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.198 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.198 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.198 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.198 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.198 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.198 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.198 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.199 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.199 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.199 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.199 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.199 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.199 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.199 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.199 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.200 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.200 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.200 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.200 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.200 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.200 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.200 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.200 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.201 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.201 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.201 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.201 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.201 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.201 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.201 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.201 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.201 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.202 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.202 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.202 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.202 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.202 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.202 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.202 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.202 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.203 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.203 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.203 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.203 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.203 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.203 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.203 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.203 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.203 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.204 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.204 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.204 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.204 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.204 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.204 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.204 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.204 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.205 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.205 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.205 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.205 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.205 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.205 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.205 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.205 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.205 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.206 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.206 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.206 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.206 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.206 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2023-11-27T15:32:22.206 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.206 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.207 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.207 INFO:teuthology.orchestra.run.smithi008.stdout: "fail_fs": { 2023-11-27T15:32:22.207 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "fail_fs", 2023-11-27T15:32:22.207 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.207 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.207 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.207 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.207 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.208 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.208 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.208 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-27T15:32:22.208 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.208 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.208 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.208 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.208 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.209 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.209 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.209 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.209 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.209 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.209 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.209 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.209 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.209 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.210 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.210 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.210 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.210 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.210 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.210 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.210 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.210 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.211 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.211 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.211 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.211 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.211 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.211 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.211 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.211 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.212 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.212 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.212 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.212 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.212 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.212 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.212 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.212 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.212 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.213 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.213 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.213 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.213 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.213 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.213 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.213 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.213 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.213 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.214 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.214 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.214 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.214 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.214 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.214 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.214 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.214 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.215 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.215 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.215 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.215 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.215 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.215 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.215 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.215 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.216 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.216 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.216 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.216 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.216 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.216 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.216 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.216 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.216 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.217 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.217 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator": { 2023-11-27T15:32:22.217 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2023-11-27T15:32:22.217 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.217 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.217 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.217 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.218 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.218 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.218 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.218 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm", 2023-11-27T15:32:22.218 INFO:teuthology.orchestra.run.smithi008.stdout: "rook", 2023-11-27T15:32:22.218 INFO:teuthology.orchestra.run.smithi008.stdout: "test_orchestrator" 2023-11-27T15:32:22.218 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.218 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Orchestrator backend", 2023-11-27T15:32:22.219 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.219 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.219 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.219 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.219 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.219 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.219 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.219 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_perf_query", 2023-11-27T15:32:22.219 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.220 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.220 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.220 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.220 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.220 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.220 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.220 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.220 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.221 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.221 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.221 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.221 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.221 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.221 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.221 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.221 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.221 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.222 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.222 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.222 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.222 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.222 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.222 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.222 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.222 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.223 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.223 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.223 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.223 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.223 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.223 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.223 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.223 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.224 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.224 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.224 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.224 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.224 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.224 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.224 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.224 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.224 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.225 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.225 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.225 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.225 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.225 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.225 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.225 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.225 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.225 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.226 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.226 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.226 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.226 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.226 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.226 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.226 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.227 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.227 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.227 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.227 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.227 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.227 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.227 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.227 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.228 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.228 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.228 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.228 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.228 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.228 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.228 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.228 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.229 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.229 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_support", 2023-11-27T15:32:22.229 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.229 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.229 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.229 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.229 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.229 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.230 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.230 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.230 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.230 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.230 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.230 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.230 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.230 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.230 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.231 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.231 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.231 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.231 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.231 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.231 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.231 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.231 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.232 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.232 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.232 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.232 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.232 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.232 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.232 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.232 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.232 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.233 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.233 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.233 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.233 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.233 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.233 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.233 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.233 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.234 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.234 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.234 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.234 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.234 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.234 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.234 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.234 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.235 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.236 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.236 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.236 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.236 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.236 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.236 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.236 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.236 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.236 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.237 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.237 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.237 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.237 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.237 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.237 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.237 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.237 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.238 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.238 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.238 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.238 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_autoscaler", 2023-11-27T15:32:22.238 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.238 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.238 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.238 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.239 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.239 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.239 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.239 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.239 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.239 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.239 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.239 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.240 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.240 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.240 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.240 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.240 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.240 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.240 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.240 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.240 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.241 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.241 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.241 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.241 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.241 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.241 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.241 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.241 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.328 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.328 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.328 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.328 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.328 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.328 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.328 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.328 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.329 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.329 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.329 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.329 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.329 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.329 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.329 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.329 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.329 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.330 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.330 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.330 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.330 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.330 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.330 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.330 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.330 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.331 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.331 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.331 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.331 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.331 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.331 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.331 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.331 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.331 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.332 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.332 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.332 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.332 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.332 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.332 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.332 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.332 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.333 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.333 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.333 INFO:teuthology.orchestra.run.smithi008.stdout: "noautoscale": { 2023-11-27T15:32:22.333 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "noautoscale", 2023-11-27T15:32:22.333 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.333 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.333 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.333 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.333 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.334 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.334 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.334 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "global autoscale flag", 2023-11-27T15:32:22.334 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-27T15:32:22.334 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.334 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.334 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.334 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:22.335 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:22.335 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:22.335 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.335 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.335 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:22.335 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.335 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.335 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.335 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.336 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.336 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.336 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.336 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.336 INFO:teuthology.orchestra.run.smithi008.stdout: "threshold": { 2023-11-27T15:32:22.336 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threshold", 2023-11-27T15:32:22.336 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:22.336 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.337 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.337 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2023-11-27T15:32:22.337 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1.0", 2023-11-27T15:32:22.337 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.337 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.337 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "scaling threshold", 2023-11-27T15:32:22.337 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-27T15:32:22.337 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.338 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.338 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.338 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.338 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.338 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.338 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "progress", 2023-11-27T15:32:22.338 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.338 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.339 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.339 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_pg_recovery_event": { 2023-11-27T15:32:22.339 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_pg_recovery_event", 2023-11-27T15:32:22.339 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.339 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.339 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.339 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.339 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.339 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.340 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.340 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-27T15:32:22.340 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.340 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.340 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.340 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.340 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2023-11-27T15:32:22.340 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2023-11-27T15:32:22.340 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.341 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.341 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.341 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:22.341 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.341 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.341 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.341 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.341 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.342 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.342 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.342 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.342 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.342 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.342 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.342 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.342 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.343 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.343 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.343 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.343 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.343 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.343 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.343 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.343 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.344 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.344 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.344 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.344 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.344 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.344 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.344 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.344 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.344 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.345 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.345 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.345 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.345 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.345 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.345 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.345 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.345 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.346 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.346 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.346 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.346 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.346 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.346 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.346 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.346 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.347 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.347 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.347 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.347 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.347 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.347 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.347 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.347 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.347 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.348 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.348 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.348 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.348 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.348 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.348 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.348 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.348 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.349 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.349 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.349 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.349 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.349 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.349 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.349 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.349 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.349 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.350 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.350 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.350 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.350 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.350 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.350 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.350 INFO:teuthology.orchestra.run.smithi008.stdout: "max_completed_events": { 2023-11-27T15:32:22.350 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_completed_events", 2023-11-27T15:32:22.351 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.351 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.351 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.351 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "50", 2023-11-27T15:32:22.351 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.351 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.351 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.351 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "number of past completed events to remember", 2023-11-27T15:32:22.352 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.352 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.352 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.352 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.352 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:22.352 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:22.352 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:22.352 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.353 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.353 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:22.353 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.353 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.353 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.353 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long the module is going to sleep", 2023-11-27T15:32:22.353 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.353 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.353 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.354 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.354 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.354 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.354 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.354 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus", 2023-11-27T15:32:22.354 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.354 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.354 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.355 INFO:teuthology.orchestra.run.smithi008.stdout: "cache": { 2023-11-27T15:32:22.355 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cache", 2023-11-27T15:32:22.355 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.355 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.355 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.355 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:22.355 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.355 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.355 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.356 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.356 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.356 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.356 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.356 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.356 INFO:teuthology.orchestra.run.smithi008.stdout: "exclude_perf_counters": { 2023-11-27T15:32:22.356 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "exclude_perf_counters", 2023-11-27T15:32:22.356 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.357 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.357 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.357 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:22.357 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.357 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.357 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.357 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-27T15:32:22.357 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-27T15:32:22.358 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.358 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.358 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.358 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.358 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.358 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.358 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.358 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.359 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.359 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.359 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.359 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.359 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.359 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.359 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.359 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.359 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.360 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.360 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.360 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.360 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.360 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.360 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.360 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.360 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.360 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.361 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.361 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.361 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.361 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.361 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.361 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.361 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.361 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.362 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.362 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.362 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.362 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.362 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.362 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.362 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.362 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.363 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.363 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.363 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.363 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.363 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.363 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.363 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.363 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.363 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.364 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.364 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.364 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.364 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.364 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.364 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.364 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.364 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.365 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.365 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.365 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.365 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.365 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.365 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.365 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.365 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.365 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.366 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.366 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.366 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.366 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.366 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.366 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools": { 2023-11-27T15:32:22.366 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools", 2023-11-27T15:32:22.366 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.367 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.367 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.367 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.367 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.367 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.367 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.367 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.367 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.368 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.368 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.368 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.368 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-27T15:32:22.368 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-27T15:32:22.368 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.368 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.368 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.369 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300", 2023-11-27T15:32:22.369 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.369 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.369 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.369 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.369 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.369 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.369 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.369 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.370 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_interval": { 2023-11-27T15:32:22.370 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_interval", 2023-11-27T15:32:22.370 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:22.370 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.370 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.370 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15.0", 2023-11-27T15:32:22.370 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.370 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.371 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.371 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.371 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.371 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.371 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.371 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.371 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2023-11-27T15:32:22.371 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2023-11-27T15:32:22.372 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.372 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.372 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.372 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2023-11-27T15:32:22.372 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.372 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.372 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.372 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-27T15:32:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2023-11-27T15:32:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2023-11-27T15:32:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.374 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "9283", 2023-11-27T15:32:22.374 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.374 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.374 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.374 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-27T15:32:22.374 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.374 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.374 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.375 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.375 INFO:teuthology.orchestra.run.smithi008.stdout: "stale_cache_strategy": { 2023-11-27T15:32:22.375 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stale_cache_strategy", 2023-11-27T15:32:22.375 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.375 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.375 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.375 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "log", 2023-11-27T15:32:22.375 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.376 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.376 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.376 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.376 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.376 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.376 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.376 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.376 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2023-11-27T15:32:22.376 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2023-11-27T15:32:22.377 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.377 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.377 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.377 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "default", 2023-11-27T15:32:22.377 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.377 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.377 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.377 INFO:teuthology.orchestra.run.smithi008.stdout: "default", 2023-11-27T15:32:22.378 INFO:teuthology.orchestra.run.smithi008.stdout: "error" 2023-11-27T15:32:22.378 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.378 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.378 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.378 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.378 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.378 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.378 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2023-11-27T15:32:22.379 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2023-11-27T15:32:22.379 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.379 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.379 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.379 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2023-11-27T15:32:22.379 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2023-11-27T15:32:22.379 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2023-11-27T15:32:22.379 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.379 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.380 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.380 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.380 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.380 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.380 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:22.380 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.380 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:22.380 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_support", 2023-11-27T15:32:22.381 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:22.381 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:22.381 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:22.381 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:22.381 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:22.381 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.381 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.381 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.381 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.382 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.382 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.382 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.382 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.382 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.382 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.382 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.383 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.383 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.383 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.383 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.383 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.383 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.383 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.383 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.384 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:22.384 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:22.384 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.384 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.384 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.384 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.384 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.384 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.384 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.384 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.385 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.385 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.385 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.385 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.385 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:22.385 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:22.385 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.385 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.385 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.386 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:22.386 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.386 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.386 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:22.386 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:22.386 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:22.386 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:22.386 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:22.387 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:22.387 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:22.387 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:22.387 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.387 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.387 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.387 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.387 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.387 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:22.388 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:22.388 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:22.388 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.388 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:22.388 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:22.388 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.388 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.388 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.389 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.389 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.389 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.389 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.389 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.389 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_snap_create": { 2023-11-27T15:32:22.389 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_snap_create", 2023-11-27T15:32:22.389 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:22.390 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.390 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.390 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:22.390 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.390 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.390 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.390 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.390 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.390 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.391 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.391 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.391 INFO:teuthology.orchestra.run.smithi008.stdout: "mirror_snapshot_schedule": { 2023-11-27T15:32:22.391 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirror_snapshot_schedule", 2023-11-27T15:32:22.391 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.391 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.391 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.391 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.391 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.392 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.392 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.392 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:22.392 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:22.392 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:22.392 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:22.392 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:22.392 INFO:teuthology.orchestra.run.smithi008.stdout: "trash_purge_schedule": { 2023-11-27T15:32:22.392 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "trash_purge_schedule", 2023-11-27T15:32:22.393 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:22.393 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:22.393 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:22.393 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:22.393 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:22.393 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:22.393 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:22.393 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.535 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.535 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.535 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.535 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.535 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.535 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.535 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.536 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "restful", 2023-11-27T15:32:30.536 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.536 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.536 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.536 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_auth": { 2023-11-27T15:32:30.536 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_auth", 2023-11-27T15:32:30.536 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.536 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.537 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.537 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.537 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.537 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.537 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.537 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.538 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.538 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.538 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.538 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.538 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2023-11-27T15:32:30.538 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2023-11-27T15:32:30.539 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.539 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.539 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.540 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.540 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.540 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.540 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.540 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.541 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.541 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.541 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.541 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.541 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.541 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.542 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.542 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.542 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.542 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.543 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.543 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.543 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.543 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.543 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.544 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.544 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.544 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.544 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.544 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.544 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.545 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.545 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.545 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.545 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.546 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.546 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.546 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.547 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.547 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.547 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.547 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.548 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.548 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.548 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.548 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.548 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.549 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.549 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.549 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.549 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.549 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.549 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.549 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.550 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.550 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.550 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.550 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.550 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.550 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.550 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.551 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.551 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.551 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.551 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.551 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.552 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.552 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.552 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.552 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.553 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.553 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.553 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.553 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.554 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.554 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.554 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.554 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.555 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.555 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.555 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.555 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.555 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.556 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.556 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2023-11-27T15:32:30.556 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2023-11-27T15:32:30.556 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.557 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.557 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.557 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.557 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.557 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.557 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.557 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.557 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.558 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.558 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.558 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.558 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2023-11-27T15:32:30.558 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2023-11-27T15:32:30.558 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.558 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.558 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.559 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.559 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.559 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.559 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.559 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.559 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.559 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.559 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.559 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.560 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.560 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.560 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.560 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rgw", 2023-11-27T15:32:30.560 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.560 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.560 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.560 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.560 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.561 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.561 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.561 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.561 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.561 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.561 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.561 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.561 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.562 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.562 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.562 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.562 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.562 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.562 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.562 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.562 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.563 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.563 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.563 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.563 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.563 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.563 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.563 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.563 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.563 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.564 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.564 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.564 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.564 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.564 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.564 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.564 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.564 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.564 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.565 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.565 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.565 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.565 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.565 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.565 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.565 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.565 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.565 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.566 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.566 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.566 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.566 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.566 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.566 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.566 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.566 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.567 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.567 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.567 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.567 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.567 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.567 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.567 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.567 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.567 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.568 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.568 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.568 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.568 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.568 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.568 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.568 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.568 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.569 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.569 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.569 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.569 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rook", 2023-11-27T15:32:30.569 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.569 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.569 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.569 INFO:teuthology.orchestra.run.smithi008.stdout: "drive_group_interval": { 2023-11-27T15:32:30.570 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "drive_group_interval", 2023-11-27T15:32:30.570 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:30.570 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.570 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.570 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300.0", 2023-11-27T15:32:30.570 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.570 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.570 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.571 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-27T15:32:30.571 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.571 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.571 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.571 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.571 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.571 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.571 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.571 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.572 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.572 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.572 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.572 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.572 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.572 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.572 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.572 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.572 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.572 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.573 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.573 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.573 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.573 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.573 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.573 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.573 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.573 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.574 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.574 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.574 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.574 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.574 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.574 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.574 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.574 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.574 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.575 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.575 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.575 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.575 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.575 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.575 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.575 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.575 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.575 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.576 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.576 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.576 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.576 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.576 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.576 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.576 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.576 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.577 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.577 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.577 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.577 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.577 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.577 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.577 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.577 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.577 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.578 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.578 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.578 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.578 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.578 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.578 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.578 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.578 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.578 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.579 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.579 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.579 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.579 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.579 INFO:teuthology.orchestra.run.smithi008.stdout: "storage_class": { 2023-11-27T15:32:30.579 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "storage_class", 2023-11-27T15:32:30.579 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.579 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.579 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.580 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "local", 2023-11-27T15:32:30.580 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.580 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.580 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.580 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-27T15:32:30.580 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.580 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.580 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.580 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.581 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.581 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.581 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.581 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "selftest", 2023-11-27T15:32:30.581 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.581 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.581 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.581 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.582 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.582 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.582 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.582 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.582 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.582 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.582 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.582 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.582 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.583 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.583 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.583 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.583 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.583 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.583 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.583 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.583 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.584 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.584 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.584 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.584 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.584 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.584 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.584 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.584 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.584 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.585 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.585 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.585 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.585 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.585 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.585 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.585 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.585 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.585 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.586 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.586 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.586 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.586 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.586 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.586 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.586 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.586 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.587 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.587 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.587 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.587 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.587 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.587 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.587 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.587 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.587 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.588 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.588 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.588 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.588 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.588 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.588 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.588 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.588 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.588 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.589 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.589 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.589 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.589 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.589 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.589 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.589 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.589 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.589 INFO:teuthology.orchestra.run.smithi008.stdout: "roption1": { 2023-11-27T15:32:30.590 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption1", 2023-11-27T15:32:30.590 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.590 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.590 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.590 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.590 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.590 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.590 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.591 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.591 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.591 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.591 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.591 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.591 INFO:teuthology.orchestra.run.smithi008.stdout: "roption2": { 2023-11-27T15:32:30.591 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption2", 2023-11-27T15:32:30.591 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.591 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.592 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.592 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "xyz", 2023-11-27T15:32:30.592 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.592 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.592 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.592 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.592 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.592 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.592 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.593 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.593 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption1": { 2023-11-27T15:32:30.593 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption1", 2023-11-27T15:32:30.593 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.593 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.593 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.593 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.593 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.593 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.594 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.594 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.594 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.594 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.594 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.594 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.594 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption2": { 2023-11-27T15:32:30.594 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption2", 2023-11-27T15:32:30.595 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.595 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.595 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.595 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.595 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.595 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.595 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.595 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.595 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.596 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.596 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.596 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.596 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption3": { 2023-11-27T15:32:30.596 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption3", 2023-11-27T15:32:30.596 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:30.596 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.596 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.596 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.597 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.597 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.597 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.597 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.597 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.597 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.597 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.597 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.597 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption4": { 2023-11-27T15:32:30.598 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption4", 2023-11-27T15:32:30.598 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.598 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.598 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.598 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.598 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.598 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.598 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.599 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.599 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.599 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.599 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.599 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.599 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption5": { 2023-11-27T15:32:30.599 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption5", 2023-11-27T15:32:30.599 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.599 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.600 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.600 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.600 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.600 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.600 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.600 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.600 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.600 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.600 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.601 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.601 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption6": { 2023-11-27T15:32:30.601 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption6", 2023-11-27T15:32:30.601 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.601 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.601 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.601 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.601 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.602 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.602 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.602 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.602 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.602 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.602 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.602 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.602 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption7": { 2023-11-27T15:32:30.602 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption7", 2023-11-27T15:32:30.603 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.603 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.603 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.603 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.603 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:30.603 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "42", 2023-11-27T15:32:30.603 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.603 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.604 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.604 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.604 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.604 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.604 INFO:teuthology.orchestra.run.smithi008.stdout: "testkey": { 2023-11-27T15:32:30.604 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testkey", 2023-11-27T15:32:30.604 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.604 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.604 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.605 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.605 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.605 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.605 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.605 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.605 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.605 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.605 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.605 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.606 INFO:teuthology.orchestra.run.smithi008.stdout: "testlkey": { 2023-11-27T15:32:30.606 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testlkey", 2023-11-27T15:32:30.606 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.606 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.606 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.606 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.606 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.606 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.607 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.607 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.607 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.607 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.607 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.607 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.607 INFO:teuthology.orchestra.run.smithi008.stdout: "testnewline": { 2023-11-27T15:32:30.607 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testnewline", 2023-11-27T15:32:30.607 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.608 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.608 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.608 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.608 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.608 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.608 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.608 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.608 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.609 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.609 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.609 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.609 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.609 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.609 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.609 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snap_schedule", 2023-11-27T15:32:30.609 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.609 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.610 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.610 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_m_granularity": { 2023-11-27T15:32:30.610 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_m_granularity", 2023-11-27T15:32:30.610 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.610 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.610 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.610 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.610 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.610 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.611 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.611 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow minute scheduled snapshots", 2023-11-27T15:32:30.611 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.611 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.611 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.611 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.611 INFO:teuthology.orchestra.run.smithi008.stdout: "dump_on_update": { 2023-11-27T15:32:30.611 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dump_on_update", 2023-11-27T15:32:30.611 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.612 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.612 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.612 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.612 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.612 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.612 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.612 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "dump database to debug log on update", 2023-11-27T15:32:30.612 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.612 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.613 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.613 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.613 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.613 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.613 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.613 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.613 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.613 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.614 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.614 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.614 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.614 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.614 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.614 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.614 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.614 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.614 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.615 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.615 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.615 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.615 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.615 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.615 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.615 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.615 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.615 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.616 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.616 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.616 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.616 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.616 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.616 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.616 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.616 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.616 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.617 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.617 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.617 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.617 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.617 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.617 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.617 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.617 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.618 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.618 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.618 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.618 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.618 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.618 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.618 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.618 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.619 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.619 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.619 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.619 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.619 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.619 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.619 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.619 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.620 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.620 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.620 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.620 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.620 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.620 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.620 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.620 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.621 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.621 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.621 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.621 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.621 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.621 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.621 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.621 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.622 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stats", 2023-11-27T15:32:30.622 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.622 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.622 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.622 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.622 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.622 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.622 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.623 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.623 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.623 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.623 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.623 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.623 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.623 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.623 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.624 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.624 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.624 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.624 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.624 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.624 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.624 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.624 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.625 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.625 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.625 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.625 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.625 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.625 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.625 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.625 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.626 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.626 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.626 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.626 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.626 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.626 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.626 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.626 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.627 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.627 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.627 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.627 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.627 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.627 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.627 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.627 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.628 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.628 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.628 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.628 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.628 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.628 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.628 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.628 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.629 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.629 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.629 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.629 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.629 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.629 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.629 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.629 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.630 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.630 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.630 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.630 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.630 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.630 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.630 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.630 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.631 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.631 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.631 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.631 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.631 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.631 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "status", 2023-11-27T15:32:30.631 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.631 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.632 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.632 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.632 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.632 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.632 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.632 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.632 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.632 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.633 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.633 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.633 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.633 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.633 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.633 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.633 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.633 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.634 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.634 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.634 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.634 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.634 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.634 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.634 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.634 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.635 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.635 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.635 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.635 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.635 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.635 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.635 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.635 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.635 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.636 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.636 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.636 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.636 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.636 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.636 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.636 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.637 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.637 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.637 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.637 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.637 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.637 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.637 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.637 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.637 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.638 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.638 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.638 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.638 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.638 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.638 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.638 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.639 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.639 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.639 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.639 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.639 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.639 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.639 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.639 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.640 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.640 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.640 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.640 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.640 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.640 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.640 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.640 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.641 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.641 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.641 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telegraf", 2023-11-27T15:32:30.641 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.641 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.641 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.641 INFO:teuthology.orchestra.run.smithi008.stdout: "address": { 2023-11-27T15:32:30.641 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "address", 2023-11-27T15:32:30.642 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.642 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.642 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.642 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-27T15:32:30.642 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.642 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.642 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.642 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.643 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.643 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.643 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.643 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.643 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:30.643 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:30.643 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.643 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.644 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.644 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15", 2023-11-27T15:32:30.644 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.644 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.644 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.644 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.644 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.644 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.645 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.645 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.645 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.645 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.645 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.645 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.645 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.645 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.645 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.646 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.646 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.646 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.646 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.646 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.646 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.646 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.647 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.647 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.647 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.647 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.647 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.647 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.647 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.647 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.647 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.648 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.648 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.648 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.648 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.648 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.648 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.648 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.648 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.649 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.649 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.649 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.649 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.649 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.649 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.649 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.649 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.650 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.650 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.650 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.650 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.650 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.650 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.650 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.650 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.651 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.651 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.651 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.651 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.651 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.651 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.651 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.651 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.652 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.652 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.652 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.652 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.652 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.652 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.652 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.652 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.653 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.653 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.653 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.653 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.653 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.653 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.653 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.653 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.653 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.654 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.654 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telemetry", 2023-11-27T15:32:30.654 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.654 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.654 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.654 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_basic": { 2023-11-27T15:32:30.654 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_basic", 2023-11-27T15:32:30.655 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.655 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.655 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.655 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.655 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.655 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.655 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.655 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-27T15:32:30.656 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.656 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.656 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.656 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.656 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_crash": { 2023-11-27T15:32:30.656 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_crash", 2023-11-27T15:32:30.656 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.656 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.657 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.657 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.657 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.657 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.657 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.657 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-27T15:32:30.657 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.657 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.658 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.658 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.658 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_device": { 2023-11-27T15:32:30.658 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_device", 2023-11-27T15:32:30.658 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.658 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.658 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.658 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.659 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.659 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.659 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.659 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-27T15:32:30.659 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.659 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.659 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.659 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.660 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_ident": { 2023-11-27T15:32:30.660 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_ident", 2023-11-27T15:32:30.660 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.660 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.660 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.660 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.660 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.660 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.661 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.661 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-27T15:32:30.661 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.661 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.661 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.661 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.661 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_perf": { 2023-11-27T15:32:30.661 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_perf", 2023-11-27T15:32:30.661 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.662 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.662 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.662 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.662 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.662 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.662 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.662 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-27T15:32:30.663 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.663 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.663 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.663 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.663 INFO:teuthology.orchestra.run.smithi008.stdout: "contact": { 2023-11-27T15:32:30.663 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "contact", 2023-11-27T15:32:30.663 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.663 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.663 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.664 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.664 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.664 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.664 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.664 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.664 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.664 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.664 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.665 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.665 INFO:teuthology.orchestra.run.smithi008.stdout: "description": { 2023-11-27T15:32:30.665 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "description", 2023-11-27T15:32:30.665 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.665 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.665 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.665 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.665 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.666 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.666 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.666 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.666 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.666 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.666 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.666 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.666 INFO:teuthology.orchestra.run.smithi008.stdout: "device_url": { 2023-11-27T15:32:30.667 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_url", 2023-11-27T15:32:30.667 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.667 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.667 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.667 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-27T15:32:30.667 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.667 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.667 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.668 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.668 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.668 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.668 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.668 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.668 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2023-11-27T15:32:30.668 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2023-11-27T15:32:30.668 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.669 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.669 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.669 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.669 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.669 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.669 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.669 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.669 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.670 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.670 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.670 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.670 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:30.670 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:30.670 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.670 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.670 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.671 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "24", 2023-11-27T15:32:30.671 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "8", 2023-11-27T15:32:30.671 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.671 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.671 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.671 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.671 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.671 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.672 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.672 INFO:teuthology.orchestra.run.smithi008.stdout: "last_opt_revision": { 2023-11-27T15:32:30.672 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "last_opt_revision", 2023-11-27T15:32:30.672 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.672 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.672 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.672 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1", 2023-11-27T15:32:30.672 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.672 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.673 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.673 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.673 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.673 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.673 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.673 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.673 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard": { 2023-11-27T15:32:30.673 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard", 2023-11-27T15:32:30.674 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.674 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.674 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.674 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.674 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.674 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.674 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.674 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.675 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.675 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.675 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.675 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.675 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard_description": { 2023-11-27T15:32:30.675 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard_description", 2023-11-27T15:32:30.675 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.675 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.676 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.676 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.676 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.676 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.676 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.676 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.676 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.676 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.677 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.677 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.677 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.677 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.677 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.677 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.677 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.677 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.677 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.678 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.678 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.678 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.678 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.678 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.678 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.678 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.679 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.679 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.679 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.679 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.679 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.679 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.679 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.679 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.679 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.680 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.680 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.680 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.680 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.680 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.680 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.680 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.680 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.681 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.681 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.681 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.681 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.681 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.681 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.681 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.681 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.682 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.682 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.682 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.682 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.682 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.682 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.682 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.682 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.683 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.683 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.683 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.683 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.683 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.683 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.683 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.683 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.684 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.684 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.684 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.684 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.684 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.684 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.684 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.684 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.685 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.685 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.685 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.685 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.685 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.685 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.685 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.685 INFO:teuthology.orchestra.run.smithi008.stdout: "organization": { 2023-11-27T15:32:30.686 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "organization", 2023-11-27T15:32:30.686 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.686 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.686 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.686 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.686 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.686 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.686 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.687 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.687 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.687 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.687 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.687 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.687 INFO:teuthology.orchestra.run.smithi008.stdout: "proxy": { 2023-11-27T15:32:30.687 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "proxy", 2023-11-27T15:32:30.687 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.687 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.688 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.688 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.688 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.688 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.688 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.688 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.688 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.688 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.689 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.689 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.689 INFO:teuthology.orchestra.run.smithi008.stdout: "url": { 2023-11-27T15:32:30.689 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url", 2023-11-27T15:32:30.689 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.689 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.689 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.689 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-27T15:32:30.690 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.690 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.690 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.690 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.690 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.690 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.690 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.690 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.691 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.691 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.691 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.691 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "test_orchestrator", 2023-11-27T15:32:30.691 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.691 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.691 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.691 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.692 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.692 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.692 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.692 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.692 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.692 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.692 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.692 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.692 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.693 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.693 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.693 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.693 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.693 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.693 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.693 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.693 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.694 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.694 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.694 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.694 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.694 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.694 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.694 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.694 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.695 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.695 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.695 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.695 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.695 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.695 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.695 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.695 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.696 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.696 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.696 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.696 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.696 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.696 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.696 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.696 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.697 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.697 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.697 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.697 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.697 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.697 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.697 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.697 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.698 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.698 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.698 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.698 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.698 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.698 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.698 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.698 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.699 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.699 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.699 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.699 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.699 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.699 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.699 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.699 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.699 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.700 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.700 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.700 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.700 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.700 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.700 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.700 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "volumes", 2023-11-27T15:32:30.700 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.701 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.701 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.701 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.701 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.701 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.701 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.701 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.701 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.702 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.702 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.702 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.702 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.702 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.702 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.702 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.702 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.703 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.703 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.703 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.703 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.703 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.703 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.703 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.703 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.704 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.704 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.704 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.704 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.704 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.704 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.704 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.704 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.705 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.705 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.705 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.705 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.705 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.705 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.705 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.705 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.706 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.706 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.706 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.706 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.706 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.706 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.706 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.706 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.707 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.707 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.707 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.707 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.707 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.707 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.707 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.707 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.707 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.708 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.708 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.708 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.708 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.708 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.708 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.708 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.708 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.709 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.709 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.709 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.709 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.709 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.709 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.709 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.709 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_clones": { 2023-11-27T15:32:30.710 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_clones", 2023-11-27T15:32:30.710 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.710 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.710 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.710 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4", 2023-11-27T15:32:30.710 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.710 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.710 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.711 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-27T15:32:30.711 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.711 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.711 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.711 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.711 INFO:teuthology.orchestra.run.smithi008.stdout: "snapshot_clone_delay": { 2023-11-27T15:32:30.711 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snapshot_clone_delay", 2023-11-27T15:32:30.711 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.712 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.712 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.712 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:30.712 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.712 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.712 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.712 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-27T15:32:30.712 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.713 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.713 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.713 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.713 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.713 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.713 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.713 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix", 2023-11-27T15:32:30.713 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.713 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.714 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.714 INFO:teuthology.orchestra.run.smithi008.stdout: "discovery_interval": { 2023-11-27T15:32:30.714 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "discovery_interval", 2023-11-27T15:32:30.714 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:30.714 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.714 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.714 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "100", 2023-11-27T15:32:30.715 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.715 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.715 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.715 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.715 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.715 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.715 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.715 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.715 INFO:teuthology.orchestra.run.smithi008.stdout: "identifier": { 2023-11-27T15:32:30.716 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "identifier", 2023-11-27T15:32:30.716 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.716 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.716 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.716 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.716 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.716 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.716 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.717 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.717 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.717 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.717 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.717 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.717 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:30.717 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:30.717 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.718 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.718 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.718 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:30.718 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.718 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.718 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.718 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.718 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.719 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.719 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.719 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.719 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.719 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.719 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.719 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.719 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.720 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.720 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.720 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.720 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.720 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.720 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.720 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.720 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.721 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.721 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.721 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.721 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.721 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.721 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.721 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.721 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.722 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.722 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.722 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.722 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.722 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.722 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.722 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.722 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.723 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.723 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.723 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.723 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.723 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.723 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.723 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.723 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.723 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.724 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.724 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.724 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.724 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.724 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.724 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.724 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.725 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.725 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.725 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.725 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.725 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.725 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.725 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.725 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.725 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.726 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.726 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.726 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.726 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.726 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.726 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.726 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.727 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.727 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.727 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.727 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.727 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.727 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.727 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.727 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.727 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.728 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_host": { 2023-11-27T15:32:30.728 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_host", 2023-11-27T15:32:30.728 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.728 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.728 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.728 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.728 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.728 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.729 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.729 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.729 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.729 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.729 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.729 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.729 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_port": { 2023-11-27T15:32:30.729 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_port", 2023-11-27T15:32:30.730 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.730 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.730 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.730 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10051", 2023-11-27T15:32:30.730 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.730 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.730 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.730 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.731 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.731 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.731 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.731 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.731 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_sender": { 2023-11-27T15:32:30.731 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_sender", 2023-11-27T15:32:30.731 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.731 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.732 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.732 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-27T15:32:30.732 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.732 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.732 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.732 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.732 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.732 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.733 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.733 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.733 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.733 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.733 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2023-11-27T15:32:30.733 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.733 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.733 INFO:teuthology.orchestra.run.smithi008.stdout: "modules": [ 2023-11-27T15:32:30.734 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm", 2023-11-27T15:32:30.734 INFO:teuthology.orchestra.run.smithi008.stdout: "dashboard", 2023-11-27T15:32:30.734 INFO:teuthology.orchestra.run.smithi008.stdout: "iostat", 2023-11-27T15:32:30.734 INFO:teuthology.orchestra.run.smithi008.stdout: "nfs", 2023-11-27T15:32:30.734 INFO:teuthology.orchestra.run.smithi008.stdout: "restful" 2023-11-27T15:32:30.734 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.734 INFO:teuthology.orchestra.run.smithi008.stdout: "available_modules": [ 2023-11-27T15:32:30.734 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.735 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alerts", 2023-11-27T15:32:30.735 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.735 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.735 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.735 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:30.735 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:30.735 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.735 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.736 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.736 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:30.736 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.736 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.736 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.736 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How frequently to reexamine health status", 2023-11-27T15:32:30.736 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.736 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.737 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.737 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.737 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.737 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.737 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.737 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.737 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.737 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.738 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.738 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.738 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.738 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.738 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.738 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.738 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.738 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.739 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.739 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.739 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.739 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.739 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.739 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.739 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.739 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.740 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.740 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.740 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.740 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.740 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.740 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.740 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.740 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.741 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.741 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.741 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.741 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.741 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.741 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.741 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.741 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.742 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.742 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.742 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.742 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.742 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.742 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.742 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.743 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.743 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.743 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.743 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.743 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.743 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.743 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.743 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.744 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.744 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.744 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.744 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.744 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.744 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.744 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.744 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.745 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.745 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.745 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.745 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.745 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.745 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.745 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.745 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.746 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.746 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_destination": { 2023-11-27T15:32:30.746 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_destination", 2023-11-27T15:32:30.746 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.746 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.746 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.746 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.746 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.747 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.747 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.747 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email address to send alerts to", 2023-11-27T15:32:30.747 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.747 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.747 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.747 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.747 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_from_name": { 2023-11-27T15:32:30.748 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_from_name", 2023-11-27T15:32:30.748 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.748 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.748 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.748 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "Ceph", 2023-11-27T15:32:30.748 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.748 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.748 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.749 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Email From: name", 2023-11-27T15:32:30.749 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.749 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.749 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.749 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.749 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_host": { 2023-11-27T15:32:30.749 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_host", 2023-11-27T15:32:30.749 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.750 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.750 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.750 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.750 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.750 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.750 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.750 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP server", 2023-11-27T15:32:30.750 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.750 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.751 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.751 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.751 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_password": { 2023-11-27T15:32:30.751 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_password", 2023-11-27T15:32:30.751 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.751 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.751 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.751 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.752 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.752 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.752 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.752 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Password to authenticate with", 2023-11-27T15:32:30.752 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.752 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.752 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.752 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.753 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_port": { 2023-11-27T15:32:30.753 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_port", 2023-11-27T15:32:30.753 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.753 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.753 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.753 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "465", 2023-11-27T15:32:30.753 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.753 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.754 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.754 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP port", 2023-11-27T15:32:30.754 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.754 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.754 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.754 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.754 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_sender": { 2023-11-27T15:32:30.754 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_sender", 2023-11-27T15:32:30.755 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.755 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.755 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.755 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.755 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.755 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.755 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.755 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SMTP envelope sender", 2023-11-27T15:32:30.756 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.756 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.756 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.756 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.756 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_ssl": { 2023-11-27T15:32:30.756 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_ssl", 2023-11-27T15:32:30.756 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.756 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.757 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.757 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.757 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.757 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.757 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.757 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-27T15:32:30.757 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.757 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.758 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.758 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.758 INFO:teuthology.orchestra.run.smithi008.stdout: "smtp_user": { 2023-11-27T15:32:30.758 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "smtp_user", 2023-11-27T15:32:30.758 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.758 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.758 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.758 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.759 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.759 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.759 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.759 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "User to authenticate as", 2023-11-27T15:32:30.759 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.759 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.759 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.759 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.760 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.760 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.760 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.760 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "balancer", 2023-11-27T15:32:30.760 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.760 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.760 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.760 INFO:teuthology.orchestra.run.smithi008.stdout: "active": { 2023-11-27T15:32:30.761 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "active", 2023-11-27T15:32:30.761 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.761 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.761 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.761 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.761 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.761 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.761 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.762 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically balance PGs across cluster", 2023-11-27T15:32:30.762 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.762 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.762 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.762 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.762 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_time": { 2023-11-27T15:32:30.762 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_time", 2023-11-27T15:32:30.762 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.763 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.763 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.763 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0000", 2023-11-27T15:32:30.763 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.763 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.763 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.763 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "beginning time of day to automatically balance", 2023-11-27T15:32:30.763 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:32:30.764 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.764 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.764 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.764 INFO:teuthology.orchestra.run.smithi008.stdout: "begin_weekday": { 2023-11-27T15:32:30.764 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "begin_weekday", 2023-11-27T15:32:30.764 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:30.764 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.764 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.765 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:30.765 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2023-11-27T15:32:30.765 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2023-11-27T15:32:30.765 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.765 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-27T15:32:30.765 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:32:30.765 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.765 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.766 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.766 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_max_iterations": { 2023-11-27T15:32:30.766 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_max_iterations", 2023-11-27T15:32:30.766 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:30.766 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.766 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.766 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "25", 2023-11-27T15:32:30.766 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:30.767 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "250", 2023-11-27T15:32:30.767 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.767 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-27T15:32:30.767 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.767 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.767 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.767 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.767 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_metrics": { 2023-11-27T15:32:30.768 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_metrics", 2023-11-27T15:32:30.768 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.768 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.768 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.768 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "pgs,objects,bytes", 2023-11-27T15:32:30.768 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.768 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.768 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.769 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-27T15:32:30.769 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-27T15:32:30.769 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.769 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.769 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.769 INFO:teuthology.orchestra.run.smithi008.stdout: "crush_compat_step": { 2023-11-27T15:32:30.769 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crush_compat_step", 2023-11-27T15:32:30.769 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:30.770 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.770 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.770 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.5", 2023-11-27T15:32:30.770 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0.001", 2023-11-27T15:32:30.770 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "0.999", 2023-11-27T15:32:30.770 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.770 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "aggressiveness of optimization", 2023-11-27T15:32:30.770 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-27T15:32:30.771 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.771 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.771 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.771 INFO:teuthology.orchestra.run.smithi008.stdout: "end_time": { 2023-11-27T15:32:30.771 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_time", 2023-11-27T15:32:30.771 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.771 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.771 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.772 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2359", 2023-11-27T15:32:30.772 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.772 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.772 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.772 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ending time of day to automatically balance", 2023-11-27T15:32:30.772 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:32:30.772 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.772 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.772 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.773 INFO:teuthology.orchestra.run.smithi008.stdout: "end_weekday": { 2023-11-27T15:32:30.773 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "end_weekday", 2023-11-27T15:32:30.773 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:30.773 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.773 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.773 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:30.773 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "0", 2023-11-27T15:32:30.773 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "6", 2023-11-27T15:32:30.774 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.774 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-27T15:32:30.774 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:32:30.774 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.774 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.774 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.774 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.774 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.775 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.775 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.775 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.775 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.775 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.775 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.775 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.775 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.776 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.776 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.776 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.776 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.776 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.776 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.776 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.776 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.777 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.777 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.777 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.777 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.777 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.777 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.777 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.777 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.778 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.778 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.778 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.778 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.778 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.778 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.778 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.778 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.779 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.779 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.779 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.779 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.779 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.779 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.779 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.781 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.781 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.781 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.781 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.781 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.781 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.781 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.781 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.782 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: "min_score": { 2023-11-27T15:32:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_score", 2023-11-27T15:32:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:30.783 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.784 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.784 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:30.784 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.784 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.784 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.784 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-27T15:32:30.784 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.784 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.785 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.785 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.785 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2023-11-27T15:32:30.785 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2023-11-27T15:32:30.785 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.785 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.785 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.785 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "upmap", 2023-11-27T15:32:30.786 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.786 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.786 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.786 INFO:teuthology.orchestra.run.smithi008.stdout: "crush-compat", 2023-11-27T15:32:30.786 INFO:teuthology.orchestra.run.smithi008.stdout: "none", 2023-11-27T15:32:30.786 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap" 2023-11-27T15:32:30.786 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.786 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Balancer mode", 2023-11-27T15:32:30.787 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.787 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.787 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.787 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.787 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_ids": { 2023-11-27T15:32:30.787 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_ids", 2023-11-27T15:32:30.787 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.787 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.788 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.788 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.788 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.788 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.788 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.788 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-27T15:32:30.788 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.788 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.789 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.789 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.789 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:30.789 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:30.789 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.789 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.789 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.789 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:30.790 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.790 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.790 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.790 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-27T15:32:30.790 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.790 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.790 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.790 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.791 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_deviation": { 2023-11-27T15:32:30.791 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_deviation", 2023-11-27T15:32:30.791 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.791 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.791 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.791 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:30.791 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:30.791 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.792 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.792 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-27T15:32:30.792 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-27T15:32:30.792 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.792 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.792 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.792 INFO:teuthology.orchestra.run.smithi008.stdout: "upmap_max_optimizations": { 2023-11-27T15:32:30.792 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "upmap_max_optimizations", 2023-11-27T15:32:30.792 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:30.793 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.793 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.793 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:30.793 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.793 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.793 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.793 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-27T15:32:30.794 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.794 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.794 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.794 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.794 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.794 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.794 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.794 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cephadm", 2023-11-27T15:32:30.795 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.795 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.795 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.795 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_down_multiplier": { 2023-11-27T15:32:30.795 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_down_multiplier", 2023-11-27T15:32:30.795 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:30.795 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.795 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.796 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2023-11-27T15:32:30.796 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.796 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.796 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.796 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-27T15:32:30.796 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.796 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.796 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.796 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.797 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_refresh_rate": { 2023-11-27T15:32:30.797 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_refresh_rate", 2023-11-27T15:32:30.797 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.797 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.797 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.797 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "20", 2023-11-27T15:32:30.797 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.797 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.798 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.798 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-27T15:32:30.798 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.798 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.798 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.798 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.798 INFO:teuthology.orchestra.run.smithi008.stdout: "agent_starting_port": { 2023-11-27T15:32:30.799 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "agent_starting_port", 2023-11-27T15:32:30.799 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.799 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.799 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.799 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4721", 2023-11-27T15:32:30.799 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.799 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.799 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.799 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-27T15:32:30.800 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.800 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.800 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.800 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.800 INFO:teuthology.orchestra.run.smithi008.stdout: "alertmanager_web_password": { 2023-11-27T15:32:30.800 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alertmanager_web_password", 2023-11-27T15:32:30.800 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.800 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.801 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.801 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:30.801 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.801 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.801 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.801 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Alertmanager web password", 2023-11-27T15:32:30.801 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.801 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.802 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.802 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.802 INFO:teuthology.orchestra.run.smithi008.stdout: "alertmanager_web_user": { 2023-11-27T15:32:30.802 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "alertmanager_web_user", 2023-11-27T15:32:30.802 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.802 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.802 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.802 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:30.803 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.803 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.803 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.803 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Alertmanager web user", 2023-11-27T15:32:30.803 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.803 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.803 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.803 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.804 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_ptrace": { 2023-11-27T15:32:30.804 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_ptrace", 2023-11-27T15:32:30.804 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.804 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.804 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.804 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.804 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.804 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.805 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.805 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-27T15:32:30.805 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-27T15:32:30.805 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.805 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.805 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.805 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_interval": { 2023-11-27T15:32:30.805 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_interval", 2023-11-27T15:32:30.806 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.806 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.806 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.806 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:30.806 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.806 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.806 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.806 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-27T15:32:30.807 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.807 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.807 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.807 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.807 INFO:teuthology.orchestra.run.smithi008.stdout: "autotune_memory_target_ratio": { 2023-11-27T15:32:30.807 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "autotune_memory_target_ratio", 2023-11-27T15:32:30.807 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:30.807 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.808 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.808 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0.7", 2023-11-27T15:32:30.808 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.808 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.808 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.808 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-27T15:32:30.808 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.809 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.809 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.809 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.809 INFO:teuthology.orchestra.run.smithi008.stdout: "cgroups_split": { 2023-11-27T15:32:30.809 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cgroups_split", 2023-11-27T15:32:30.809 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.809 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.809 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.809 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.810 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.810 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.810 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.810 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-27T15:32:30.810 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.810 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.810 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.811 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.811 INFO:teuthology.orchestra.run.smithi008.stdout: "config_checks_enabled": { 2023-11-27T15:32:30.811 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_checks_enabled", 2023-11-27T15:32:30.811 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.811 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.811 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.811 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.811 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.811 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.812 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.812 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-27T15:32:30.812 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.812 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.812 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.812 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.812 INFO:teuthology.orchestra.run.smithi008.stdout: "config_dashboard": { 2023-11-27T15:32:30.812 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_dashboard", 2023-11-27T15:32:30.813 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.813 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.813 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.813 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.813 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.813 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.813 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.813 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-27T15:32:30.814 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.814 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.814 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.814 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.814 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_alertmanager": { 2023-11-27T15:32:30.814 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_alertmanager", 2023-11-27T15:32:30.814 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.814 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.815 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.815 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-27T15:32:30.815 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.815 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.815 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.815 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:30.815 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.815 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.816 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.816 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.816 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_base": { 2023-11-27T15:32:30.816 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_base", 2023-11-27T15:32:30.816 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.816 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.816 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.816 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-27T15:32:30.817 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.817 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.817 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.817 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Container image name, without the tag", 2023-11-27T15:32:30.817 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.817 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.817 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.817 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.818 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_elasticsearch": { 2023-11-27T15:32:30.818 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_elasticsearch", 2023-11-27T15:32:30.818 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.818 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.818 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.818 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-27T15:32:30.818 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.818 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.819 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.819 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "elasticsearch container image", 2023-11-27T15:32:30.819 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.819 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.819 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.819 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.819 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_grafana": { 2023-11-27T15:32:30.819 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_grafana", 2023-11-27T15:32:30.820 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.820 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.820 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.820 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-27T15:32:30.820 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.820 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.820 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.820 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:30.820 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.821 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.821 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.821 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.821 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_haproxy": { 2023-11-27T15:32:30.821 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_haproxy", 2023-11-27T15:32:30.821 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.821 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.821 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.822 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-27T15:32:30.822 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.822 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.822 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.822 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "HAproxy container image", 2023-11-27T15:32:30.822 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.822 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.822 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.823 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.823 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_agent": { 2023-11-27T15:32:30.823 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_agent", 2023-11-27T15:32:30.823 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.823 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.823 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.823 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-27T15:32:30.823 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.824 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.824 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.824 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger agent container image", 2023-11-27T15:32:30.824 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.824 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.824 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.824 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.824 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_collector": { 2023-11-27T15:32:30.825 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_collector", 2023-11-27T15:32:30.825 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.825 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.825 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.825 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-27T15:32:30.825 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.825 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.825 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.826 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger collector container image", 2023-11-27T15:32:30.826 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.826 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.826 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.826 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.826 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_jaeger_query": { 2023-11-27T15:32:30.826 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_jaeger_query", 2023-11-27T15:32:30.827 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.827 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.827 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.827 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-27T15:32:30.827 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.827 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.827 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.827 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Jaeger query container image", 2023-11-27T15:32:30.828 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.828 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.828 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.828 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.828 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_keepalived": { 2023-11-27T15:32:30.828 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_keepalived", 2023-11-27T15:32:30.828 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.828 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.829 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.829 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-27T15:32:30.829 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.829 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.829 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.829 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Keepalived container image", 2023-11-27T15:32:30.829 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.829 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.830 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.830 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.830 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_loki": { 2023-11-27T15:32:30.830 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_loki", 2023-11-27T15:32:30.830 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.830 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.830 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.830 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-27T15:32:30.831 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.831 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.831 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.831 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Loki container image", 2023-11-27T15:32:30.831 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.831 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.831 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.831 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.832 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_node_exporter": { 2023-11-27T15:32:30.832 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_node_exporter", 2023-11-27T15:32:30.832 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.832 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.832 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.832 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-27T15:32:30.832 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.832 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.833 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.833 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:30.833 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.833 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.833 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.833 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.833 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_prometheus": { 2023-11-27T15:32:30.833 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_prometheus", 2023-11-27T15:32:30.834 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.834 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.834 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.834 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-27T15:32:30.834 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.834 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.834 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.834 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus container image", 2023-11-27T15:32:30.835 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.835 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.835 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.835 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.835 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_promtail": { 2023-11-27T15:32:30.835 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_promtail", 2023-11-27T15:32:30.835 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.835 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.836 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.836 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-27T15:32:30.836 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.836 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.836 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.836 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Promtail container image", 2023-11-27T15:32:30.836 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.836 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.837 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.837 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.837 INFO:teuthology.orchestra.run.smithi008.stdout: "container_image_snmp_gateway": { 2023-11-27T15:32:30.837 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_image_snmp_gateway", 2023-11-27T15:32:30.837 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.837 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.837 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.837 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-27T15:32:30.838 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.838 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.838 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.838 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "SNMP Gateway container image", 2023-11-27T15:32:30.838 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.838 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.838 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.838 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.839 INFO:teuthology.orchestra.run.smithi008.stdout: "container_init": { 2023-11-27T15:32:30.839 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "container_init", 2023-11-27T15:32:30.839 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.839 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.839 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.839 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.839 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.840 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.840 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.840 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Run podman/docker with `--init`", 2023-11-27T15:32:30.840 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.840 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.840 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.840 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.840 INFO:teuthology.orchestra.run.smithi008.stdout: "daemon_cache_timeout": { 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "daemon_cache_timeout", 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-27T15:32:30.842 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.842 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.842 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.842 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.842 INFO:teuthology.orchestra.run.smithi008.stdout: "default_cephadm_command_timeout": { 2023-11-27T15:32:30.842 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_cephadm_command_timeout", 2023-11-27T15:32:30.842 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.843 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.843 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.843 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "900", 2023-11-27T15:32:30.843 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.843 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.843 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.843 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-27T15:32:30.843 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.844 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.844 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.844 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.844 INFO:teuthology.orchestra.run.smithi008.stdout: "default_registry": { 2023-11-27T15:32:30.844 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "default_registry", 2023-11-27T15:32:30.844 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.844 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.844 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.845 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "docker.io", 2023-11-27T15:32:30.845 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.845 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.845 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.845 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-27T15:32:30.845 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.845 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.845 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.846 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.846 INFO:teuthology.orchestra.run.smithi008.stdout: "device_cache_timeout": { 2023-11-27T15:32:30.846 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_cache_timeout", 2023-11-27T15:32:30.846 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.846 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.846 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.846 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1800", 2023-11-27T15:32:30.846 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.847 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.847 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.847 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache device inventory", 2023-11-27T15:32:30.847 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.847 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.847 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.847 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.847 INFO:teuthology.orchestra.run.smithi008.stdout: "device_enhanced_scan": { 2023-11-27T15:32:30.848 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_enhanced_scan", 2023-11-27T15:32:30.848 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.848 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.848 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.848 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.848 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.848 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.848 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.849 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-27T15:32:30.849 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.849 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.849 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.849 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.849 INFO:teuthology.orchestra.run.smithi008.stdout: "facts_cache_timeout": { 2023-11-27T15:32:30.849 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "facts_cache_timeout", 2023-11-27T15:32:30.849 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.850 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.850 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.850 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:30.850 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.850 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.850 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.850 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "seconds to cache host facts data", 2023-11-27T15:32:30.850 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.851 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.851 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.851 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.851 INFO:teuthology.orchestra.run.smithi008.stdout: "host_check_interval": { 2023-11-27T15:32:30.851 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "host_check_interval", 2023-11-27T15:32:30.851 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.851 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.851 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.852 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:30.852 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.852 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.852 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.852 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to perform a host check", 2023-11-27T15:32:30.852 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.852 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.852 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.852 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.853 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.853 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.853 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.853 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.853 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.853 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.853 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.854 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.854 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.854 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.854 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.854 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.854 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.854 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.854 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.854 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.855 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.855 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.855 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.855 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.855 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.855 INFO:teuthology.orchestra.run.smithi008.stdout: "log_refresh_metadata": { 2023-11-27T15:32:30.855 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_refresh_metadata", 2023-11-27T15:32:30.855 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.856 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.856 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.856 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.856 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.856 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.856 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.856 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-27T15:32:30.856 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.857 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.857 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.857 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.857 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.857 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.857 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.857 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.858 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.858 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.858 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.858 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.858 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.858 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-27T15:32:30.858 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.858 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.859 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.859 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.859 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.859 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.859 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.859 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.859 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.859 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.859 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.860 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.860 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.860 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.860 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.860 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.860 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.860 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.861 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.861 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.861 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.861 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.861 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.861 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.861 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.861 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.862 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.862 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.862 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.862 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.862 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.862 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.862 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.862 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.863 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.863 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.863 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.863 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.863 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.863 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-27T15:32:30.863 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-27T15:32:30.863 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.864 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.864 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.864 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.864 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.864 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.864 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.864 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-27T15:32:30.864 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.865 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.865 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.865 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.865 INFO:teuthology.orchestra.run.smithi008.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-27T15:32:30.865 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-27T15:32:30.865 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.865 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.865 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.866 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "*", 2023-11-27T15:32:30.866 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.866 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.866 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.866 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-27T15:32:30.866 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.866 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.866 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.867 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.867 INFO:teuthology.orchestra.run.smithi008.stdout: "max_count_per_host": { 2023-11-27T15:32:30.867 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_count_per_host", 2023-11-27T15:32:30.867 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.867 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.867 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.867 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:30.867 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.868 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.868 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.868 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of daemons per service per host", 2023-11-27T15:32:30.868 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.868 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.868 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.868 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.868 INFO:teuthology.orchestra.run.smithi008.stdout: "max_osd_draining_count": { 2023-11-27T15:32:30.869 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_osd_draining_count", 2023-11-27T15:32:30.869 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.869 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.869 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.869 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:30.869 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.869 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.869 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.870 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-27T15:32:30.870 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.870 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.870 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.870 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.870 INFO:teuthology.orchestra.run.smithi008.stdout: "migration_current": { 2023-11-27T15:32:30.870 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "migration_current", 2023-11-27T15:32:30.870 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.871 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.871 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.871 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.871 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.871 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.871 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.871 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "internal - do not modify", 2023-11-27T15:32:30.872 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.872 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.872 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.872 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.872 INFO:teuthology.orchestra.run.smithi008.stdout: "mode": { 2023-11-27T15:32:30.872 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mode", 2023-11-27T15:32:30.872 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.872 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.872 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.873 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "root", 2023-11-27T15:32:30.873 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.873 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.873 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.873 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm-package", 2023-11-27T15:32:30.873 INFO:teuthology.orchestra.run.smithi008.stdout: "root" 2023-11-27T15:32:30.873 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.873 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "mode for remote execution of cephadm", 2023-11-27T15:32:30.874 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.874 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.874 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.874 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.874 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_alerts_path": { 2023-11-27T15:32:30.874 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_alerts_path", 2023-11-27T15:32:30.874 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.875 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.875 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.875 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-27T15:32:30.875 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.875 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.875 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.875 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-27T15:32:30.875 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.876 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.876 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.876 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.876 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_web_password": { 2023-11-27T15:32:30.876 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_web_password", 2023-11-27T15:32:30.876 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.876 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.876 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.876 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:30.877 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.877 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.877 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.877 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus web password", 2023-11-27T15:32:30.877 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.877 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.877 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.877 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.878 INFO:teuthology.orchestra.run.smithi008.stdout: "prometheus_web_user": { 2023-11-27T15:32:30.878 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus_web_user", 2023-11-27T15:32:30.878 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.878 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.878 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.878 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:30.878 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.878 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.879 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.879 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Prometheus web user", 2023-11-27T15:32:30.879 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.879 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.879 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.879 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.879 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_insecure": { 2023-11-27T15:32:30.879 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_insecure", 2023-11-27T15:32:30.880 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.880 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.880 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.880 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.880 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.880 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.880 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.880 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-27T15:32:30.881 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.881 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.881 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.881 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.881 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_password": { 2023-11-27T15:32:30.881 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_password", 2023-11-27T15:32:30.882 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.882 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.882 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.882 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.882 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.882 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.883 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.883 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-27T15:32:30.883 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.883 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.883 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.883 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.884 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_url": { 2023-11-27T15:32:30.884 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_url", 2023-11-27T15:32:30.884 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.884 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.884 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.884 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.884 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.885 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.885 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.885 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-27T15:32:30.885 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.885 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.885 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.885 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.886 INFO:teuthology.orchestra.run.smithi008.stdout: "registry_username": { 2023-11-27T15:32:30.886 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "registry_username", 2023-11-27T15:32:30.886 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.886 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.886 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.886 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.887 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.887 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.887 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.887 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-27T15:32:30.887 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.887 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.887 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.888 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.888 INFO:teuthology.orchestra.run.smithi008.stdout: "secure_monitoring_stack": { 2023-11-27T15:32:30.888 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "secure_monitoring_stack", 2023-11-27T15:32:30.888 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.888 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.888 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.888 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.889 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.889 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.889 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.889 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-27T15:32:30.889 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.889 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.890 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.890 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.890 INFO:teuthology.orchestra.run.smithi008.stdout: "service_discovery_port": { 2023-11-27T15:32:30.890 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "service_discovery_port", 2023-11-27T15:32:30.890 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.890 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.891 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.891 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8765", 2023-11-27T15:32:30.891 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.891 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.891 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.891 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "cephadm service discovery port", 2023-11-27T15:32:30.892 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.892 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.892 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.892 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.892 INFO:teuthology.orchestra.run.smithi008.stdout: "ssh_config_file": { 2023-11-27T15:32:30.892 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssh_config_file", 2023-11-27T15:32:30.892 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.892 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.893 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.893 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.893 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.893 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.893 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.893 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-27T15:32:30.894 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.894 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.894 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.894 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.894 INFO:teuthology.orchestra.run.smithi008.stdout: "use_agent": { 2023-11-27T15:32:30.894 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_agent", 2023-11-27T15:32:30.894 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.895 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.895 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.895 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.895 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.895 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.895 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.895 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-27T15:32:30.896 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.896 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.896 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.896 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.896 INFO:teuthology.orchestra.run.smithi008.stdout: "use_repo_digest": { 2023-11-27T15:32:30.896 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "use_repo_digest", 2023-11-27T15:32:30.896 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.897 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.897 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.897 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.897 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.897 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.897 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.897 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-27T15:32:30.898 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.898 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.898 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.898 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.898 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_failed_host_check": { 2023-11-27T15:32:30.898 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_failed_host_check", 2023-11-27T15:32:30.898 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.899 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.899 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.899 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.899 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.899 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.899 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.900 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if the host check fails", 2023-11-27T15:32:30.900 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.900 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.900 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.900 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.900 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_daemons": { 2023-11-27T15:32:30.900 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_daemons", 2023-11-27T15:32:30.900 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.901 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.901 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.901 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.901 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.901 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.901 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.901 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-27T15:32:30.902 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.902 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.902 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.902 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.902 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_on_stray_hosts": { 2023-11-27T15:32:30.902 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_on_stray_hosts", 2023-11-27T15:32:30.902 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.903 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.903 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.903 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.903 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.903 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.903 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.903 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-27T15:32:30.904 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.904 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.904 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.904 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.904 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.904 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.904 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.904 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crash", 2023-11-27T15:32:30.905 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.905 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.905 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.905 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:30.905 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:30.905 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.905 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.906 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.906 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.906 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.906 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.906 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.906 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.906 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.907 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.907 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.907 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.907 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.907 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.907 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.907 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.908 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.908 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.908 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.908 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:30.908 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:30.908 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.908 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.909 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.909 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.909 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.909 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.909 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.909 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.909 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.910 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.910 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.910 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.910 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:30.910 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:30.910 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.910 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.911 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.911 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:30.911 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.911 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.911 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:30.911 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:30.911 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:30.912 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:30.912 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:30.912 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:30.912 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:30.912 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:30.912 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.912 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.912 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.913 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.913 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.913 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:30.913 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:30.913 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.913 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.913 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.913 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.914 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.914 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.914 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.914 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.914 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.914 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.914 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.915 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.915 INFO:teuthology.orchestra.run.smithi008.stdout: "retain_interval": { 2023-11-27T15:32:30.915 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retain_interval", 2023-11-27T15:32:30.915 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.915 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.915 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.915 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "31536000", 2023-11-27T15:32:30.916 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.916 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.916 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.916 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-27T15:32:30.916 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.916 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.916 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.916 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.917 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_recent_interval": { 2023-11-27T15:32:30.917 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_recent_interval", 2023-11-27T15:32:30.917 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:30.917 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.917 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:30.917 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1209600", 2023-11-27T15:32:30.917 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.918 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.918 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.918 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-27T15:32:30.918 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.918 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.918 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.918 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.919 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:30.919 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.919 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:30.919 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dashboard", 2023-11-27T15:32:30.919 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:30.919 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:30.919 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:30.920 INFO:teuthology.orchestra.run.smithi008.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-27T15:32:30.920 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-27T15:32:30.920 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.920 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.920 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.920 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:30.920 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.921 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.921 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.921 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.921 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.921 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.921 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.921 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.922 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_HOST": { 2023-11-27T15:32:30.922 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-27T15:32:30.922 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.922 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.922 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.922 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.922 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.923 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.923 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.923 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.923 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.923 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.923 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.923 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.923 INFO:teuthology.orchestra.run.smithi008.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-27T15:32:30.924 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-27T15:32:30.924 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.924 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.924 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.924 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.924 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.924 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.924 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.925 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.925 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.925 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.925 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.925 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.925 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_ENABLED": { 2023-11-27T15:32:30.925 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_ENABLED", 2023-11-27T15:32:30.926 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.926 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.926 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.926 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.926 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.926 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.926 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.927 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.927 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.927 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.927 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.927 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.927 INFO:teuthology.orchestra.run.smithi008.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-27T15:32:30.927 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-27T15:32:30.927 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.928 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.928 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.928 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.928 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.928 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.928 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.928 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.929 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.929 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.929 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.929 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.929 INFO:teuthology.orchestra.run.smithi008.stdout: "ENABLE_BROWSABLE_API": { 2023-11-27T15:32:30.929 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-27T15:32:30.929 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.930 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.930 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.930 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.930 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.930 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.930 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.930 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.931 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.931 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.931 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.931 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.931 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-27T15:32:30.931 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-27T15:32:30.931 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.931 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.932 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.932 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.932 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.932 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.932 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.932 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.932 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.933 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.933 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.933 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.933 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-27T15:32:30.933 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-27T15:32:30.933 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.933 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.933 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.934 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.934 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.934 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.934 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.934 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.934 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.934 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.935 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.935 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.935 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-27T15:32:30.935 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-27T15:32:30.935 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.935 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.935 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.936 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.936 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.936 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.936 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.936 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.936 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.936 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.936 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.937 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.937 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-27T15:32:30.937 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-27T15:32:30.937 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.937 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.937 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.937 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.938 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.938 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.938 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.938 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.938 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.938 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.938 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.939 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.939 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-27T15:32:30.939 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-27T15:32:30.939 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.939 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.939 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.939 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.939 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.940 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.940 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.940 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.940 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.940 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.940 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.940 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.941 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-27T15:32:30.941 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-27T15:32:30.941 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.941 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.941 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.941 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.941 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.941 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.942 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.942 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.942 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.942 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.942 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.942 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.942 INFO:teuthology.orchestra.run.smithi008.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-27T15:32:30.943 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-27T15:32:30.943 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.943 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.943 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.943 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.943 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.943 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.943 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.944 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.944 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.944 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.944 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.944 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.944 INFO:teuthology.orchestra.run.smithi008.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-27T15:32:30.944 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-27T15:32:30.945 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.945 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.945 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.945 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.945 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.945 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.945 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.945 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.946 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.946 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.946 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.946 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.946 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_PASSWORD": { 2023-11-27T15:32:30.946 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-27T15:32:30.946 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.947 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.947 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.947 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:30.947 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.947 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.947 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.947 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.947 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.948 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.948 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.948 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.948 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-27T15:32:30.948 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-27T15:32:30.948 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.948 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.949 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.949 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.949 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.949 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.949 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.949 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.949 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.950 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.950 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.950 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.950 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_URL": { 2023-11-27T15:32:30.950 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_URL", 2023-11-27T15:32:30.950 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.950 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.951 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.951 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.951 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.951 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.951 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.951 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.951 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.951 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.952 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.952 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.952 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_API_USERNAME": { 2023-11-27T15:32:30.952 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-27T15:32:30.952 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.952 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.952 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.952 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:30.953 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.953 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.953 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.953 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.953 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.953 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.953 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.954 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.954 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-27T15:32:30.954 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-27T15:32:30.954 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.954 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.954 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.954 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.954 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.955 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.955 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.955 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.955 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.955 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.955 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.955 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.955 INFO:teuthology.orchestra.run.smithi008.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-27T15:32:30.956 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-27T15:32:30.956 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.956 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.956 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.956 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.956 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.956 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.957 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.957 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.957 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.957 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.957 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.957 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.957 INFO:teuthology.orchestra.run.smithi008.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-27T15:32:30.957 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-27T15:32:30.958 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.958 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.958 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.958 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.958 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.958 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.958 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.959 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.959 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.959 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.959 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.959 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.959 INFO:teuthology.orchestra.run.smithi008.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-27T15:32:30.959 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-27T15:32:30.960 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.960 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.960 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.960 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.960 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.960 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.960 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.960 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.961 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.961 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.961 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.961 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.961 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_HOST": { 2023-11-27T15:32:30.961 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-27T15:32:30.961 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.961 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.962 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.962 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.962 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.962 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.962 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.962 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.962 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.963 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.963 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.963 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.963 INFO:teuthology.orchestra.run.smithi008.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-27T15:32:30.963 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-27T15:32:30.963 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.963 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.964 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.964 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.964 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.964 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.964 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.964 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.965 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.965 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.965 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.965 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.965 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-27T15:32:30.965 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-27T15:32:30.965 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.966 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.966 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.966 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.966 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.966 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.966 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.966 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.967 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.967 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.967 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.967 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.967 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-27T15:32:30.967 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-27T15:32:30.968 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.968 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.968 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.968 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.968 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.968 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.968 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.969 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.969 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.969 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.969 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.969 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.969 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-27T15:32:30.969 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-27T15:32:30.969 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.970 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.970 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.970 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.970 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.970 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.970 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.970 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.971 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.971 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.971 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.971 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.971 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-27T15:32:30.971 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-27T15:32:30.971 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.971 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.972 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.972 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.972 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.972 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.972 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.972 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.972 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.973 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.973 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.973 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.973 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-27T15:32:30.973 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-27T15:32:30.973 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.973 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.973 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.974 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.974 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.974 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.974 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.974 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.974 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.974 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.975 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.975 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.975 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-27T15:32:30.975 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-27T15:32:30.975 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.975 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.975 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.976 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.976 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.976 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.976 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.976 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.976 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.976 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.977 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.977 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.977 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-27T15:32:30.977 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-27T15:32:30.977 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.977 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.978 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.978 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:30.978 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.978 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.978 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.978 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.978 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.979 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.979 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.979 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.979 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_ENABLED": { 2023-11-27T15:32:30.979 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-27T15:32:30.979 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.979 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.979 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.980 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.980 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.980 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.980 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.980 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.980 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.980 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.981 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.981 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.981 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-27T15:32:30.981 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-27T15:32:30.981 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.981 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.981 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.982 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-27T15:32:30.982 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.982 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.982 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.982 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.982 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.982 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.983 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.983 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.983 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-27T15:32:30.983 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-27T15:32:30.983 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.983 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.983 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.983 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:30.984 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.984 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.984 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.984 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.984 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.984 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.984 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.984 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.985 INFO:teuthology.orchestra.run.smithi008.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-27T15:32:30.985 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-27T15:32:30.985 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.985 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.985 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.985 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8", 2023-11-27T15:32:30.985 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.986 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.986 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.986 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.986 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.986 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.986 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.986 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.987 INFO:teuthology.orchestra.run.smithi008.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-27T15:32:30.987 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-27T15:32:30.987 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.987 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.987 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.987 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "45", 2023-11-27T15:32:30.987 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.987 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.988 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.988 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.988 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.988 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.988 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.988 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.988 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ACCESS_KEY": { 2023-11-27T15:32:30.989 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-27T15:32:30.989 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.989 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.989 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.989 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.989 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.989 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.989 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.990 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.990 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.990 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.990 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.990 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.990 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-27T15:32:30.990 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-27T15:32:30.991 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.991 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.991 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.991 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "admin", 2023-11-27T15:32:30.991 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.991 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.991 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.991 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.992 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.992 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.992 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.992 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.992 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SECRET_KEY": { 2023-11-27T15:32:30.992 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-27T15:32:30.993 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:30.993 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.993 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.993 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:30.993 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.993 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.993 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.993 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.994 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.994 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.994 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.994 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.994 INFO:teuthology.orchestra.run.smithi008.stdout: "RGW_API_SSL_VERIFY": { 2023-11-27T15:32:30.994 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-27T15:32:30.994 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:30.995 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.995 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.995 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:30.995 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.995 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.995 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.995 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.995 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.996 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.996 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.996 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.996 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-27T15:32:30.996 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-27T15:32:30.996 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.996 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.996 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.997 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:30.997 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.997 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.997 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.997 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.997 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.997 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:30.998 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:30.998 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:30.998 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-27T15:32:30.998 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-27T15:32:30.998 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:30.998 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:30.998 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:30.999 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:30.999 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:30.999 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:30.999 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:30.999 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:30.999 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:30.999 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.000 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.000 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.000 INFO:teuthology.orchestra.run.smithi008.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-27T15:32:31.000 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-27T15:32:31.000 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.000 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.001 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.001 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:31.001 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.001 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.001 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.001 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.001 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.001 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.002 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.002 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.002 INFO:teuthology.orchestra.run.smithi008.stdout: "cross_origin_url": { 2023-11-27T15:32:31.002 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cross_origin_url", 2023-11-27T15:32:31.002 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.002 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.002 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.002 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.003 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.003 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.003 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.003 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.003 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.003 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.003 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.004 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.004 INFO:teuthology.orchestra.run.smithi008.stdout: "crt_file": { 2023-11-27T15:32:31.004 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "crt_file", 2023-11-27T15:32:31.004 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.004 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.004 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.004 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.004 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.005 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.005 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.005 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.005 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.005 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.005 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.005 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.006 INFO:teuthology.orchestra.run.smithi008.stdout: "debug": { 2023-11-27T15:32:31.006 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "debug", 2023-11-27T15:32:31.006 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.006 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.006 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.006 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.006 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.007 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.007 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.007 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Enable/disable debug options", 2023-11-27T15:32:31.007 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.007 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.007 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.007 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.008 INFO:teuthology.orchestra.run.smithi008.stdout: "jwt_token_ttl": { 2023-11-27T15:32:31.008 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "jwt_token_ttl", 2023-11-27T15:32:31.008 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.008 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.008 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.008 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "28800", 2023-11-27T15:32:31.009 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.009 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.009 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.009 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.009 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.009 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.009 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.010 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.010 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2023-11-27T15:32:31.010 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2023-11-27T15:32:31.010 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.010 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.010 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.011 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.011 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.011 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.011 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.011 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.011 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.011 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.011 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.012 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.012 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.012 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.012 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.012 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.012 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.012 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.013 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.013 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.013 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.013 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.013 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.013 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.013 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.014 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.014 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.014 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.014 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.014 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.014 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.014 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.014 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.015 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.015 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.015 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.015 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.015 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.015 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.015 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.016 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.016 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.016 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.016 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.016 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.016 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.016 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.017 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.017 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.017 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.017 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.017 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.017 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.017 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.018 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.018 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.018 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.018 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.018 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.018 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.018 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.019 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.019 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.019 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.019 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.019 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.019 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.019 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.019 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.020 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.020 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.020 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.020 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.020 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.020 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.020 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.021 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.021 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.021 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.021 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.021 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.021 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.021 INFO:teuthology.orchestra.run.smithi008.stdout: "motd": { 2023-11-27T15:32:31.021 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "motd", 2023-11-27T15:32:31.022 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.022 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.022 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.022 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.022 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.022 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.022 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.023 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "The message of the day", 2023-11-27T15:32:31.023 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.023 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.023 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.023 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.023 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect_resolve_ip_addr": { 2023-11-27T15:32:31.023 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "redirect_resolve_ip_addr", 2023-11-27T15:32:31.023 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.024 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.024 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.024 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.024 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.024 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.024 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.024 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.025 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.025 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.025 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.025 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.025 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2023-11-27T15:32:31.025 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2023-11-27T15:32:31.025 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.025 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.026 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.026 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2023-11-27T15:32:31.026 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.026 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.026 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.026 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.026 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.026 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.027 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.027 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.027 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2023-11-27T15:32:31.027 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2023-11-27T15:32:31.027 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.027 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.027 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.028 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8080", 2023-11-27T15:32:31.028 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.028 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.028 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.028 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.028 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.028 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.028 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.029 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.029 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2023-11-27T15:32:31.029 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2023-11-27T15:32:31.029 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.029 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.029 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.029 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.030 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.030 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.030 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.030 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.030 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.030 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.031 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.031 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.031 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl_server_port": { 2023-11-27T15:32:31.031 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl_server_port", 2023-11-27T15:32:31.031 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.031 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.031 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.031 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8443", 2023-11-27T15:32:31.032 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.032 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.032 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.032 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.032 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.032 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.032 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.032 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.033 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2023-11-27T15:32:31.033 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2023-11-27T15:32:31.033 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.033 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.033 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.033 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "redirect", 2023-11-27T15:32:31.033 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.033 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.034 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.034 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.034 INFO:teuthology.orchestra.run.smithi008.stdout: "redirect" 2023-11-27T15:32:31.034 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.034 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.034 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.034 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.035 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.035 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.035 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2023-11-27T15:32:31.035 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2023-11-27T15:32:31.035 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.035 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.035 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.035 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2023-11-27T15:32:31.036 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2023-11-27T15:32:31.036 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2023-11-27T15:32:31.036 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.036 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.036 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.036 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.036 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.037 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.037 INFO:teuthology.orchestra.run.smithi008.stdout: "url_prefix": { 2023-11-27T15:32:31.037 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url_prefix", 2023-11-27T15:32:31.037 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.037 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.037 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.037 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.037 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.038 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.038 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.038 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.038 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.038 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.038 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.038 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.039 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.039 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.039 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.039 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "devicehealth", 2023-11-27T15:32:31.039 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.039 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.040 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.040 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_monitoring": { 2023-11-27T15:32:31.040 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_monitoring", 2023-11-27T15:32:31.040 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.040 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.040 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.041 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.041 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.041 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.041 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.041 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "monitor device health metrics", 2023-11-27T15:32:31.041 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.042 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.042 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.042 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.042 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.042 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.042 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.042 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.043 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.043 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.043 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.043 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.043 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.043 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.043 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.044 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.044 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.044 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.044 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.044 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.044 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.044 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.044 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.045 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.045 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.045 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.045 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.045 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.045 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.045 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.045 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.046 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.046 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.046 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.046 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.046 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.046 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.046 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.047 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.047 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.047 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.047 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.047 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.047 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.047 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.048 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.048 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.048 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.048 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.048 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.048 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.048 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.048 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.049 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.049 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.049 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.049 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.049 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.049 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.049 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.050 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.050 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.050 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.050 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.050 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.050 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.050 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.050 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.051 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.051 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.051 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.051 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.051 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.051 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.051 INFO:teuthology.orchestra.run.smithi008.stdout: "mark_out_threshold": { 2023-11-27T15:32:31.052 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mark_out_threshold", 2023-11-27T15:32:31.052 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:31.052 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.052 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.052 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "2419200", 2023-11-27T15:32:31.052 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.052 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.052 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.053 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-27T15:32:31.053 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.053 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.053 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.053 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.053 INFO:teuthology.orchestra.run.smithi008.stdout: "pool_name": { 2023-11-27T15:32:31.053 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pool_name", 2023-11-27T15:32:31.053 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.054 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.054 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.054 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "device_health_metrics", 2023-11-27T15:32:31.054 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.054 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.054 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.054 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-27T15:32:31.055 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.055 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.055 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.055 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.055 INFO:teuthology.orchestra.run.smithi008.stdout: "retention_period": { 2023-11-27T15:32:31.055 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "retention_period", 2023-11-27T15:32:31.055 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:31.056 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.056 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.056 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15552000", 2023-11-27T15:32:31.056 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.056 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.056 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.056 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long to retain device health metrics", 2023-11-27T15:32:31.056 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.057 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.057 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.057 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.057 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_frequency": { 2023-11-27T15:32:31.057 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_frequency", 2023-11-27T15:32:31.057 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:31.057 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.058 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.058 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2023-11-27T15:32:31.058 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.058 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.058 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.058 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-27T15:32:31.058 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.059 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.059 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.059 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.059 INFO:teuthology.orchestra.run.smithi008.stdout: "self_heal": { 2023-11-27T15:32:31.059 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "self_heal", 2023-11-27T15:32:31.059 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.059 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.059 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.060 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.060 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.060 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.060 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.060 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-27T15:32:31.060 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.060 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.060 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.061 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.061 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:31.061 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:31.061 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:31.061 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.061 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.061 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:31.062 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.062 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.062 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.062 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how frequently to wake up and check device health", 2023-11-27T15:32:31.062 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.062 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.062 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.063 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.063 INFO:teuthology.orchestra.run.smithi008.stdout: "warn_threshold": { 2023-11-27T15:32:31.063 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "warn_threshold", 2023-11-27T15:32:31.063 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:31.063 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.063 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.063 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7257600", 2023-11-27T15:32:31.063 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.064 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.064 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.064 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-27T15:32:31.064 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.064 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.064 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.064 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.064 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.065 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.065 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.065 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "diskprediction_local", 2023-11-27T15:32:31.065 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.065 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.065 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.065 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.065 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.066 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.066 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.066 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.066 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.066 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.066 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.066 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.067 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.067 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.067 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.067 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.067 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.067 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.068 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.068 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.068 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.068 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.068 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.068 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.068 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.069 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.069 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.069 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.069 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.069 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.069 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.069 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.069 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.070 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.070 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.070 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.070 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.070 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.070 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.070 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.071 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.071 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.071 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.071 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.071 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.071 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.071 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.072 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.072 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.072 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.072 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.072 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.072 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.072 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.072 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.073 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.073 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.073 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.073 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.073 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.073 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.073 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.074 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.074 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.074 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.074 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.074 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.074 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.074 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.074 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.075 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.075 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.075 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.075 INFO:teuthology.orchestra.run.smithi008.stdout: "predict_interval": { 2023-11-27T15:32:31.075 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predict_interval", 2023-11-27T15:32:31.075 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.075 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.076 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.076 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "86400", 2023-11-27T15:32:31.076 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.076 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.076 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.076 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.076 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.077 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.077 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.077 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.077 INFO:teuthology.orchestra.run.smithi008.stdout: "predictor_model": { 2023-11-27T15:32:31.077 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "predictor_model", 2023-11-27T15:32:31.077 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.077 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.078 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.078 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "prophetstor", 2023-11-27T15:32:31.078 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.078 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.078 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.078 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.078 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.079 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.079 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.079 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.079 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:31.079 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:31.079 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.079 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.079 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.080 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "600", 2023-11-27T15:32:31.080 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.080 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.080 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.080 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.080 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.080 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.080 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.081 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.081 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.081 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.081 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.081 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "influx", 2023-11-27T15:32:31.081 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": false, 2023-11-27T15:32:31.081 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "influxdb python module not found", 2023-11-27T15:32:31.082 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.082 INFO:teuthology.orchestra.run.smithi008.stdout: "batch_size": { 2023-11-27T15:32:31.082 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "batch_size", 2023-11-27T15:32:31.082 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.082 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.082 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.082 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5000", 2023-11-27T15:32:31.083 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.083 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.083 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.083 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-27T15:32:31.083 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.083 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.083 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.083 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.084 INFO:teuthology.orchestra.run.smithi008.stdout: "database": { 2023-11-27T15:32:31.084 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "database", 2023-11-27T15:32:31.084 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.084 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.084 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.084 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "ceph", 2023-11-27T15:32:31.084 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.085 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.085 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.085 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-27T15:32:31.085 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.085 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.085 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: "hostname": { 2023-11-27T15:32:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "hostname", 2023-11-27T15:32:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.087 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.087 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.087 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.087 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server hostname", 2023-11-27T15:32:31.087 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.087 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.088 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.088 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.088 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:31.088 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:31.088 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:31.088 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.088 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.089 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "30", 2023-11-27T15:32:31.089 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "5", 2023-11-27T15:32:31.089 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.089 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.089 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-27T15:32:31.089 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.089 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.089 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.090 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.090 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.090 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.090 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.090 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.090 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.090 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.091 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.091 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.091 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.091 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.091 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.091 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.091 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.091 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.092 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.092 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.092 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.092 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.092 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.092 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.092 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.092 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.093 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.093 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.093 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.093 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.093 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.093 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.093 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.094 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.094 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.094 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.094 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.094 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.094 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.094 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.094 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.095 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.095 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.095 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.095 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.095 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.095 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.095 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.095 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.096 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.096 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.096 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.096 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.096 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.096 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.096 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.097 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.097 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.097 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.097 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.097 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.097 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.097 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.098 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.098 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.098 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.098 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.098 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.098 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.098 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.099 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.099 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.099 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.099 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.099 INFO:teuthology.orchestra.run.smithi008.stdout: "password": { 2023-11-27T15:32:31.099 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "password", 2023-11-27T15:32:31.099 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.099 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.100 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.100 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.100 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.100 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.100 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.100 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "password of InfluxDB server user", 2023-11-27T15:32:31.100 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.100 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.101 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.101 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.101 INFO:teuthology.orchestra.run.smithi008.stdout: "port": { 2023-11-27T15:32:31.101 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "port", 2023-11-27T15:32:31.101 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.101 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.101 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.102 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "8086", 2023-11-27T15:32:31.102 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.102 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.102 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.103 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "InfluxDB server port", 2023-11-27T15:32:31.103 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.103 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.103 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.103 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.103 INFO:teuthology.orchestra.run.smithi008.stdout: "ssl": { 2023-11-27T15:32:31.103 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ssl", 2023-11-27T15:32:31.104 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.104 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.104 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.104 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "false", 2023-11-27T15:32:31.104 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.104 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.104 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.104 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:32:31.105 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.105 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.105 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.105 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.105 INFO:teuthology.orchestra.run.smithi008.stdout: "threads": { 2023-11-27T15:32:31.105 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threads", 2023-11-27T15:32:31.105 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.105 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.106 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.106 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:31.106 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:31.106 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "32", 2023-11-27T15:32:31.106 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.106 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-27T15:32:31.106 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.106 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.107 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.107 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.107 INFO:teuthology.orchestra.run.smithi008.stdout: "username": { 2023-11-27T15:32:31.107 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "username", 2023-11-27T15:32:31.107 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.107 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.107 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.108 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.108 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.108 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.108 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.108 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "username of InfluxDB server user", 2023-11-27T15:32:31.108 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.108 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.108 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.109 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.109 INFO:teuthology.orchestra.run.smithi008.stdout: "verify_ssl": { 2023-11-27T15:32:31.109 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "verify_ssl", 2023-11-27T15:32:31.109 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.109 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.109 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.109 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "true", 2023-11-27T15:32:31.110 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.110 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.110 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.110 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:32:31.110 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.110 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.110 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.110 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.111 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.111 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.111 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.111 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "insights", 2023-11-27T15:32:31.111 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.111 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.111 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.111 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.112 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.112 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.112 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.112 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.112 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.112 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.112 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.112 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.113 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.113 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.113 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.113 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.113 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.113 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.113 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.113 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.114 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.114 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.114 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.114 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.114 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.114 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.114 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.115 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.115 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.115 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.115 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.115 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.115 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.115 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.115 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.116 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.116 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.116 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.116 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.116 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.116 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.116 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.116 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.117 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.117 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.117 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.117 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.117 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.117 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.117 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.117 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.118 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.118 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.118 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.118 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.118 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.118 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.118 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.119 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.119 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.119 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.119 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.119 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.119 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.119 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.119 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.120 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.120 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.120 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.120 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.120 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.120 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.120 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.120 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.121 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.121 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.121 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "iostat", 2023-11-27T15:32:31.121 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.121 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.121 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.121 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.122 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.122 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.122 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.122 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.122 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.122 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.122 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.122 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.123 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.123 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.123 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.123 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.123 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.123 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.123 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.123 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.124 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.124 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.124 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.124 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.124 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.124 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.124 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.124 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.125 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.125 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.125 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.125 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.125 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.125 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.125 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.125 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.126 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.126 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.126 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.126 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.126 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.126 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.126 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.126 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.127 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.127 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.127 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.127 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.127 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.127 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.127 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.128 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.128 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.128 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.128 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.128 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.128 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.128 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.128 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.129 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.129 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.129 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.129 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.129 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.129 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.129 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.129 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.130 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.130 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.130 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.130 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.130 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.130 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.130 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.130 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.131 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.131 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "k8sevents", 2023-11-27T15:32:31.131 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.131 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.131 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.131 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_event_retention_days": { 2023-11-27T15:32:31.131 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "ceph_event_retention_days", 2023-11-27T15:32:31.131 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.132 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.132 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.132 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "7", 2023-11-27T15:32:31.132 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.132 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.132 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.132 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-27T15:32:31.132 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.133 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.133 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.133 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.133 INFO:teuthology.orchestra.run.smithi008.stdout: "config_check_secs": { 2023-11-27T15:32:31.133 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "config_check_secs", 2023-11-27T15:32:31.133 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.133 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.134 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.134 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:31.134 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "10", 2023-11-27T15:32:31.134 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.134 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.134 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-27T15:32:31.134 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.134 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.135 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.135 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.135 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.135 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.135 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.135 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.135 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.136 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.136 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.136 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.136 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.136 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.136 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.136 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.136 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.137 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.137 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.137 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.137 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.137 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.137 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.137 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.137 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.138 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.138 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.138 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.138 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.138 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.138 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.138 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.139 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.139 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.139 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.139 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.139 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.139 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.139 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.139 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.140 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.140 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.140 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.140 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.140 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.140 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.140 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.140 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.141 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.141 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.141 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.141 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.141 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.141 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.141 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.141 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.142 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.142 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.142 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.142 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.142 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.142 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.142 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.143 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.143 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.143 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.143 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.143 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.143 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.144 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.144 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.144 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.144 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.144 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.144 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.144 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.144 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.145 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "localpool", 2023-11-27T15:32:31.145 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.145 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.145 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.145 INFO:teuthology.orchestra.run.smithi008.stdout: "failure_domain": { 2023-11-27T15:32:31.145 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "failure_domain", 2023-11-27T15:32:31.145 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.146 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.146 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.146 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "host", 2023-11-27T15:32:31.146 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.146 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.146 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.147 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "failure domain for any created local pool", 2023-11-27T15:32:31.147 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-27T15:32:31.147 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.147 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.147 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.147 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.147 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.148 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.148 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.148 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.148 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.148 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.148 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.148 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.149 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.149 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.149 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.149 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.149 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.149 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.149 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.150 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.150 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.150 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.150 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.150 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.150 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.150 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.150 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.151 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.151 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.151 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.151 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.151 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.151 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.151 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.152 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.152 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.152 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.152 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.152 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.152 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.152 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.153 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.153 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.153 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.153 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.153 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.153 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.153 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.153 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.154 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.154 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.154 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.154 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.154 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.154 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.154 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.155 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.155 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.155 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.155 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.155 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.155 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.156 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.156 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.156 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.156 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.156 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.156 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.156 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.157 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.157 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.157 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.157 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.157 INFO:teuthology.orchestra.run.smithi008.stdout: "min_size": { 2023-11-27T15:32:31.157 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "min_size", 2023-11-27T15:32:31.157 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.158 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.158 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.158 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.158 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.158 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.158 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.159 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default min_size for any created local pool", 2023-11-27T15:32:31.159 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-27T15:32:31.159 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.159 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.159 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.159 INFO:teuthology.orchestra.run.smithi008.stdout: "num_rep": { 2023-11-27T15:32:31.159 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "num_rep", 2023-11-27T15:32:31.160 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.160 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.160 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.160 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3", 2023-11-27T15:32:31.160 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.160 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.161 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.161 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default replica count for any created local pool", 2023-11-27T15:32:31.161 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.161 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.161 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.161 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.161 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_num": { 2023-11-27T15:32:31.162 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_num", 2023-11-27T15:32:31.162 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.162 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.162 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.162 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "128", 2023-11-27T15:32:31.162 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.162 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.163 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.163 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "default pg_num for any created local pool", 2023-11-27T15:32:31.163 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.163 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.163 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.163 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.163 INFO:teuthology.orchestra.run.smithi008.stdout: "prefix": { 2023-11-27T15:32:31.164 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prefix", 2023-11-27T15:32:31.164 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.164 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.164 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.164 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.164 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.164 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.165 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.165 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "name prefix for any created local pool", 2023-11-27T15:32:31.165 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.165 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.165 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.165 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.165 INFO:teuthology.orchestra.run.smithi008.stdout: "subtree": { 2023-11-27T15:32:31.166 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "subtree", 2023-11-27T15:32:31.166 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.166 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.166 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.166 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "rack", 2023-11-27T15:32:31.166 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.166 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.167 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.167 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-27T15:32:31.167 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-27T15:32:31.167 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.167 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.167 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.167 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.168 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.168 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.168 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mds_autoscaler", 2023-11-27T15:32:31.168 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.168 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.168 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.168 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.169 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.169 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.169 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.169 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.169 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.169 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.169 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.169 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.170 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.170 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.170 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.170 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.170 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.170 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.170 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.171 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.171 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.171 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.171 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.171 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.171 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.171 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.171 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.172 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.172 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.172 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.172 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.172 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.172 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.172 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.173 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.173 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.173 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.173 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.173 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.173 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.173 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.173 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.174 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.174 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.174 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.174 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.174 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.174 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.174 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.175 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.175 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.175 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.175 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.175 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.175 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.175 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.175 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.176 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.176 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.176 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.176 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.176 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.176 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.177 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.177 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.177 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.177 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.177 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.177 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.177 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.178 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.178 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.178 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.178 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.178 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.178 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.178 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirroring", 2023-11-27T15:32:31.179 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.179 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.179 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.179 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.179 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.179 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.179 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.180 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.180 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.180 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.180 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.180 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.180 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.180 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.181 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.181 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.181 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.181 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.181 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.181 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.182 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.182 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.182 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.182 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.182 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.182 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.182 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.183 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.183 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.183 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.183 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.183 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.183 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.183 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.184 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.184 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.184 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.184 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.184 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.184 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.184 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.184 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.185 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.185 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.185 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.185 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.185 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.185 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.185 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.186 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.186 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.186 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.186 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.186 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.186 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.186 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.186 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.187 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.187 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.187 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.187 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.187 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.187 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.187 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.188 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.188 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.188 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.188 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.188 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.188 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.188 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.189 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.189 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.189 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.189 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.189 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.189 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "nfs", 2023-11-27T15:32:31.189 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.190 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.190 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.190 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.190 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.190 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.190 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.190 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.191 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.191 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.191 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.191 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.191 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.191 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.191 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.192 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.192 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.192 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.192 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.192 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.192 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.192 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.192 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.193 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.193 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.193 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.193 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.193 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.193 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.193 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.194 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.194 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.194 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.194 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.194 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.194 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.194 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.195 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.195 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.195 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.195 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.195 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.195 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.195 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.195 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.196 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.196 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.196 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.196 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.196 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.196 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.196 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.197 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.197 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.197 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.197 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.197 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.197 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.197 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.198 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.198 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.198 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.198 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.198 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.198 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.198 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.199 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.199 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.199 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.199 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.199 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.199 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.200 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.200 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.200 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.200 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.200 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2023-11-27T15:32:31.200 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.200 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.200 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.201 INFO:teuthology.orchestra.run.smithi008.stdout: "fail_fs": { 2023-11-27T15:32:31.201 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "fail_fs", 2023-11-27T15:32:31.201 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.201 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.201 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.201 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.201 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.202 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.202 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.202 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-27T15:32:31.202 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.202 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.202 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.203 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.203 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.203 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.203 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.203 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.203 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.203 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.204 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.204 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.204 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.204 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.204 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.204 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.204 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.205 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.205 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.205 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.205 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.205 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.205 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.205 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.206 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.206 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.206 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.206 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.206 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.206 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.206 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.207 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.207 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.207 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.207 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.207 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.207 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.207 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.208 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.208 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.208 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.208 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.208 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.208 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.208 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.209 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.209 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.209 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.209 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.209 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.209 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.209 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.210 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.210 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.210 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.210 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.210 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.210 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.210 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.211 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.211 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.211 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.211 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.211 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.211 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.211 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.211 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.212 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.212 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.212 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.212 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.212 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.212 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.212 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.213 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator": { 2023-11-27T15:32:31.213 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "orchestrator", 2023-11-27T15:32:31.213 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.213 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.213 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.213 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.213 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.213 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.214 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.214 INFO:teuthology.orchestra.run.smithi008.stdout: "cephadm", 2023-11-27T15:32:31.214 INFO:teuthology.orchestra.run.smithi008.stdout: "rook", 2023-11-27T15:32:31.214 INFO:teuthology.orchestra.run.smithi008.stdout: "test_orchestrator" 2023-11-27T15:32:31.214 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.214 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Orchestrator backend", 2023-11-27T15:32:31.214 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.215 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.215 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.215 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.215 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.215 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.215 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.215 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_perf_query", 2023-11-27T15:32:31.216 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.216 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.216 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.216 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.216 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.216 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.216 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.216 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.217 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.217 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.217 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.217 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.217 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.217 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.217 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.218 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.218 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.218 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.218 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.218 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.218 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.218 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.219 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.219 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.219 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.219 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.219 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.219 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.219 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.219 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.220 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.220 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.220 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.220 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.220 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.220 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.220 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.220 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.221 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.221 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.221 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.221 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.221 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.221 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.221 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.222 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.222 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.222 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.222 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.222 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.222 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.222 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.222 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.223 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.223 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.223 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.223 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.223 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.223 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.223 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.224 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.224 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.224 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.224 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.224 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.224 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.224 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.224 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.225 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.225 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.225 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.225 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.225 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.225 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.225 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.226 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.226 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "osd_support", 2023-11-27T15:32:31.226 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.226 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.226 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.226 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.226 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.227 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.227 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.227 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.227 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.227 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.227 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.227 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.228 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.228 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.228 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.228 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.228 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.228 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.228 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.229 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.229 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.229 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.229 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.229 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.229 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.229 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.230 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.230 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.230 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.230 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.230 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.230 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.231 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.231 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.231 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.231 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.231 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.231 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.231 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.232 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.232 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.232 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.232 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.232 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.232 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.232 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.233 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.233 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.233 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.233 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.233 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.233 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.233 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.234 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.234 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.234 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.234 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.234 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.234 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.235 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.235 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.235 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.235 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.235 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.235 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.235 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.235 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.236 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.236 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.236 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.236 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.236 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.236 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.236 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.237 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.237 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.237 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "pg_autoscaler", 2023-11-27T15:32:31.237 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.237 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.237 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.237 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.238 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.238 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.238 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.238 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.238 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.238 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.238 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.239 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.239 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.239 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.239 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.239 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.239 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.239 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.239 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.240 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.240 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.240 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.240 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.240 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.240 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.240 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.241 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.241 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.241 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.241 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.241 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.241 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.241 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.241 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.242 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.242 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.242 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.242 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.242 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.242 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.242 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.242 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.243 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.243 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.243 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.243 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.243 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.243 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.243 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.244 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.244 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.244 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.244 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.244 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.244 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.244 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.244 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.245 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.245 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.245 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.245 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.245 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.245 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.245 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.246 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.246 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.246 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.246 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.246 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.246 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.246 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.247 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.247 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.247 INFO:teuthology.orchestra.run.smithi008.stdout: "noautoscale": { 2023-11-27T15:32:31.247 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "noautoscale", 2023-11-27T15:32:31.247 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.247 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.247 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.248 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.248 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.248 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.248 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.248 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "global autoscale flag", 2023-11-27T15:32:31.248 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-27T15:32:31.248 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.249 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.249 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.249 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:31.249 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:31.249 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:31.249 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.249 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.250 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:31.250 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.250 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.250 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.250 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.250 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.250 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.250 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.251 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.251 INFO:teuthology.orchestra.run.smithi008.stdout: "threshold": { 2023-11-27T15:32:31.251 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "threshold", 2023-11-27T15:32:31.251 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:31.251 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.251 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.251 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "3.0", 2023-11-27T15:32:31.251 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1.0", 2023-11-27T15:32:31.252 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.252 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.252 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "scaling threshold", 2023-11-27T15:32:31.252 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-27T15:32:31.252 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.252 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.252 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.253 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.253 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.253 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.253 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "progress", 2023-11-27T15:32:31.253 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.253 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.253 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.254 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_pg_recovery_event": { 2023-11-27T15:32:31.254 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_pg_recovery_event", 2023-11-27T15:32:31.254 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.254 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.254 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.254 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.254 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.255 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.255 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.255 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-27T15:32:31.255 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.255 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.255 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.255 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.255 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2023-11-27T15:32:31.256 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2023-11-27T15:32:31.256 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.256 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.256 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.256 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.256 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.256 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.257 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.257 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.257 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.257 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.257 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.257 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.257 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.257 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.258 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.258 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.258 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.258 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.258 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.258 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.259 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.259 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.259 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.259 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.259 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.259 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.259 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.259 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.260 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.260 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.260 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.260 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.260 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.260 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.260 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.261 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.261 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.261 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.261 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.261 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.261 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.261 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.262 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.262 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.262 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.262 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.262 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.262 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.262 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.263 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.263 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.263 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.263 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.263 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.263 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.263 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.263 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.264 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.264 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.264 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.264 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.264 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.264 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.264 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.264 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.265 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.265 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.265 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.265 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.265 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.265 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.265 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.266 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.266 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.266 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.266 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.266 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.266 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.266 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.266 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.267 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.267 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.267 INFO:teuthology.orchestra.run.smithi008.stdout: "max_completed_events": { 2023-11-27T15:32:31.267 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_completed_events", 2023-11-27T15:32:31.267 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.267 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.267 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.267 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "50", 2023-11-27T15:32:31.268 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.268 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.268 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.268 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "number of past completed events to remember", 2023-11-27T15:32:31.268 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.268 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.269 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.269 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.269 INFO:teuthology.orchestra.run.smithi008.stdout: "sleep_interval": { 2023-11-27T15:32:31.269 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "sleep_interval", 2023-11-27T15:32:31.269 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:31.269 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.269 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.270 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "5", 2023-11-27T15:32:31.270 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.270 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.270 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.270 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "how long the module is going to sleep", 2023-11-27T15:32:31.270 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.270 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.271 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.271 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.271 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.271 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.271 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.271 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "prometheus", 2023-11-27T15:32:31.272 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.272 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.272 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.272 INFO:teuthology.orchestra.run.smithi008.stdout: "cache": { 2023-11-27T15:32:31.272 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "cache", 2023-11-27T15:32:31.272 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.272 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.273 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.273 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.273 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.273 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.273 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.273 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.273 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.273 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.274 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.274 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.274 INFO:teuthology.orchestra.run.smithi008.stdout: "exclude_perf_counters": { 2023-11-27T15:32:31.274 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "exclude_perf_counters", 2023-11-27T15:32:31.274 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.274 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.275 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.275 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.275 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.275 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.275 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.275 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-27T15:32:31.275 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-27T15:32:31.275 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.276 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.276 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.276 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.276 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.276 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.276 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.276 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.277 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.277 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.277 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.277 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.277 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.277 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.278 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.278 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.278 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.278 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.278 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.278 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.278 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.279 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.279 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.279 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.279 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.279 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.279 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.279 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.280 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.280 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.280 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.280 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.280 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.280 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.280 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.281 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.281 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.281 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.281 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.281 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.281 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.281 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.282 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.282 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.282 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.282 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.282 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.282 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.282 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.282 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.283 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.283 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.283 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.283 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.283 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.283 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.283 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.284 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.284 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.284 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.284 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.284 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.284 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.284 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.285 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.285 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.285 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.285 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.285 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.285 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.285 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.285 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.286 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.286 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools": { 2023-11-27T15:32:31.286 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools", 2023-11-27T15:32:31.286 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.286 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.286 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.286 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.287 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.287 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.287 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.287 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.287 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.287 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.287 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.288 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.288 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-27T15:32:31.288 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-27T15:32:31.288 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.288 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.288 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.288 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300", 2023-11-27T15:32:31.288 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.289 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.289 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.289 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.289 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.289 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.289 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.289 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.290 INFO:teuthology.orchestra.run.smithi008.stdout: "scrape_interval": { 2023-11-27T15:32:31.290 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "scrape_interval", 2023-11-27T15:32:31.290 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:31.290 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.290 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.290 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15.0", 2023-11-27T15:32:31.290 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.290 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.291 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.291 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.291 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.291 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.291 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.291 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.291 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2023-11-27T15:32:31.292 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2023-11-27T15:32:31.292 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.292 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.292 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.292 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "::", 2023-11-27T15:32:31.292 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.292 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.293 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.293 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-27T15:32:31.293 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.293 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.293 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.293 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.293 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2023-11-27T15:32:31.293 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2023-11-27T15:32:31.294 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.294 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.294 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.294 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "9283", 2023-11-27T15:32:31.294 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.294 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.295 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.295 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-27T15:32:31.295 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.295 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.295 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.295 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.295 INFO:teuthology.orchestra.run.smithi008.stdout: "stale_cache_strategy": { 2023-11-27T15:32:31.296 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stale_cache_strategy", 2023-11-27T15:32:31.296 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.296 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.296 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.296 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "log", 2023-11-27T15:32:31.296 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.296 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.297 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.297 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.297 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.297 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.297 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.297 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.297 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_behaviour": { 2023-11-27T15:32:31.298 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_behaviour", 2023-11-27T15:32:31.298 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.298 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.298 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.298 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "default", 2023-11-27T15:32:31.298 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.298 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.299 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.299 INFO:teuthology.orchestra.run.smithi008.stdout: "default", 2023-11-27T15:32:31.299 INFO:teuthology.orchestra.run.smithi008.stdout: "error" 2023-11-27T15:32:31.299 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.299 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.299 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.299 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.299 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.300 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.300 INFO:teuthology.orchestra.run.smithi008.stdout: "standby_error_status_code": { 2023-11-27T15:32:31.300 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "standby_error_status_code", 2023-11-27T15:32:31.300 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.300 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.300 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.300 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "500", 2023-11-27T15:32:31.301 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "400", 2023-11-27T15:32:31.301 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "599", 2023-11-27T15:32:31.301 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.301 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.301 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.301 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.301 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.301 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.302 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.302 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.302 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.302 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_support", 2023-11-27T15:32:31.302 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.302 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.302 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.303 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.303 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.303 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.303 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.303 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.303 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.303 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.304 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.304 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.304 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.304 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.304 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.304 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.304 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.305 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.305 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.305 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.305 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.305 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.305 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.305 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.305 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.306 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.306 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.306 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.306 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.306 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.306 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.306 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.307 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.307 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.307 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.307 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.307 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.307 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.307 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.308 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.308 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.308 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.308 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.308 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.308 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.308 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.308 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.309 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.309 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.309 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.309 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.309 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.309 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.309 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.310 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.310 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.310 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.310 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.310 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.310 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.310 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.310 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.311 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.311 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.311 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.311 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.311 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.311 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.311 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.312 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.312 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.312 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.312 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.312 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_snap_create": { 2023-11-27T15:32:31.312 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_snap_create", 2023-11-27T15:32:31.312 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.312 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.313 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.313 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10", 2023-11-27T15:32:31.313 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.313 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.313 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.313 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.313 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.314 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.314 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.314 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.314 INFO:teuthology.orchestra.run.smithi008.stdout: "mirror_snapshot_schedule": { 2023-11-27T15:32:31.314 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "mirror_snapshot_schedule", 2023-11-27T15:32:31.314 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.314 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.315 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.315 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.315 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.315 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.315 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.315 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.315 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.315 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.316 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.316 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.316 INFO:teuthology.orchestra.run.smithi008.stdout: "trash_purge_schedule": { 2023-11-27T15:32:31.316 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "trash_purge_schedule", 2023-11-27T15:32:31.316 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.316 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.316 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.316 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.317 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.317 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.317 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.317 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.317 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.317 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.317 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.318 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.318 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.318 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.318 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.318 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "restful", 2023-11-27T15:32:31.318 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.318 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.318 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.319 INFO:teuthology.orchestra.run.smithi008.stdout: "enable_auth": { 2023-11-27T15:32:31.319 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enable_auth", 2023-11-27T15:32:31.319 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.319 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.319 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.319 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.319 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.320 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.321 INFO:teuthology.orchestra.run.smithi008.stdout: "key_file": { 2023-11-27T15:32:31.321 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "key_file", 2023-11-27T15:32:31.321 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.321 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.321 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.321 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.321 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.321 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.322 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.322 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.322 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.322 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.322 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.322 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.322 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.323 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.323 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.323 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.323 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.323 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.323 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.323 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.324 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.324 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.324 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.324 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.324 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.324 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.324 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.325 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.325 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.325 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.325 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.325 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.325 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.325 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.325 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.326 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.326 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.326 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.326 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.326 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.326 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.327 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.327 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.327 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.327 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.327 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.327 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.327 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.328 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.328 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.328 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.328 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.328 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.328 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.328 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.329 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.329 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.329 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.329 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.329 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.329 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.329 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.330 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.330 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.330 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.330 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.330 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.330 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.330 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.330 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.331 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.331 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.331 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.331 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.331 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.331 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.331 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.332 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.332 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.332 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.332 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.332 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.332 INFO:teuthology.orchestra.run.smithi008.stdout: "server_addr": { 2023-11-27T15:32:31.332 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_addr", 2023-11-27T15:32:31.333 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.333 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.333 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.333 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.333 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.333 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.333 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.333 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.334 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.334 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.334 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.334 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.334 INFO:teuthology.orchestra.run.smithi008.stdout: "server_port": { 2023-11-27T15:32:31.334 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "server_port", 2023-11-27T15:32:31.334 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.335 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.335 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.335 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.335 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.335 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.335 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.335 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.335 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.336 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.336 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.336 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.336 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.336 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.336 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.336 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rgw", 2023-11-27T15:32:31.337 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.337 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.337 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.337 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.337 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.337 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.337 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.338 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.338 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.338 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.338 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.338 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.338 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.338 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.339 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.339 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.339 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.339 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.339 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.339 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.339 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.340 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.340 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.340 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.340 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.340 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.340 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.340 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.340 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.341 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.341 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.341 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.341 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.341 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.341 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.341 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.341 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.342 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.342 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.342 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.342 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.342 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.342 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.342 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.343 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.343 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.343 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.343 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.343 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.343 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.343 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.343 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.344 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.344 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.344 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.344 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.344 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.344 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.344 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.345 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.345 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.345 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.345 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.345 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.345 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.345 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.345 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.346 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.346 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.346 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.346 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.346 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.346 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.347 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.347 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.347 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.347 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rook", 2023-11-27T15:32:31.347 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.347 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.347 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.347 INFO:teuthology.orchestra.run.smithi008.stdout: "drive_group_interval": { 2023-11-27T15:32:31.348 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "drive_group_interval", 2023-11-27T15:32:31.348 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:31.348 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.348 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.348 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "300.0", 2023-11-27T15:32:31.348 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.348 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.349 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.349 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-27T15:32:31.349 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.349 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.349 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.349 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.349 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.349 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.350 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.350 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.350 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.350 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.350 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.350 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.350 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.351 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.351 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.351 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.351 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.351 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.351 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.351 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.351 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.352 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.352 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.352 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.352 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.352 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.352 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.352 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.353 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.353 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.353 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.353 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.353 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.353 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.353 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.353 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.354 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.354 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.354 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.354 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.354 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.354 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.354 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.357 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.357 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.357 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.357 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.357 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.357 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.357 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.358 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.358 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.358 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.358 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.358 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.358 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.358 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.359 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.359 INFO:teuthology.orchestra.run.smithi008.stdout: "storage_class": { 2023-11-27T15:32:31.359 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "storage_class", 2023-11-27T15:32:31.359 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.359 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.359 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.359 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "local", 2023-11-27T15:32:31.359 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.360 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.360 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.360 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-27T15:32:31.360 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.360 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.360 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.360 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.360 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.361 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.361 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.361 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "selftest", 2023-11-27T15:32:31.361 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.361 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.361 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.361 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.362 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.362 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.362 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.362 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.362 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.362 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.362 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.363 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.363 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.363 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.363 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.363 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.363 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.363 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.364 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.364 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.364 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.364 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.364 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.364 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.365 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.365 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.365 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.365 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.365 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.365 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.365 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.366 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.366 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.366 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.366 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.366 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.366 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.367 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.367 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.367 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.367 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.367 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.367 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.367 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.368 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.368 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.368 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.368 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.368 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.368 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.368 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.369 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.369 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.369 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.369 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.369 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.369 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.369 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.370 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.370 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.370 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.370 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.370 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.370 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.371 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.371 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.371 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.371 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.371 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.371 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.371 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.372 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.372 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.372 INFO:teuthology.orchestra.run.smithi008.stdout: "roption1": { 2023-11-27T15:32:31.372 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption1", 2023-11-27T15:32:31.372 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.372 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.372 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.372 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.373 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.373 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.373 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.373 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.373 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.373 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.373 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.373 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.374 INFO:teuthology.orchestra.run.smithi008.stdout: "roption2": { 2023-11-27T15:32:31.374 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "roption2", 2023-11-27T15:32:31.374 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.374 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.374 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.374 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "xyz", 2023-11-27T15:32:31.374 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.375 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.375 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.375 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.375 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.375 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.375 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.375 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.376 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption1": { 2023-11-27T15:32:31.376 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption1", 2023-11-27T15:32:31.376 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.376 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.376 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.376 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.376 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.376 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.377 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.377 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.377 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.377 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.377 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.378 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.378 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption2": { 2023-11-27T15:32:31.378 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption2", 2023-11-27T15:32:31.378 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.379 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.379 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.379 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.379 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.379 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.380 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.380 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.380 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.380 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.380 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.381 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.381 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption3": { 2023-11-27T15:32:31.381 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption3", 2023-11-27T15:32:31.381 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "float", 2023-11-27T15:32:31.381 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.382 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.382 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.382 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.382 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.382 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.383 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.383 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.383 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.383 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.383 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.384 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption4": { 2023-11-27T15:32:31.384 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption4", 2023-11-27T15:32:31.384 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.384 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.384 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.385 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.385 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.385 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.385 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.385 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.385 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.386 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.386 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.386 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.386 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption5": { 2023-11-27T15:32:31.386 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption5", 2023-11-27T15:32:31.386 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.386 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.387 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.387 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.387 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.387 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.387 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.387 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.387 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.388 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.388 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.388 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.388 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption6": { 2023-11-27T15:32:31.388 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption6", 2023-11-27T15:32:31.388 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.388 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.389 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.389 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.389 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.389 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.389 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.389 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.390 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.390 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.390 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.390 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.390 INFO:teuthology.orchestra.run.smithi008.stdout: "rwoption7": { 2023-11-27T15:32:31.390 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rwoption7", 2023-11-27T15:32:31.390 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.391 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.391 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.391 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.391 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "1", 2023-11-27T15:32:31.391 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "42", 2023-11-27T15:32:31.391 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.391 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.391 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.392 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.392 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.392 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.392 INFO:teuthology.orchestra.run.smithi008.stdout: "testkey": { 2023-11-27T15:32:31.392 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testkey", 2023-11-27T15:32:31.392 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.392 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.393 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.393 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.393 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.393 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.393 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.393 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.393 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.393 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.394 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.394 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.394 INFO:teuthology.orchestra.run.smithi008.stdout: "testlkey": { 2023-11-27T15:32:31.394 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testlkey", 2023-11-27T15:32:31.394 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.394 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.394 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.395 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.395 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.395 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.395 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.395 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.395 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.395 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.396 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.396 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.396 INFO:teuthology.orchestra.run.smithi008.stdout: "testnewline": { 2023-11-27T15:32:31.396 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "testnewline", 2023-11-27T15:32:31.396 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.396 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.396 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.396 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.397 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.397 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.397 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.397 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.397 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.397 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.397 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.397 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.398 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.398 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.398 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.398 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snap_schedule", 2023-11-27T15:32:31.398 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.398 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.399 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.399 INFO:teuthology.orchestra.run.smithi008.stdout: "allow_m_granularity": { 2023-11-27T15:32:31.399 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "allow_m_granularity", 2023-11-27T15:32:31.399 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.399 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.399 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.399 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.400 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.400 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.400 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.400 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "allow minute scheduled snapshots", 2023-11-27T15:32:31.400 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.400 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.400 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.401 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.401 INFO:teuthology.orchestra.run.smithi008.stdout: "dump_on_update": { 2023-11-27T15:32:31.401 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "dump_on_update", 2023-11-27T15:32:31.401 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.401 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.401 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.401 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.402 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.402 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.402 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.402 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "dump database to debug log on update", 2023-11-27T15:32:31.402 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.402 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.402 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.403 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.403 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.403 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.403 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.403 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.403 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.403 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.404 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.404 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.404 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.404 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.404 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.404 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.404 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.405 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.405 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.405 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.405 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.405 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.405 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.405 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.406 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.406 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.406 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.406 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.406 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.406 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.406 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.407 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.407 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.407 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.407 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.407 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.409 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.409 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.409 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.409 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.409 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.410 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.410 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.410 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.410 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.410 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.410 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.411 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.411 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.411 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.411 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.411 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.411 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.411 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.412 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.412 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.412 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.412 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.412 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.412 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.412 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.412 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.413 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.413 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.413 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.413 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.413 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.413 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.413 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.414 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.414 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.414 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.414 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.414 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.414 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.414 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.415 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.415 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "stats", 2023-11-27T15:32:31.415 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.415 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.415 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.415 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.415 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.416 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.416 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.416 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.416 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.416 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.416 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.416 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.417 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.417 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.417 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.417 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.417 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.417 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.417 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.417 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.418 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.418 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.418 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.418 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.418 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.418 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.419 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.419 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.419 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.419 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.419 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.419 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.419 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.420 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.420 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.420 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.420 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.420 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.420 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.420 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.420 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.421 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.421 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.421 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.421 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.421 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.421 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.421 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.422 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.422 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.422 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.422 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.422 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.422 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.422 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.423 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.423 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.423 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.423 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.423 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.423 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.423 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.423 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.424 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.424 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.424 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.424 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.424 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.424 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.424 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.424 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.425 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.425 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.425 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.425 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.425 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.425 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "status", 2023-11-27T15:32:31.425 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.426 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.426 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.426 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.426 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.426 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.426 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.426 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.427 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.427 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.427 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.427 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.427 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.427 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.427 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.428 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.428 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.428 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.428 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.428 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.428 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.428 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.429 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.429 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.429 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.429 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.429 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.429 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.429 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.429 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.430 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.430 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.430 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.430 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.430 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.430 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.431 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.431 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.431 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.431 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.431 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.431 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.431 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.431 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.432 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.432 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.432 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.432 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.432 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.432 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.433 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.433 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.433 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.433 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.433 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.433 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.433 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.433 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.434 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.434 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.434 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.434 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.434 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.434 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.434 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.435 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.435 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.435 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.435 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.435 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.435 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.435 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.436 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.436 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.436 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.436 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.436 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telegraf", 2023-11-27T15:32:31.436 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.436 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.436 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.437 INFO:teuthology.orchestra.run.smithi008.stdout: "address": { 2023-11-27T15:32:31.437 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "address", 2023-11-27T15:32:31.437 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.437 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.437 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.437 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-27T15:32:31.437 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.438 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.438 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.438 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.438 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.438 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.438 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.438 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.439 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:31.439 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:31.439 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:31.439 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.439 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.439 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "15", 2023-11-27T15:32:31.439 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.440 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.440 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.440 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.440 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.440 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.440 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.440 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.441 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.441 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.441 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.441 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.441 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.441 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.441 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.441 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.442 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.442 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.442 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.442 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.442 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.442 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.442 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.443 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.443 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.443 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.443 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.443 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.443 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.443 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.443 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.444 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.444 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.444 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.444 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.444 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.444 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.444 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.445 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.445 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.445 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.445 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.445 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.445 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.445 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.446 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.446 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.446 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.446 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.446 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.446 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.446 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.446 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.447 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.447 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.447 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.447 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.447 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.447 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.447 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.448 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.448 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.448 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.448 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.448 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.448 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.448 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.449 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.449 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.449 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.449 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.449 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.449 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.449 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.449 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.450 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.450 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.450 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.450 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.450 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.450 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.450 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "telemetry", 2023-11-27T15:32:31.451 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.451 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.451 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.451 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_basic": { 2023-11-27T15:32:31.451 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_basic", 2023-11-27T15:32:31.451 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.451 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.452 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.452 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.452 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.452 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.452 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.452 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-27T15:32:31.452 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.452 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.453 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.453 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.453 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_crash": { 2023-11-27T15:32:31.453 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_crash", 2023-11-27T15:32:31.453 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.453 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.453 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.453 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.454 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.454 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.454 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.454 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-27T15:32:31.454 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.454 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.454 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.455 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.455 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_device": { 2023-11-27T15:32:31.455 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_device", 2023-11-27T15:32:31.455 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.455 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.455 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.455 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "True", 2023-11-27T15:32:31.456 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.456 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.456 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.456 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-27T15:32:31.456 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.456 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.456 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.456 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.457 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_ident": { 2023-11-27T15:32:31.457 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_ident", 2023-11-27T15:32:31.457 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.457 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.457 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.457 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.458 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.458 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.458 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.458 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-27T15:32:31.458 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.458 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.459 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.459 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.459 INFO:teuthology.orchestra.run.smithi008.stdout: "channel_perf": { 2023-11-27T15:32:31.459 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "channel_perf", 2023-11-27T15:32:31.459 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.459 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.459 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.460 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.460 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.460 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.460 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.460 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-27T15:32:31.460 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.461 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.461 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.461 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.461 INFO:teuthology.orchestra.run.smithi008.stdout: "contact": { 2023-11-27T15:32:31.461 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "contact", 2023-11-27T15:32:31.461 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.461 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.461 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.462 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.462 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.462 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.462 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.462 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.462 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.463 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.463 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.463 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.463 INFO:teuthology.orchestra.run.smithi008.stdout: "description": { 2023-11-27T15:32:31.463 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "description", 2023-11-27T15:32:31.463 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.463 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.463 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.464 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.464 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.464 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.464 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.464 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.464 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.464 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.464 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.465 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.465 INFO:teuthology.orchestra.run.smithi008.stdout: "device_url": { 2023-11-27T15:32:31.465 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "device_url", 2023-11-27T15:32:31.465 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.465 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.465 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.465 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-27T15:32:31.466 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.466 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.466 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.466 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.466 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.466 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.466 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.467 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.467 INFO:teuthology.orchestra.run.smithi008.stdout: "enabled": { 2023-11-27T15:32:31.467 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "enabled", 2023-11-27T15:32:31.467 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.467 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.467 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.467 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.467 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.468 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.468 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.468 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.468 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.468 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.468 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.468 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.469 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:31.469 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:31.469 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.469 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.469 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.469 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "24", 2023-11-27T15:32:31.469 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "8", 2023-11-27T15:32:31.470 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.470 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.470 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.470 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.470 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.470 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.470 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.470 INFO:teuthology.orchestra.run.smithi008.stdout: "last_opt_revision": { 2023-11-27T15:32:31.471 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "last_opt_revision", 2023-11-27T15:32:31.471 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.471 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.471 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.471 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "1", 2023-11-27T15:32:31.471 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.471 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.472 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.472 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.472 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.472 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.472 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.472 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.472 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard": { 2023-11-27T15:32:31.473 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard", 2023-11-27T15:32:31.473 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.473 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.473 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.473 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.473 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.473 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.473 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.474 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.474 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.474 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.474 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.474 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.474 INFO:teuthology.orchestra.run.smithi008.stdout: "leaderboard_description": { 2023-11-27T15:32:31.474 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "leaderboard_description", 2023-11-27T15:32:31.475 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.475 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.475 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.475 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.475 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.475 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.475 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.475 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.476 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.476 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.476 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.476 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.476 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.476 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.476 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.477 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.477 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.477 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.477 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.477 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.477 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.477 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.477 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.478 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.478 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.478 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.478 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.478 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.478 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.479 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.479 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.479 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.479 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.479 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.479 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.479 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.479 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.480 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.480 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.480 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.480 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.480 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.480 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.480 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.481 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.481 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.481 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.481 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.481 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.481 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.481 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.482 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.482 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.482 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.482 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.482 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.482 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.482 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.483 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.483 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.483 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.483 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.483 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.483 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.484 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.484 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.484 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.484 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.484 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.484 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.484 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.484 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.485 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.485 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.485 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.485 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.485 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.485 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.485 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.485 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.486 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.486 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.486 INFO:teuthology.orchestra.run.smithi008.stdout: "organization": { 2023-11-27T15:32:31.486 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "organization", 2023-11-27T15:32:31.486 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.486 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.486 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.487 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.487 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.487 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.487 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.487 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.487 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.487 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.487 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.488 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.488 INFO:teuthology.orchestra.run.smithi008.stdout: "proxy": { 2023-11-27T15:32:31.488 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "proxy", 2023-11-27T15:32:31.488 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.488 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.488 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.488 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.489 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.489 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.489 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.489 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.489 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.489 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.489 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.490 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.490 INFO:teuthology.orchestra.run.smithi008.stdout: "url": { 2023-11-27T15:32:31.490 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "url", 2023-11-27T15:32:31.490 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.490 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.490 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.490 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-27T15:32:31.491 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.491 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.491 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.491 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.491 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.491 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.491 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.492 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.492 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.492 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.492 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.492 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "test_orchestrator", 2023-11-27T15:32:31.492 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.492 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.493 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.493 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.493 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.493 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.493 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.493 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.493 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.494 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.494 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.494 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.494 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.494 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.494 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.494 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.495 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.495 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.495 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.495 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.495 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.495 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.495 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.496 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.496 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.496 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.496 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.496 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.496 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.496 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.497 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.497 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.497 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.497 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.497 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.497 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.497 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.498 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.498 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.498 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.498 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.498 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.498 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.499 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.499 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.499 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.499 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.499 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.499 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.499 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.499 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.500 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.500 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.500 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.500 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.500 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.500 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.500 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.501 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.501 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.501 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.501 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.501 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.501 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.501 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.502 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.502 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.502 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.502 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.502 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.502 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.502 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.503 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.503 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.503 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.503 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.503 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "volumes", 2023-11-27T15:32:31.503 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.503 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.504 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.504 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.504 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.504 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.504 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.504 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.504 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.504 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.505 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.505 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.505 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.505 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.505 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.505 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.505 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.506 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.506 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.506 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.506 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.506 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.506 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.506 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.507 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.507 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.507 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.507 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.507 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.507 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.507 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.508 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.508 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.508 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.508 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.508 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.508 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.508 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.509 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.509 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.509 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.509 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.509 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.509 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.509 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.510 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.510 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.510 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.510 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.510 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.510 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.510 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.510 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.511 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.511 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.511 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.511 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.511 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.511 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.511 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.512 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.512 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.512 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.512 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.512 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.512 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.512 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.513 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.513 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.513 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.513 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.513 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.513 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.513 INFO:teuthology.orchestra.run.smithi008.stdout: "max_concurrent_clones": { 2023-11-27T15:32:31.513 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "max_concurrent_clones", 2023-11-27T15:32:31.514 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.514 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.514 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.514 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "4", 2023-11-27T15:32:31.514 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.514 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.514 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.515 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-27T15:32:31.515 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.515 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.515 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.515 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.515 INFO:teuthology.orchestra.run.smithi008.stdout: "snapshot_clone_delay": { 2023-11-27T15:32:31.515 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "snapshot_clone_delay", 2023-11-27T15:32:31.515 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.516 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.516 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.516 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "0", 2023-11-27T15:32:31.516 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.516 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.516 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.516 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-27T15:32:31.517 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.517 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.517 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.517 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.517 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.517 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.517 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.517 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix", 2023-11-27T15:32:31.518 INFO:teuthology.orchestra.run.smithi008.stdout: "can_run": true, 2023-11-27T15:32:31.518 INFO:teuthology.orchestra.run.smithi008.stdout: "error_string": "", 2023-11-27T15:32:31.518 INFO:teuthology.orchestra.run.smithi008.stdout: "module_options": { 2023-11-27T15:32:31.518 INFO:teuthology.orchestra.run.smithi008.stdout: "discovery_interval": { 2023-11-27T15:32:31.518 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "discovery_interval", 2023-11-27T15:32:31.518 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "uint", 2023-11-27T15:32:31.518 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.519 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.519 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "100", 2023-11-27T15:32:31.519 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.519 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.519 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.519 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.519 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.519 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.520 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.520 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.520 INFO:teuthology.orchestra.run.smithi008.stdout: "identifier": { 2023-11-27T15:32:31.520 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "identifier", 2023-11-27T15:32:31.520 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.520 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.520 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.521 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.521 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.521 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.521 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.521 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.521 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.521 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.521 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.522 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.522 INFO:teuthology.orchestra.run.smithi008.stdout: "interval": { 2023-11-27T15:32:31.522 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "interval", 2023-11-27T15:32:31.522 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "secs", 2023-11-27T15:32:31.522 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.522 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.522 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "60", 2023-11-27T15:32:31.523 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.523 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.523 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.523 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.523 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.523 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.523 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.523 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.524 INFO:teuthology.orchestra.run.smithi008.stdout: "log_level": { 2023-11-27T15:32:31.524 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_level", 2023-11-27T15:32:31.524 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.524 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.524 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.524 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.524 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.524 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.525 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.525 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.525 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.525 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.525 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.525 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.525 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.526 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.526 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.526 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.526 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.526 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.526 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.526 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster": { 2023-11-27T15:32:31.527 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster", 2023-11-27T15:32:31.527 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.527 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.527 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.527 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.527 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.527 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.527 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.528 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.528 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.528 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.528 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.528 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.528 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_cluster_level": { 2023-11-27T15:32:31.528 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_cluster_level", 2023-11-27T15:32:31.528 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.529 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.529 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.529 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "info", 2023-11-27T15:32:31.529 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.529 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.529 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [ 2023-11-27T15:32:31.529 INFO:teuthology.orchestra.run.smithi008.stdout: "", 2023-11-27T15:32:31.530 INFO:teuthology.orchestra.run.smithi008.stdout: "critical", 2023-11-27T15:32:31.530 INFO:teuthology.orchestra.run.smithi008.stdout: "debug", 2023-11-27T15:32:31.530 INFO:teuthology.orchestra.run.smithi008.stdout: "error", 2023-11-27T15:32:31.530 INFO:teuthology.orchestra.run.smithi008.stdout: "info", 2023-11-27T15:32:31.530 INFO:teuthology.orchestra.run.smithi008.stdout: "warning" 2023-11-27T15:32:31.530 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.530 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.531 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.531 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.531 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.531 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.531 INFO:teuthology.orchestra.run.smithi008.stdout: "log_to_file": { 2023-11-27T15:32:31.531 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "log_to_file", 2023-11-27T15:32:31.531 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "bool", 2023-11-27T15:32:31.532 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.532 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 1, 2023-11-27T15:32:31.532 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "False", 2023-11-27T15:32:31.532 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.532 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.532 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.533 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.533 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.533 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.533 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.533 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.533 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_host": { 2023-11-27T15:32:31.533 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_host", 2023-11-27T15:32:31.533 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.534 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.534 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.534 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "", 2023-11-27T15:32:31.534 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.534 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.534 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.534 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.535 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.535 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.535 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.535 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.535 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_port": { 2023-11-27T15:32:31.535 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_port", 2023-11-27T15:32:31.535 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "int", 2023-11-27T15:32:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "10051", 2023-11-27T15:32:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.536 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.537 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.537 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.537 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.537 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.537 INFO:teuthology.orchestra.run.smithi008.stdout: "zabbix_sender": { 2023-11-27T15:32:31.537 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "zabbix_sender", 2023-11-27T15:32:31.538 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "str", 2023-11-27T15:32:31.538 INFO:teuthology.orchestra.run.smithi008.stdout: "level": "advanced", 2023-11-27T15:32:31.538 INFO:teuthology.orchestra.run.smithi008.stdout: "flags": 0, 2023-11-27T15:32:31.538 INFO:teuthology.orchestra.run.smithi008.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-27T15:32:31.539 INFO:teuthology.orchestra.run.smithi008.stdout: "min": "", 2023-11-27T15:32:31.539 INFO:teuthology.orchestra.run.smithi008.stdout: "max": "", 2023-11-27T15:32:31.539 INFO:teuthology.orchestra.run.smithi008.stdout: "enum_allowed": [], 2023-11-27T15:32:31.539 INFO:teuthology.orchestra.run.smithi008.stdout: "desc": "", 2023-11-27T15:32:31.540 INFO:teuthology.orchestra.run.smithi008.stdout: "long_desc": "", 2023-11-27T15:32:31.540 INFO:teuthology.orchestra.run.smithi008.stdout: "tags": [], 2023-11-27T15:32:31.540 INFO:teuthology.orchestra.run.smithi008.stdout: "see_also": [] 2023-11-27T15:32:31.540 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.540 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.541 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.541 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.541 INFO:teuthology.orchestra.run.smithi008.stdout: "services": { 2023-11-27T15:32:31.541 INFO:teuthology.orchestra.run.smithi008.stdout: "dashboard": "https://172.21.15.8:8443/" 2023-11-27T15:32:31.541 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.541 INFO:teuthology.orchestra.run.smithi008.stdout: "always_on_modules": { 2023-11-27T15:32:31.542 INFO:teuthology.orchestra.run.smithi008.stdout: "octopus": [ 2023-11-27T15:32:31.542 INFO:teuthology.orchestra.run.smithi008.stdout: "balancer", 2023-11-27T15:32:31.542 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2023-11-27T15:32:31.542 INFO:teuthology.orchestra.run.smithi008.stdout: "devicehealth", 2023-11-27T15:32:31.542 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator", 2023-11-27T15:32:31.542 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_autoscaler", 2023-11-27T15:32:31.543 INFO:teuthology.orchestra.run.smithi008.stdout: "progress", 2023-11-27T15:32:31.543 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_support", 2023-11-27T15:32:31.543 INFO:teuthology.orchestra.run.smithi008.stdout: "status", 2023-11-27T15:32:31.543 INFO:teuthology.orchestra.run.smithi008.stdout: "telemetry", 2023-11-27T15:32:31.543 INFO:teuthology.orchestra.run.smithi008.stdout: "volumes" 2023-11-27T15:32:31.543 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.543 INFO:teuthology.orchestra.run.smithi008.stdout: "pacific": [ 2023-11-27T15:32:31.544 INFO:teuthology.orchestra.run.smithi008.stdout: "balancer", 2023-11-27T15:32:31.544 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2023-11-27T15:32:31.544 INFO:teuthology.orchestra.run.smithi008.stdout: "devicehealth", 2023-11-27T15:32:31.544 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator", 2023-11-27T15:32:31.544 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_autoscaler", 2023-11-27T15:32:31.544 INFO:teuthology.orchestra.run.smithi008.stdout: "progress", 2023-11-27T15:32:31.544 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_support", 2023-11-27T15:32:31.545 INFO:teuthology.orchestra.run.smithi008.stdout: "status", 2023-11-27T15:32:31.545 INFO:teuthology.orchestra.run.smithi008.stdout: "telemetry", 2023-11-27T15:32:31.545 INFO:teuthology.orchestra.run.smithi008.stdout: "volumes" 2023-11-27T15:32:31.545 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.545 INFO:teuthology.orchestra.run.smithi008.stdout: "quincy": [ 2023-11-27T15:32:31.545 INFO:teuthology.orchestra.run.smithi008.stdout: "balancer", 2023-11-27T15:32:31.546 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2023-11-27T15:32:31.546 INFO:teuthology.orchestra.run.smithi008.stdout: "devicehealth", 2023-11-27T15:32:31.546 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator", 2023-11-27T15:32:31.546 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_autoscaler", 2023-11-27T15:32:31.546 INFO:teuthology.orchestra.run.smithi008.stdout: "progress", 2023-11-27T15:32:31.546 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_support", 2023-11-27T15:32:31.546 INFO:teuthology.orchestra.run.smithi008.stdout: "status", 2023-11-27T15:32:31.547 INFO:teuthology.orchestra.run.smithi008.stdout: "telemetry", 2023-11-27T15:32:31.547 INFO:teuthology.orchestra.run.smithi008.stdout: "volumes" 2023-11-27T15:32:31.547 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2023-11-27T15:32:31.547 INFO:teuthology.orchestra.run.smithi008.stdout: "reef": [ 2023-11-27T15:32:31.547 INFO:teuthology.orchestra.run.smithi008.stdout: "balancer", 2023-11-27T15:32:31.547 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2023-11-27T15:32:31.548 INFO:teuthology.orchestra.run.smithi008.stdout: "devicehealth", 2023-11-27T15:32:31.548 INFO:teuthology.orchestra.run.smithi008.stdout: "orchestrator", 2023-11-27T15:32:31.548 INFO:teuthology.orchestra.run.smithi008.stdout: "pg_autoscaler", 2023-11-27T15:32:31.548 INFO:teuthology.orchestra.run.smithi008.stdout: "progress", 2023-11-27T15:32:31.548 INFO:teuthology.orchestra.run.smithi008.stdout: "rbd_support", 2023-11-27T15:32:31.548 INFO:teuthology.orchestra.run.smithi008.stdout: "status", 2023-11-27T15:32:31.548 INFO:teuthology.orchestra.run.smithi008.stdout: "telemetry", 2023-11-27T15:32:31.549 INFO:teuthology.orchestra.run.smithi008.stdout: "volumes" 2023-11-27T15:32:31.549 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2023-11-27T15:32:31.549 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.549 INFO:teuthology.orchestra.run.smithi008.stdout: "last_failure_osd_epoch": 3, 2023-11-27T15:32:31.549 INFO:teuthology.orchestra.run.smithi008.stdout: "active_clients": [ 2023-11-27T15:32:31.549 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.550 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "devicehealth", 2023-11-27T15:32:31.550 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2023-11-27T15:32:31.550 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.550 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2023-11-27T15:32:31.550 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:0", 2023-11-27T15:32:31.550 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 2984393656 2023-11-27T15:32:31.551 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.551 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2023-11-27T15:32:31.551 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.551 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.551 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "libcephsqlite", 2023-11-27T15:32:31.551 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2023-11-27T15:32:31.551 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.552 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2023-11-27T15:32:31.552 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:0", 2023-11-27T15:32:31.552 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 3264476100 2023-11-27T15:32:31.552 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.552 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2023-11-27T15:32:31.552 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.553 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.553 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "rbd_support", 2023-11-27T15:32:31.553 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2023-11-27T15:32:31.553 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.553 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2023-11-27T15:32:31.553 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:0", 2023-11-27T15:32:31.553 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 3796022999 2023-11-27T15:32:31.554 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.554 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2023-11-27T15:32:31.554 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:31.554 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.554 INFO:teuthology.orchestra.run.smithi008.stdout: "name": "volumes", 2023-11-27T15:32:31.554 INFO:teuthology.orchestra.run.smithi008.stdout: "addrvec": [ 2023-11-27T15:32:31.555 INFO:teuthology.orchestra.run.smithi008.stdout: { 2023-11-27T15:32:31.555 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v2", 2023-11-27T15:32:31.555 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "172.21.15.8:0", 2023-11-27T15:32:31.555 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 3193049984 2023-11-27T15:32:31.555 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.555 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2023-11-27T15:32:31.555 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:31.556 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2023-11-27T15:32:31.556 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:32:31.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3557587010 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc7000a11d0 msgr2=0x7fc7000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:31.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc7000a11d0 0x7fc7000a3680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc714101320 tx=0x7fc710009260 comp rx=0 tx=0).stop 2023-11-27T15:32:31.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3557587010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc714101e50 msgr2=0x7fc714073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:32:31.558 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc714101e50 0x7fc714073420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00d010 tx=0x7fc70c00bc80 comp rx=0 tx=0).stop 2023-11-27T15:32:31.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3557587010 shutdown_connections 2023-11-27T15:32:31.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fc7000a11d0 0x7fc7000a3680 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:31.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fc7141002d0 0x7fc7140748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:31.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc714101e50 0x7fc714073420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:31.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 --2- 172.21.15.8:0/3557587010 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fc7141014c0 0x7fc714072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:32:31.559 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3557587010 >> 172.21.15.8:0/3557587010 conn(0x7fc7140fb8b0 msgr2=0x7fc714104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:32:31.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3557587010 shutdown_connections 2023-11-27T15:32:31.560 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:32:20.561+0000 7fc71c7ff700 1 -- 172.21.15.8:0/3557587010 wait complete. 2023-11-27T15:32:31.565 DEBUG:tasks.check_counter:Getting stats from a 2023-11-27T15:32:31.566 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.a perf dump 2023-11-27T15:32:31.567 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:20 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/192412563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:32:31.567 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:20 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3849108104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:32:31.568 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:20 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3557587010' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-27T15:32:31.568 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:21 smithi029 ceph-mon[131166]: pgmap v555: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2023-11-27T15:32:31.568 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:23 smithi029 ceph-mon[131166]: pgmap v556: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2023-11-27T15:32:31.568 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:25 smithi029 ceph-mon[131166]: pgmap v557: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2023-11-27T15:32:31.568 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:27 smithi029 ceph-mon[131166]: pgmap v558: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2023-11-27T15:32:31.568 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:29 smithi029 ceph-mon[131166]: pgmap v559: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2023-11-27T15:32:31.569 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:20 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/192412563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:32:31.569 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:20 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3849108104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:32:31.569 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:20 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3557587010' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-27T15:32:31.569 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:21 smithi060 ceph-mon[131518]: pgmap v555: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2023-11-27T15:32:31.569 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:23 smithi060 ceph-mon[131518]: pgmap v556: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2023-11-27T15:32:31.569 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:25 smithi060 ceph-mon[131518]: pgmap v557: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2023-11-27T15:32:31.570 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:27 smithi060 ceph-mon[131518]: pgmap v558: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2023-11-27T15:32:31.570 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:29 smithi060 ceph-mon[131518]: pgmap v559: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2023-11-27T15:32:31.570 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:20 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/192412563' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:32:31.570 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:20 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3849108104' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:32:31.571 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:20 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3557587010' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-27T15:32:31.571 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:21 smithi008 ceph-mon[128788]: pgmap v555: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8.0 KiB/s wr, 13 op/s 2023-11-27T15:32:31.571 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:23 smithi008 ceph-mon[128788]: pgmap v556: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2023-11-27T15:32:31.571 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:25 smithi008 ceph-mon[128788]: pgmap v557: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2023-11-27T15:32:31.571 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:27 smithi008 ceph-mon[128788]: pgmap v558: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2023-11-27T15:32:31.571 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:29 smithi008 ceph-mon[128788]: pgmap v559: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.6 KiB/s wr, 7 op/s 2023-11-27T15:32:31.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:31 smithi060 ceph-mon[131518]: pgmap v560: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-11-27T15:32:31.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:31 smithi008 ceph-mon[128788]: pgmap v560: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-11-27T15:32:31.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:31 smithi029 ceph-mon[131166]: pgmap v560: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-11-27T15:32:32.453 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:32:33.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:33 smithi060 ceph-mon[131518]: pgmap v561: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-11-27T15:32:33.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:33 smithi008 ceph-mon[128788]: pgmap v561: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-11-27T15:32:33.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:33 smithi029 ceph-mon[131166]: pgmap v561: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-11-27T15:32:35.395 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:32:35.395 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:32:35.395 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 94207, 2023-11-27T15:32:35.396 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 167080, 2023-11-27T15:32:35.396 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 37728372, 2023-11-27T15:32:35.396 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 174901023, 2023-11-27T15:32:35.396 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 66, 2023-11-27T15:32:35.396 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 8, 2023-11-27T15:32:35.396 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 22.959888050, 2023-11-27T15:32:35.397 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 14.689756032, 2023-11-27T15:32:35.397 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 49.264221288, 2023-11-27T15:32:35.397 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 1.679695994, 2023-11-27T15:32:35.397 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:35.397 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 167076, 2023-11-27T15:32:35.398 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 32.700220567, 2023-11-27T15:32:35.398 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000195720 2023-11-27T15:32:35.398 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.398 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:35.398 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 190663, 2023-11-27T15:32:35.399 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.064292687, 2023-11-27T15:32:35.399 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000337 2023-11-27T15:32:35.399 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.399 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 28060816, 2023-11-27T15:32:35.399 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 86198048 2023-11-27T15:32:35.399 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.400 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:32:35.400 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 29027, 2023-11-27T15:32:35.400 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 30132, 2023-11-27T15:32:35.400 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 42122284, 2023-11-27T15:32:35.400 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 65660855, 2023-11-27T15:32:35.400 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 8, 2023-11-27T15:32:35.401 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 5, 2023-11-27T15:32:35.401 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 4.020735829, 2023-11-27T15:32:35.401 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 1.472650435, 2023-11-27T15:32:35.401 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.844534419, 2023-11-27T15:32:35.401 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 1.522705699, 2023-11-27T15:32:35.401 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:35.402 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 30132, 2023-11-27T15:32:35.402 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 21.469806873, 2023-11-27T15:32:35.402 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000712525 2023-11-27T15:32:35.402 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.402 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:35.403 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 10019, 2023-11-27T15:32:35.403 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.003698807, 2023-11-27T15:32:35.403 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000369 2023-11-27T15:32:35.403 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.403 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:32:35.403 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 279040 2023-11-27T15:32:35.404 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.404 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:32:35.404 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 42152, 2023-11-27T15:32:35.404 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 36962, 2023-11-27T15:32:35.404 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 23648151, 2023-11-27T15:32:35.405 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 79156464, 2023-11-27T15:32:35.405 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 12, 2023-11-27T15:32:35.405 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 6, 2023-11-27T15:32:35.405 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 4.361695055, 2023-11-27T15:32:35.405 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 1.558561908, 2023-11-27T15:32:35.406 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 1.462119271, 2023-11-27T15:32:35.406 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 1.301183154, 2023-11-27T15:32:35.406 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:35.406 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 36962, 2023-11-27T15:32:35.406 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 24.482284655, 2023-11-27T15:32:35.407 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000662363 2023-11-27T15:32:35.407 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.407 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:35.407 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 27129, 2023-11-27T15:32:35.407 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.005824067, 2023-11-27T15:32:35.408 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000214 2023-11-27T15:32:35.408 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.408 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 547328, 2023-11-27T15:32:35.408 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 94736 2023-11-27T15:32:35.408 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.408 INFO:teuthology.orchestra.run.smithi008.stdout: "cct": { 2023-11-27T15:32:35.409 INFO:teuthology.orchestra.run.smithi008.stdout: "total_workers": 1, 2023-11-27T15:32:35.409 INFO:teuthology.orchestra.run.smithi008.stdout: "unhealthy_workers": 0 2023-11-27T15:32:35.409 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.409 INFO:teuthology.orchestra.run.smithi008.stdout: "finisher-MDSRank": { 2023-11-27T15:32:35.409 INFO:teuthology.orchestra.run.smithi008.stdout: "queue_len": 0, 2023-11-27T15:32:35.410 INFO:teuthology.orchestra.run.smithi008.stdout: "complete_latency": { 2023-11-27T15:32:35.410 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 5217, 2023-11-27T15:32:35.410 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 64.741861706, 2023-11-27T15:32:35.410 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.012409787 2023-11-27T15:32:35.410 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:35.411 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.411 INFO:teuthology.orchestra.run.smithi008.stdout: "finisher-PurgeQueue": { 2023-11-27T15:32:35.411 INFO:teuthology.orchestra.run.smithi008.stdout: "queue_len": 0, 2023-11-27T15:32:35.411 INFO:teuthology.orchestra.run.smithi008.stdout: "complete_latency": { 2023-11-27T15:32:35.411 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 17214, 2023-11-27T15:32:35.412 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 4.706624801, 2023-11-27T15:32:35.412 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000273418 2023-11-27T15:32:35.412 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:35.412 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.412 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2023-11-27T15:32:35.413 INFO:teuthology.orchestra.run.smithi008.stdout: "request": 51953, 2023-11-27T15:32:35.413 INFO:teuthology.orchestra.run.smithi008.stdout: "reply": 41535, 2023-11-27T15:32:35.413 INFO:teuthology.orchestra.run.smithi008.stdout: "reply_latency": { 2023-11-27T15:32:35.413 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 41535, 2023-11-27T15:32:35.413 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 642.323680630, 2023-11-27T15:32:35.413 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.015464636 2023-11-27T15:32:35.414 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.414 INFO:teuthology.orchestra.run.smithi008.stdout: "slow_reply": 0, 2023-11-27T15:32:35.414 INFO:teuthology.orchestra.run.smithi008.stdout: "forward": 10418, 2023-11-27T15:32:35.414 INFO:teuthology.orchestra.run.smithi008.stdout: "dir_fetch_complete": 628, 2023-11-27T15:32:35.414 INFO:teuthology.orchestra.run.smithi008.stdout: "dir_fetch_keys": 0, 2023-11-27T15:32:35.415 INFO:teuthology.orchestra.run.smithi008.stdout: "dir_commit": 7180, 2023-11-27T15:32:35.415 INFO:teuthology.orchestra.run.smithi008.stdout: "dir_split": 133, 2023-11-27T15:32:35.415 INFO:teuthology.orchestra.run.smithi008.stdout: "dir_merge": 98, 2023-11-27T15:32:35.415 INFO:teuthology.orchestra.run.smithi008.stdout: "inodes": 57, 2023-11-27T15:32:35.415 INFO:teuthology.orchestra.run.smithi008.stdout: "inodes_top": 23, 2023-11-27T15:32:35.416 INFO:teuthology.orchestra.run.smithi008.stdout: "inodes_bottom": 18, 2023-11-27T15:32:35.416 INFO:teuthology.orchestra.run.smithi008.stdout: "inodes_pin_tail": 16, 2023-11-27T15:32:35.416 INFO:teuthology.orchestra.run.smithi008.stdout: "inodes_pinned": 23, 2023-11-27T15:32:35.416 INFO:teuthology.orchestra.run.smithi008.stdout: "inodes_expired": 22216, 2023-11-27T15:32:35.416 INFO:teuthology.orchestra.run.smithi008.stdout: "inodes_with_caps": 5, 2023-11-27T15:32:35.417 INFO:teuthology.orchestra.run.smithi008.stdout: "caps": 5, 2023-11-27T15:32:35.417 INFO:teuthology.orchestra.run.smithi008.stdout: "subtrees": 4, 2023-11-27T15:32:35.417 INFO:teuthology.orchestra.run.smithi008.stdout: "traverse": 62673, 2023-11-27T15:32:35.417 INFO:teuthology.orchestra.run.smithi008.stdout: "traverse_hit": 51672, 2023-11-27T15:32:35.417 INFO:teuthology.orchestra.run.smithi008.stdout: "traverse_forward": 10033, 2023-11-27T15:32:35.417 INFO:teuthology.orchestra.run.smithi008.stdout: "traverse_discover": 0, 2023-11-27T15:32:35.418 INFO:teuthology.orchestra.run.smithi008.stdout: "traverse_dir_fetch": 297, 2023-11-27T15:32:35.418 INFO:teuthology.orchestra.run.smithi008.stdout: "traverse_remote_ino": 0, 2023-11-27T15:32:35.418 INFO:teuthology.orchestra.run.smithi008.stdout: "traverse_lock": 0, 2023-11-27T15:32:35.418 INFO:teuthology.orchestra.run.smithi008.stdout: "load_cent": 508, 2023-11-27T15:32:35.418 INFO:teuthology.orchestra.run.smithi008.stdout: "q": 0, 2023-11-27T15:32:35.419 INFO:teuthology.orchestra.run.smithi008.stdout: "exported": 610, 2023-11-27T15:32:35.419 INFO:teuthology.orchestra.run.smithi008.stdout: "exported_inodes": 19211, 2023-11-27T15:32:35.419 INFO:teuthology.orchestra.run.smithi008.stdout: "imported": 667, 2023-11-27T15:32:35.419 INFO:teuthology.orchestra.run.smithi008.stdout: "imported_inodes": 0, 2023-11-27T15:32:35.419 INFO:teuthology.orchestra.run.smithi008.stdout: "openino_dir_fetch": 0, 2023-11-27T15:32:35.420 INFO:teuthology.orchestra.run.smithi008.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:32:35.420 INFO:teuthology.orchestra.run.smithi008.stdout: "openino_peer_discover": 0, 2023-11-27T15:32:35.420 INFO:teuthology.orchestra.run.smithi008.stdout: "root_rfiles": 4, 2023-11-27T15:32:35.420 INFO:teuthology.orchestra.run.smithi008.stdout: "root_rbytes": 246, 2023-11-27T15:32:35.420 INFO:teuthology.orchestra.run.smithi008.stdout: "root_rsnaps": 0, 2023-11-27T15:32:35.420 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:32:35.421 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_set_tag": 0, 2023-11-27T15:32:35.421 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:32:35.421 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:32:35.421 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:32:35.421 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:32:35.421 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:32:35.422 INFO:teuthology.orchestra.run.smithi008.stdout: "scrub_file_inodes": 0, 2023-11-27T15:32:35.422 INFO:teuthology.orchestra.run.smithi008.stdout: "handle_inode_file_caps": 8, 2023-11-27T15:32:35.422 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_cap_op_revoke": 92, 2023-11-27T15:32:35.422 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_cap_op_grant": 30369, 2023-11-27T15:32:35.422 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_cap_op_trunc": 1, 2023-11-27T15:32:35.423 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:32:35.423 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:32:35.423 INFO:teuthology.orchestra.run.smithi008.stdout: "handle_client_caps": 17810, 2023-11-27T15:32:35.423 INFO:teuthology.orchestra.run.smithi008.stdout: "handle_client_caps_dirty": 17578, 2023-11-27T15:32:35.423 INFO:teuthology.orchestra.run.smithi008.stdout: "handle_client_cap_release": 281, 2023-11-27T15:32:35.423 INFO:teuthology.orchestra.run.smithi008.stdout: "process_request_cap_release": 21923 2023-11-27T15:32:35.424 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.424 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_cache": { 2023-11-27T15:32:35.424 INFO:teuthology.orchestra.run.smithi008.stdout: "dir_update": 1117, 2023-11-27T15:32:35.424 INFO:teuthology.orchestra.run.smithi008.stdout: "dir_update_receipt": 0, 2023-11-27T15:32:35.424 INFO:teuthology.orchestra.run.smithi008.stdout: "dir_try_discover": 0, 2023-11-27T15:32:35.424 INFO:teuthology.orchestra.run.smithi008.stdout: "dir_send_discover": 2, 2023-11-27T15:32:35.425 INFO:teuthology.orchestra.run.smithi008.stdout: "dir_handle_discover": 1348, 2023-11-27T15:32:35.425 INFO:teuthology.orchestra.run.smithi008.stdout: "num_strays": 2, 2023-11-27T15:32:35.425 INFO:teuthology.orchestra.run.smithi008.stdout: "num_strays_delayed": 0, 2023-11-27T15:32:35.425 INFO:teuthology.orchestra.run.smithi008.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:32:35.425 INFO:teuthology.orchestra.run.smithi008.stdout: "strays_created": 10562, 2023-11-27T15:32:35.425 INFO:teuthology.orchestra.run.smithi008.stdout: "strays_enqueued": 10560, 2023-11-27T15:32:35.426 INFO:teuthology.orchestra.run.smithi008.stdout: "strays_reintegrated": 0, 2023-11-27T15:32:35.426 INFO:teuthology.orchestra.run.smithi008.stdout: "strays_migrated": 0, 2023-11-27T15:32:35.426 INFO:teuthology.orchestra.run.smithi008.stdout: "num_recovering_processing": 0, 2023-11-27T15:32:35.426 INFO:teuthology.orchestra.run.smithi008.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:32:35.426 INFO:teuthology.orchestra.run.smithi008.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:32:35.426 INFO:teuthology.orchestra.run.smithi008.stdout: "recovery_started": 0, 2023-11-27T15:32:35.427 INFO:teuthology.orchestra.run.smithi008.stdout: "recovery_completed": 0, 2023-11-27T15:32:35.427 INFO:teuthology.orchestra.run.smithi008.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:32:35.427 INFO:teuthology.orchestra.run.smithi008.stdout: "ireq_exportdir": 610, 2023-11-27T15:32:35.427 INFO:teuthology.orchestra.run.smithi008.stdout: "ireq_flush": 0, 2023-11-27T15:32:35.427 INFO:teuthology.orchestra.run.smithi008.stdout: "ireq_fragmentdir": 236, 2023-11-27T15:32:35.427 INFO:teuthology.orchestra.run.smithi008.stdout: "ireq_fragstats": 0, 2023-11-27T15:32:35.428 INFO:teuthology.orchestra.run.smithi008.stdout: "ireq_inodestats": 0 2023-11-27T15:32:35.428 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.428 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_log": { 2023-11-27T15:32:35.428 INFO:teuthology.orchestra.run.smithi008.stdout: "evadd": 61779, 2023-11-27T15:32:35.428 INFO:teuthology.orchestra.run.smithi008.stdout: "evex": 61737, 2023-11-27T15:32:35.428 INFO:teuthology.orchestra.run.smithi008.stdout: "evtrm": 61737, 2023-11-27T15:32:35.429 INFO:teuthology.orchestra.run.smithi008.stdout: "ev": 42, 2023-11-27T15:32:35.429 INFO:teuthology.orchestra.run.smithi008.stdout: "evexg": 0, 2023-11-27T15:32:35.429 INFO:teuthology.orchestra.run.smithi008.stdout: "evexd": 26, 2023-11-27T15:32:35.429 INFO:teuthology.orchestra.run.smithi008.stdout: "segadd": 95, 2023-11-27T15:32:35.429 INFO:teuthology.orchestra.run.smithi008.stdout: "segex": 94, 2023-11-27T15:32:35.429 INFO:teuthology.orchestra.run.smithi008.stdout: "segtrm": 94, 2023-11-27T15:32:35.430 INFO:teuthology.orchestra.run.smithi008.stdout: "seg": 2, 2023-11-27T15:32:35.430 INFO:teuthology.orchestra.run.smithi008.stdout: "segexg": 0, 2023-11-27T15:32:35.430 INFO:teuthology.orchestra.run.smithi008.stdout: "segexd": 1, 2023-11-27T15:32:35.430 INFO:teuthology.orchestra.run.smithi008.stdout: "expos": 169355884, 2023-11-27T15:32:35.430 INFO:teuthology.orchestra.run.smithi008.stdout: "wrpos": 169432900, 2023-11-27T15:32:35.430 INFO:teuthology.orchestra.run.smithi008.stdout: "rdpos": 0, 2023-11-27T15:32:35.431 INFO:teuthology.orchestra.run.smithi008.stdout: "jlat": { 2023-11-27T15:32:35.431 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 2395, 2023-11-27T15:32:35.431 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 110.633595252, 2023-11-27T15:32:35.431 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.046193567 2023-11-27T15:32:35.431 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.431 INFO:teuthology.orchestra.run.smithi008.stdout: "replayed": 0 2023-11-27T15:32:35.432 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.432 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_mem": { 2023-11-27T15:32:35.432 INFO:teuthology.orchestra.run.smithi008.stdout: "ino": 34, 2023-11-27T15:32:35.432 INFO:teuthology.orchestra.run.smithi008.stdout: "ino+": 30048, 2023-11-27T15:32:35.432 INFO:teuthology.orchestra.run.smithi008.stdout: "ino-": 30014, 2023-11-27T15:32:35.432 INFO:teuthology.orchestra.run.smithi008.stdout: "dir": 42, 2023-11-27T15:32:35.433 INFO:teuthology.orchestra.run.smithi008.stdout: "dir+": 3275, 2023-11-27T15:32:35.433 INFO:teuthology.orchestra.run.smithi008.stdout: "dir-": 3233, 2023-11-27T15:32:35.433 INFO:teuthology.orchestra.run.smithi008.stdout: "dn": 57, 2023-11-27T15:32:35.433 INFO:teuthology.orchestra.run.smithi008.stdout: "dn+": 52836, 2023-11-27T15:32:35.433 INFO:teuthology.orchestra.run.smithi008.stdout: "dn-": 52779, 2023-11-27T15:32:35.433 INFO:teuthology.orchestra.run.smithi008.stdout: "cap": 5, 2023-11-27T15:32:35.433 INFO:teuthology.orchestra.run.smithi008.stdout: "cap+": 40606, 2023-11-27T15:32:35.434 INFO:teuthology.orchestra.run.smithi008.stdout: "cap-": 40601, 2023-11-27T15:32:35.434 INFO:teuthology.orchestra.run.smithi008.stdout: "rss": 180828, 2023-11-27T15:32:35.434 INFO:teuthology.orchestra.run.smithi008.stdout: "heap": 207156 2023-11-27T15:32:35.434 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.434 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_server": { 2023-11-27T15:32:35.435 INFO:teuthology.orchestra.run.smithi008.stdout: "dispatch_client_request": 52930, 2023-11-27T15:32:35.435 INFO:teuthology.orchestra.run.smithi008.stdout: "dispatch_server_request": 269, 2023-11-27T15:32:35.435 INFO:teuthology.orchestra.run.smithi008.stdout: "handle_client_request": 51953, 2023-11-27T15:32:35.435 INFO:teuthology.orchestra.run.smithi008.stdout: "handle_client_session": 413, 2023-11-27T15:32:35.435 INFO:teuthology.orchestra.run.smithi008.stdout: "handle_peer_request": 544, 2023-11-27T15:32:35.435 INFO:teuthology.orchestra.run.smithi008.stdout: "req_create_latency": { 2023-11-27T15:32:35.436 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 19764, 2023-11-27T15:32:35.436 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 571.272393269, 2023-11-27T15:32:35.436 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.028904695 2023-11-27T15:32:35.436 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.436 INFO:teuthology.orchestra.run.smithi008.stdout: "req_getattr_latency": { 2023-11-27T15:32:35.436 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 54, 2023-11-27T15:32:35.437 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.084430364, 2023-11-27T15:32:35.437 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.001563525 2023-11-27T15:32:35.437 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.437 INFO:teuthology.orchestra.run.smithi008.stdout: "req_getfilelock_latency": { 2023-11-27T15:32:35.437 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.437 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.438 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.438 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.438 INFO:teuthology.orchestra.run.smithi008.stdout: "req_link_latency": { 2023-11-27T15:32:35.438 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.438 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.439 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.439 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.439 INFO:teuthology.orchestra.run.smithi008.stdout: "req_lookup_latency": { 2023-11-27T15:32:35.439 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 10202, 2023-11-27T15:32:35.439 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 17.766632732, 2023-11-27T15:32:35.439 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.001741485 2023-11-27T15:32:35.440 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.440 INFO:teuthology.orchestra.run.smithi008.stdout: "req_lookuphash_latency": { 2023-11-27T15:32:35.440 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.440 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.440 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.440 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.440 INFO:teuthology.orchestra.run.smithi008.stdout: "req_lookupino_latency": { 2023-11-27T15:32:35.441 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 1, 2023-11-27T15:32:35.441 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000205213, 2023-11-27T15:32:35.441 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000205213 2023-11-27T15:32:35.441 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.441 INFO:teuthology.orchestra.run.smithi008.stdout: "req_lookupname_latency": { 2023-11-27T15:32:35.441 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.442 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.442 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.442 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.442 INFO:teuthology.orchestra.run.smithi008.stdout: "req_lookupparent_latency": { 2023-11-27T15:32:35.442 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.442 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.443 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.443 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.443 INFO:teuthology.orchestra.run.smithi008.stdout: "req_lookupsnap_latency": { 2023-11-27T15:32:35.443 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.443 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.443 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.443 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.444 INFO:teuthology.orchestra.run.smithi008.stdout: "req_lssnap_latency": { 2023-11-27T15:32:35.444 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.444 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.444 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.444 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.444 INFO:teuthology.orchestra.run.smithi008.stdout: "req_mkdir_latency": { 2023-11-27T15:32:35.445 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 125, 2023-11-27T15:32:35.445 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.113055310, 2023-11-27T15:32:35.445 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000904442 2023-11-27T15:32:35.445 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.445 INFO:teuthology.orchestra.run.smithi008.stdout: "req_mknod_latency": { 2023-11-27T15:32:35.445 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.446 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.446 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.446 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.446 INFO:teuthology.orchestra.run.smithi008.stdout: "req_mksnap_latency": { 2023-11-27T15:32:35.446 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.446 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.446 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.447 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.447 INFO:teuthology.orchestra.run.smithi008.stdout: "req_open_latency": { 2023-11-27T15:32:35.447 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.447 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.447 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.447 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.448 INFO:teuthology.orchestra.run.smithi008.stdout: "req_readdir_latency": { 2023-11-27T15:32:35.448 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 780, 2023-11-27T15:32:35.448 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 1.817497999, 2023-11-27T15:32:35.448 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.002330125 2023-11-27T15:32:35.448 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.448 INFO:teuthology.orchestra.run.smithi008.stdout: "req_rename_latency": { 2023-11-27T15:32:35.448 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 2, 2023-11-27T15:32:35.449 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.002139166, 2023-11-27T15:32:35.449 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.001069583 2023-11-27T15:32:35.449 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.449 INFO:teuthology.orchestra.run.smithi008.stdout: "req_renamesnap_latency": { 2023-11-27T15:32:35.449 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.449 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.449 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.450 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.450 INFO:teuthology.orchestra.run.smithi008.stdout: "req_snapdiff_latency": { 2023-11-27T15:32:35.450 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.450 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.450 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.450 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.451 INFO:teuthology.orchestra.run.smithi008.stdout: "req_rmdir_latency": { 2023-11-27T15:32:35.451 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 113, 2023-11-27T15:32:35.451 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.405153435, 2023-11-27T15:32:35.451 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.003585428 2023-11-27T15:32:35.451 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.451 INFO:teuthology.orchestra.run.smithi008.stdout: "req_rmsnap_latency": { 2023-11-27T15:32:35.451 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.452 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.452 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.452 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.452 INFO:teuthology.orchestra.run.smithi008.stdout: "req_rmxattr_latency": { 2023-11-27T15:32:35.452 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 3, 2023-11-27T15:32:35.452 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.001610594, 2023-11-27T15:32:35.453 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000536864 2023-11-27T15:32:35.453 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.453 INFO:teuthology.orchestra.run.smithi008.stdout: "req_setattr_latency": { 2023-11-27T15:32:35.453 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 4, 2023-11-27T15:32:35.453 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.011166387, 2023-11-27T15:32:35.453 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.002791596 2023-11-27T15:32:35.453 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.454 INFO:teuthology.orchestra.run.smithi008.stdout: "req_setdirlayout_latency": { 2023-11-27T15:32:35.454 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.454 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.454 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.454 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.454 INFO:teuthology.orchestra.run.smithi008.stdout: "req_setfilelock_latency": { 2023-11-27T15:32:35.455 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 22, 2023-11-27T15:32:35.455 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 10.018607081, 2023-11-27T15:32:35.455 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.455391230 2023-11-27T15:32:35.455 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.455 INFO:teuthology.orchestra.run.smithi008.stdout: "req_setlayout_latency": { 2023-11-27T15:32:35.455 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.455 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.456 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.456 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.456 INFO:teuthology.orchestra.run.smithi008.stdout: "req_setxattr_latency": { 2023-11-27T15:32:35.456 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 16, 2023-11-27T15:32:35.456 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.029447084, 2023-11-27T15:32:35.456 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.001840442 2023-11-27T15:32:35.456 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.457 INFO:teuthology.orchestra.run.smithi008.stdout: "req_symlink_latency": { 2023-11-27T15:32:35.457 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.457 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.457 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.457 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.457 INFO:teuthology.orchestra.run.smithi008.stdout: "req_unlink_latency": { 2023-11-27T15:32:35.457 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 10449, 2023-11-27T15:32:35.458 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 40.801341996, 2023-11-27T15:32:35.458 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.003904808 2023-11-27T15:32:35.458 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.458 INFO:teuthology.orchestra.run.smithi008.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:32:35.458 INFO:teuthology.orchestra.run.smithi008.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:32:35.458 INFO:teuthology.orchestra.run.smithi008.stdout: "req_getvxattr_latency": { 2023-11-27T15:32:35.459 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.459 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.459 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.459 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:35.459 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.459 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_sessions": { 2023-11-27T15:32:35.459 INFO:teuthology.orchestra.run.smithi008.stdout: "session_count": 1, 2023-11-27T15:32:35.460 INFO:teuthology.orchestra.run.smithi008.stdout: "session_add": 3, 2023-11-27T15:32:35.460 INFO:teuthology.orchestra.run.smithi008.stdout: "session_remove": 2, 2023-11-27T15:32:35.460 INFO:teuthology.orchestra.run.smithi008.stdout: "sessions_open": 1, 2023-11-27T15:32:35.460 INFO:teuthology.orchestra.run.smithi008.stdout: "sessions_stale": 0, 2023-11-27T15:32:35.460 INFO:teuthology.orchestra.run.smithi008.stdout: "total_load": 8710, 2023-11-27T15:32:35.460 INFO:teuthology.orchestra.run.smithi008.stdout: "average_load": 8710, 2023-11-27T15:32:35.460 INFO:teuthology.orchestra.run.smithi008.stdout: "avg_session_uptime": 413 2023-11-27T15:32:35.461 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.461 INFO:teuthology.orchestra.run.smithi008.stdout: "mempool": { 2023-11-27T15:32:35.461 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_bytes": 360, 2023-11-27T15:32:35.461 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_items": 360, 2023-11-27T15:32:35.461 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:32:35.461 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:32:35.461 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:32:35.462 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:32:35.462 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:32:35.462 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:32:35.462 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:32:35.462 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:32:35.462 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:32:35.463 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:32:35.463 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:32:35.463 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:32:35.463 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:32:35.463 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_items": 0, 2023-11-27T15:32:35.463 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:32:35.463 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_items": 0, 2023-11-27T15:32:35.464 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:32:35.464 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:32:35.464 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:32:35.464 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:32:35.464 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:32:35.464 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:32:35.464 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:32:35.465 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_items": 0, 2023-11-27T15:32:35.465 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:32:35.465 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:32:35.465 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:32:35.465 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_items": 0, 2023-11-27T15:32:35.465 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_bytes": 0, 2023-11-27T15:32:35.465 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_items": 0, 2023-11-27T15:32:35.466 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:32:35.466 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:32:35.466 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:32:35.466 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:32:35.466 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_bytes": 292433, 2023-11-27T15:32:35.466 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_items": 131, 2023-11-27T15:32:35.466 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_bytes": 704, 2023-11-27T15:32:35.467 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_items": 8, 2023-11-27T15:32:35.467 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_bytes": 0, 2023-11-27T15:32:35.467 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_items": 0, 2023-11-27T15:32:35.467 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:32:35.467 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_items": 0, 2023-11-27T15:32:35.467 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:32:35.468 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_items": 0, 2023-11-27T15:32:35.468 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_bytes": 8792, 2023-11-27T15:32:35.468 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_items": 168, 2023-11-27T15:32:35.468 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:32:35.468 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:32:35.468 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_bytes": 0, 2023-11-27T15:32:35.468 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_items": 0, 2023-11-27T15:32:35.469 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_bytes": 177514, 2023-11-27T15:32:35.469 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_items": 1370, 2023-11-27T15:32:35.469 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_bytes": 0, 2023-11-27T15:32:35.469 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_items": 0, 2023-11-27T15:32:35.469 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_bytes": 0, 2023-11-27T15:32:35.469 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_items": 0 2023-11-27T15:32:35.469 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.470 INFO:teuthology.orchestra.run.smithi008.stdout: "objecter": { 2023-11-27T15:32:35.470 INFO:teuthology.orchestra.run.smithi008.stdout: "op_active": 0, 2023-11-27T15:32:35.470 INFO:teuthology.orchestra.run.smithi008.stdout: "op_laggy": 0, 2023-11-27T15:32:35.470 INFO:teuthology.orchestra.run.smithi008.stdout: "op_send": 72451, 2023-11-27T15:32:35.470 INFO:teuthology.orchestra.run.smithi008.stdout: "op_send_bytes": 215310615, 2023-11-27T15:32:35.470 INFO:teuthology.orchestra.run.smithi008.stdout: "op_resend": 30, 2023-11-27T15:32:35.470 INFO:teuthology.orchestra.run.smithi008.stdout: "op_reply": 72451, 2023-11-27T15:32:35.471 INFO:teuthology.orchestra.run.smithi008.stdout: "op_latency": { 2023-11-27T15:32:35.471 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 72451, 2023-11-27T15:32:35.471 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 26606.481584822, 2023-11-27T15:32:35.471 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.367234152 2023-11-27T15:32:35.471 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.471 INFO:teuthology.orchestra.run.smithi008.stdout: "op_inflight": 0, 2023-11-27T15:32:35.471 INFO:teuthology.orchestra.run.smithi008.stdout: "oplen_avg": { 2023-11-27T15:32:35.472 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 72451, 2023-11-27T15:32:35.472 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 142861 2023-11-27T15:32:35.472 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.472 INFO:teuthology.orchestra.run.smithi008.stdout: "op": 72451, 2023-11-27T15:32:35.472 INFO:teuthology.orchestra.run.smithi008.stdout: "op_r": 866, 2023-11-27T15:32:35.472 INFO:teuthology.orchestra.run.smithi008.stdout: "op_w": 71585, 2023-11-27T15:32:35.472 INFO:teuthology.orchestra.run.smithi008.stdout: "op_rmw": 0, 2023-11-27T15:32:35.473 INFO:teuthology.orchestra.run.smithi008.stdout: "op_pg": 0, 2023-11-27T15:32:35.473 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_stat": 5985, 2023-11-27T15:32:35.473 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_create": 38589, 2023-11-27T15:32:35.473 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_read": 238, 2023-11-27T15:32:35.473 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_write": 2633, 2023-11-27T15:32:35.473 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_writefull": 218, 2023-11-27T15:32:35.473 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_writesame": 0, 2023-11-27T15:32:35.474 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_append": 0, 2023-11-27T15:32:35.474 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_zero": 0, 2023-11-27T15:32:35.474 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_truncate": 5, 2023-11-27T15:32:35.474 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_delete": 22838, 2023-11-27T15:32:35.474 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_mapext": 0, 2023-11-27T15:32:35.474 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_sparse_read": 0, 2023-11-27T15:32:35.474 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_clonerange": 0, 2023-11-27T15:32:35.475 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_getxattr": 0, 2023-11-27T15:32:35.475 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_setxattr": 58489, 2023-11-27T15:32:35.475 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:32:35.475 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_rmxattr": 0, 2023-11-27T15:32:35.475 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:32:35.475 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_call": 0, 2023-11-27T15:32:35.475 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_watch": 0, 2023-11-27T15:32:35.476 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_notify": 0, 2023-11-27T15:32:35.476 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:32:35.476 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_pgls": 0, 2023-11-27T15:32:35.476 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:32:35.476 INFO:teuthology.orchestra.run.smithi008.stdout: "osdop_other": 1, 2023-11-27T15:32:35.476 INFO:teuthology.orchestra.run.smithi008.stdout: "linger_active": 0, 2023-11-27T15:32:35.476 INFO:teuthology.orchestra.run.smithi008.stdout: "linger_send": 0, 2023-11-27T15:32:35.477 INFO:teuthology.orchestra.run.smithi008.stdout: "linger_resend": 0, 2023-11-27T15:32:35.477 INFO:teuthology.orchestra.run.smithi008.stdout: "linger_ping": 0, 2023-11-27T15:32:35.477 INFO:teuthology.orchestra.run.smithi008.stdout: "poolop_active": 0, 2023-11-27T15:32:35.477 INFO:teuthology.orchestra.run.smithi008.stdout: "poolop_send": 0, 2023-11-27T15:32:35.477 INFO:teuthology.orchestra.run.smithi008.stdout: "poolop_resend": 0, 2023-11-27T15:32:35.477 INFO:teuthology.orchestra.run.smithi008.stdout: "poolstat_active": 0, 2023-11-27T15:32:35.477 INFO:teuthology.orchestra.run.smithi008.stdout: "poolstat_send": 0, 2023-11-27T15:32:35.477 INFO:teuthology.orchestra.run.smithi008.stdout: "poolstat_resend": 0, 2023-11-27T15:32:35.478 INFO:teuthology.orchestra.run.smithi008.stdout: "statfs_active": 0, 2023-11-27T15:32:35.478 INFO:teuthology.orchestra.run.smithi008.stdout: "statfs_send": 0, 2023-11-27T15:32:35.478 INFO:teuthology.orchestra.run.smithi008.stdout: "statfs_resend": 0, 2023-11-27T15:32:35.478 INFO:teuthology.orchestra.run.smithi008.stdout: "command_active": 0, 2023-11-27T15:32:35.478 INFO:teuthology.orchestra.run.smithi008.stdout: "command_send": 0, 2023-11-27T15:32:35.478 INFO:teuthology.orchestra.run.smithi008.stdout: "command_resend": 0, 2023-11-27T15:32:35.479 INFO:teuthology.orchestra.run.smithi008.stdout: "map_epoch": 94, 2023-11-27T15:32:35.479 INFO:teuthology.orchestra.run.smithi008.stdout: "map_full": 0, 2023-11-27T15:32:35.479 INFO:teuthology.orchestra.run.smithi008.stdout: "map_inc": 11, 2023-11-27T15:32:35.479 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_sessions": 12, 2023-11-27T15:32:35.479 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_session_open": 12, 2023-11-27T15:32:35.479 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_session_close": 0, 2023-11-27T15:32:35.479 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_laggy": 0, 2023-11-27T15:32:35.480 INFO:teuthology.orchestra.run.smithi008.stdout: "omap_wr": 12264, 2023-11-27T15:32:35.480 INFO:teuthology.orchestra.run.smithi008.stdout: "omap_rd": 1256, 2023-11-27T15:32:35.480 INFO:teuthology.orchestra.run.smithi008.stdout: "omap_del": 345 2023-11-27T15:32:35.480 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.480 INFO:teuthology.orchestra.run.smithi008.stdout: "oft": { 2023-11-27T15:32:35.480 INFO:teuthology.orchestra.run.smithi008.stdout: "omap_total_objs": 1, 2023-11-27T15:32:35.480 INFO:teuthology.orchestra.run.smithi008.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:32:35.481 INFO:teuthology.orchestra.run.smithi008.stdout: "omap_total_updates": 0, 2023-11-27T15:32:35.481 INFO:teuthology.orchestra.run.smithi008.stdout: "omap_total_removes": 0 2023-11-27T15:32:35.481 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.481 INFO:teuthology.orchestra.run.smithi008.stdout: "osd-slow-ops": { 2023-11-27T15:32:35.481 INFO:teuthology.orchestra.run.smithi008.stdout: "slow_ops_count": 53069 2023-11-27T15:32:35.481 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.482 INFO:teuthology.orchestra.run.smithi008.stdout: "purge_queue": { 2023-11-27T15:32:35.482 INFO:teuthology.orchestra.run.smithi008.stdout: "pq_executing_ops": 0, 2023-11-27T15:32:35.482 INFO:teuthology.orchestra.run.smithi008.stdout: "pq_executing_ops_high_water": 16, 2023-11-27T15:32:35.482 INFO:teuthology.orchestra.run.smithi008.stdout: "pq_executing": 0, 2023-11-27T15:32:35.482 INFO:teuthology.orchestra.run.smithi008.stdout: "pq_executing_high_water": 13, 2023-11-27T15:32:35.482 INFO:teuthology.orchestra.run.smithi008.stdout: "pq_executed": 10560, 2023-11-27T15:32:35.483 INFO:teuthology.orchestra.run.smithi008.stdout: "pq_item_in_journal": 0 2023-11-27T15:32:35.483 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.483 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:32:35.483 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2023-11-27T15:32:35.483 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 104857600, 2023-11-27T15:32:35.483 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2023-11-27T15:32:35.483 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 165382, 2023-11-27T15:32:35.483 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 85157822, 2023-11-27T15:32:35.484 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:35.484 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 165382, 2023-11-27T15:32:35.484 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 0, 2023-11-27T15:32:35.484 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 0, 2023-11-27T15:32:35.484 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 165382, 2023-11-27T15:32:35.484 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 85157822, 2023-11-27T15:32:35.484 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2023-11-27T15:32:35.485 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.485 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.485 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.485 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:35.485 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.485 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-objecter_bytes": { 2023-11-27T15:32:35.485 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2023-11-27T15:32:35.486 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 104857600, 2023-11-27T15:32:35.486 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2023-11-27T15:32:35.486 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 0, 2023-11-27T15:32:35.486 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 0, 2023-11-27T15:32:35.486 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:35.486 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:35.486 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 72451, 2023-11-27T15:32:35.487 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 216450719, 2023-11-27T15:32:35.487 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 72451, 2023-11-27T15:32:35.487 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 216450719, 2023-11-27T15:32:35.487 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2023-11-27T15:32:35.487 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.487 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.487 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.488 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:35.488 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.488 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-objecter_ops": { 2023-11-27T15:32:35.488 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2023-11-27T15:32:35.488 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 1024, 2023-11-27T15:32:35.488 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2023-11-27T15:32:35.488 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 0, 2023-11-27T15:32:35.489 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 0, 2023-11-27T15:32:35.489 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:35.489 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:35.489 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 72451, 2023-11-27T15:32:35.489 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 72451, 2023-11-27T15:32:35.489 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 72451, 2023-11-27T15:32:35.489 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 72451, 2023-11-27T15:32:35.490 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2023-11-27T15:32:35.490 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.490 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.490 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.490 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:35.490 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.490 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:32:35.491 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2023-11-27T15:32:35.491 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 3758096384, 2023-11-27T15:32:35.491 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2023-11-27T15:32:35.491 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 10560, 2023-11-27T15:32:35.491 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 1150152, 2023-11-27T15:32:35.491 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:35.491 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 10560, 2023-11-27T15:32:35.492 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 0, 2023-11-27T15:32:35.492 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 0, 2023-11-27T15:32:35.492 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 238, 2023-11-27T15:32:35.492 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 1150152, 2023-11-27T15:32:35.492 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2023-11-27T15:32:35.492 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.492 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.493 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.493 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:35.493 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:35.493 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-write_buf_throttle-0x5585e06f7cc0": { 2023-11-27T15:32:35.493 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2023-11-27T15:32:35.493 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 3758096384, 2023-11-27T15:32:35.493 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2023-11-27T15:32:35.493 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 61779, 2023-11-27T15:32:35.494 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 165238596, 2023-11-27T15:32:35.494 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:35.494 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 61779, 2023-11-27T15:32:35.494 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 0, 2023-11-27T15:32:35.494 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 0, 2023-11-27T15:32:35.494 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 2395, 2023-11-27T15:32:35.495 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 165238596, 2023-11-27T15:32:35.495 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2023-11-27T15:32:35.495 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:35.495 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:35.495 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:35.495 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:35.495 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:35.495 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:32:35.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:35 smithi060 ceph-mon[131518]: pgmap v562: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:32:35.853 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:35 smithi008 ceph-mon[128788]: pgmap v562: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:32:35.857 INFO:tasks.check_counter:Daemon mds.a mds.exported=610 2023-11-27T15:32:35.857 INFO:tasks.check_counter:Daemon mds.a mds.imported=667 2023-11-27T15:32:35.857 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=1117 2023-11-27T15:32:35.857 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2023-11-27T15:32:35.857 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=133 2023-11-27T15:32:35.857 DEBUG:tasks.check_counter:Getting stats from d 2023-11-27T15:32:35.858 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.d perf dump 2023-11-27T15:32:35.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:35 smithi029 ceph-mon[131166]: pgmap v562: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:32:36.718 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:32:37.680 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:37 smithi008 ceph-mon[128788]: pgmap v563: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:32:37.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:37 smithi060 ceph-mon[131518]: pgmap v563: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:32:37.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:37 smithi029 ceph-mon[131166]: pgmap v563: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:32:38.414 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:32:38.414 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:32:38.414 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 161, 2023-11-27T15:32:38.414 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 158, 2023-11-27T15:32:38.415 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 455360, 2023-11-27T15:32:38.415 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 89232, 2023-11-27T15:32:38.415 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 4, 2023-11-27T15:32:38.415 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 0, 2023-11-27T15:32:38.415 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.035326920, 2023-11-27T15:32:38.415 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.013571021, 2023-11-27T15:32:38.415 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.017057658, 2023-11-27T15:32:38.416 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.003743442, 2023-11-27T15:32:38.416 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:38.416 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 157, 2023-11-27T15:32:38.416 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.014944944, 2023-11-27T15:32:38.416 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000095190 2023-11-27T15:32:38.416 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.416 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:38.417 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:38.417 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:38.417 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:38.417 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.417 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 455360, 2023-11-27T15:32:38.417 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 89232 2023-11-27T15:32:38.417 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.418 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:32:38.418 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 5, 2023-11-27T15:32:38.418 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 121, 2023-11-27T15:32:38.418 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 3152, 2023-11-27T15:32:38.418 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 97152, 2023-11-27T15:32:38.418 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 3, 2023-11-27T15:32:38.418 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 2, 2023-11-27T15:32:38.419 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.014195242, 2023-11-27T15:32:38.419 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.010409069, 2023-11-27T15:32:38.419 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.002297553, 2023-11-27T15:32:38.419 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:32:38.419 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:38.419 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 120, 2023-11-27T15:32:38.419 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.008676440, 2023-11-27T15:32:38.420 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000072303 2023-11-27T15:32:38.420 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.420 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:38.420 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:38.420 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:38.420 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:38.420 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.421 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-27T15:32:38.421 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 97152 2023-11-27T15:32:38.421 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.421 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:32:38.421 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 0, 2023-11-27T15:32:38.421 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 2, 2023-11-27T15:32:38.421 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:32:38.421 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 0, 2023-11-27T15:32:38.422 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 2, 2023-11-27T15:32:38.422 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 1, 2023-11-27T15:32:38.422 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.001047638, 2023-11-27T15:32:38.422 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:32:38.422 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.000862257, 2023-11-27T15:32:38.422 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:32:38.423 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:38.423 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:38.423 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:38.423 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:38.423 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.423 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:38.423 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:38.424 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:38.424 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:38.424 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.424 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:32:38.424 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:32:38.424 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.424 INFO:teuthology.orchestra.run.smithi008.stdout: "cct": { 2023-11-27T15:32:38.424 INFO:teuthology.orchestra.run.smithi008.stdout: "total_workers": 0, 2023-11-27T15:32:38.425 INFO:teuthology.orchestra.run.smithi008.stdout: "unhealthy_workers": 0 2023-11-27T15:32:38.425 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.425 INFO:teuthology.orchestra.run.smithi008.stdout: "mempool": { 2023-11-27T15:32:38.425 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:32:38.425 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_items": 0, 2023-11-27T15:32:38.425 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:32:38.425 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:32:38.425 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:32:38.426 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:32:38.426 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:32:38.426 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:32:38.426 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:32:38.426 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:32:38.426 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:32:38.426 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:32:38.427 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:32:38.427 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:32:38.427 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:32:38.427 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_items": 0, 2023-11-27T15:32:38.427 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:32:38.427 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_items": 0, 2023-11-27T15:32:38.427 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:32:38.428 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:32:38.428 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:32:38.428 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:32:38.428 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:32:38.428 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:32:38.428 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:32:38.428 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_items": 0, 2023-11-27T15:32:38.429 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:32:38.429 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:32:38.429 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:32:38.429 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_items": 0, 2023-11-27T15:32:38.429 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_bytes": 0, 2023-11-27T15:32:38.429 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_items": 0, 2023-11-27T15:32:38.429 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:32:38.429 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:32:38.430 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:32:38.430 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:32:38.430 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_bytes": 149998, 2023-11-27T15:32:38.430 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_items": 30, 2023-11-27T15:32:38.430 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:32:38.431 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_items": 0, 2023-11-27T15:32:38.431 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_bytes": 0, 2023-11-27T15:32:38.431 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_items": 0, 2023-11-27T15:32:38.431 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:32:38.431 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_items": 0, 2023-11-27T15:32:38.431 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:32:38.431 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_items": 0, 2023-11-27T15:32:38.432 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_bytes": 0, 2023-11-27T15:32:38.432 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_items": 0, 2023-11-27T15:32:38.432 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:32:38.432 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:32:38.432 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_bytes": 0, 2023-11-27T15:32:38.432 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_items": 0, 2023-11-27T15:32:38.432 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_bytes": 928, 2023-11-27T15:32:38.433 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_items": 2, 2023-11-27T15:32:38.433 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_bytes": 0, 2023-11-27T15:32:38.433 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_items": 0, 2023-11-27T15:32:38.433 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_bytes": 0, 2023-11-27T15:32:38.433 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_items": 0 2023-11-27T15:32:38.433 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:38.433 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:32:38.433 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2023-11-27T15:32:38.434 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 104857600, 2023-11-27T15:32:38.434 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2023-11-27T15:32:38.434 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 162, 2023-11-27T15:32:38.434 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 432614, 2023-11-27T15:32:38.434 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:38.434 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 162, 2023-11-27T15:32:38.434 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 0, 2023-11-27T15:32:38.435 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 0, 2023-11-27T15:32:38.435 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 162, 2023-11-27T15:32:38.435 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 432614, 2023-11-27T15:32:38.435 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2023-11-27T15:32:38.435 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:38.435 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:38.435 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:38.436 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:38.436 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:38.436 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:32:38.918 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-11-27T15:32:38.918 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-11-27T15:32:38.919 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-11-27T15:32:38.919 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-11-27T15:32:38.919 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-11-27T15:32:38.919 DEBUG:tasks.check_counter:Getting stats from g 2023-11-27T15:32:38.919 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.g perf dump 2023-11-27T15:32:39.782 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:32:39.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:39 smithi060 ceph-mon[131518]: pgmap v564: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-11-27T15:32:39.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:39 smithi008 ceph-mon[128788]: pgmap v564: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-11-27T15:32:39.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:39 smithi029 ceph-mon[131166]: pgmap v564: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2023-11-27T15:32:41.366 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:32:41.366 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:32:41.366 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 4, 2023-11-27T15:32:41.367 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 4, 2023-11-27T15:32:41.367 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 2992, 2023-11-27T15:32:41.367 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 448, 2023-11-27T15:32:41.367 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 4, 2023-11-27T15:32:41.367 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 0, 2023-11-27T15:32:41.367 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.004737450, 2023-11-27T15:32:41.367 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.000069085, 2023-11-27T15:32:41.368 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.001328439, 2023-11-27T15:32:41.368 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:32:41.368 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:41.368 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 3, 2023-11-27T15:32:41.368 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.001409203, 2023-11-27T15:32:41.368 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000469734 2023-11-27T15:32:41.368 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.369 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:41.369 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:41.369 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:41.369 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:41.369 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.369 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-27T15:32:41.369 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 448 2023-11-27T15:32:41.370 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.370 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:32:41.370 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 158, 2023-11-27T15:32:41.370 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 155, 2023-11-27T15:32:41.370 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 454544, 2023-11-27T15:32:41.370 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 87456, 2023-11-27T15:32:41.370 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 2, 2023-11-27T15:32:41.371 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 1, 2023-11-27T15:32:41.371 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.032602865, 2023-11-27T15:32:41.371 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.012896010, 2023-11-27T15:32:41.371 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.018054572, 2023-11-27T15:32:41.371 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.004744182, 2023-11-27T15:32:41.371 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:41.371 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 154, 2023-11-27T15:32:41.371 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.012136575, 2023-11-27T15:32:41.372 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000078808 2023-11-27T15:32:41.372 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.372 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:41.372 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:41.372 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:41.372 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:41.372 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.373 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 454544, 2023-11-27T15:32:41.373 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 87456 2023-11-27T15:32:41.373 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.373 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:32:41.373 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 1, 2023-11-27T15:32:41.373 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 117, 2023-11-27T15:32:41.373 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 160, 2023-11-27T15:32:41.374 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 96352, 2023-11-27T15:32:41.374 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 3, 2023-11-27T15:32:41.374 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 2, 2023-11-27T15:32:41.374 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.012959227, 2023-11-27T15:32:41.374 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.009337283, 2023-11-27T15:32:41.374 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.002015569, 2023-11-27T15:32:41.374 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:32:41.375 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:41.375 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 115, 2023-11-27T15:32:41.375 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.007981988, 2023-11-27T15:32:41.375 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000069408 2023-11-27T15:32:41.375 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.375 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:41.376 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:41.376 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:41.376 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:41.376 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.376 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:32:41.376 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 96352 2023-11-27T15:32:41.376 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.377 INFO:teuthology.orchestra.run.smithi008.stdout: "cct": { 2023-11-27T15:32:41.377 INFO:teuthology.orchestra.run.smithi008.stdout: "total_workers": 0, 2023-11-27T15:32:41.377 INFO:teuthology.orchestra.run.smithi008.stdout: "unhealthy_workers": 0 2023-11-27T15:32:41.377 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.377 INFO:teuthology.orchestra.run.smithi008.stdout: "mempool": { 2023-11-27T15:32:41.377 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:32:41.377 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_items": 0, 2023-11-27T15:32:41.378 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:32:41.378 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:32:41.378 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:32:41.378 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:32:41.378 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:32:41.378 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:32:41.379 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:32:41.379 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:32:41.379 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:32:41.379 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:32:41.379 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:32:41.379 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:32:41.379 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:32:41.380 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_items": 0, 2023-11-27T15:32:41.380 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:32:41.380 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_items": 0, 2023-11-27T15:32:41.380 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:32:41.380 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:32:41.380 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:32:41.380 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:32:41.380 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:32:41.381 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:32:41.381 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:32:41.381 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_items": 0, 2023-11-27T15:32:41.381 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:32:41.381 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:32:41.381 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:32:41.381 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_items": 0, 2023-11-27T15:32:41.382 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_bytes": 0, 2023-11-27T15:32:41.382 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_items": 0, 2023-11-27T15:32:41.382 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:32:41.382 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:32:41.382 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:32:41.382 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:32:41.382 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_bytes": 149998, 2023-11-27T15:32:41.383 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_items": 30, 2023-11-27T15:32:41.383 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:32:41.383 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_items": 0, 2023-11-27T15:32:41.383 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_bytes": 0, 2023-11-27T15:32:41.383 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_items": 0, 2023-11-27T15:32:41.383 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:32:41.383 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_items": 0, 2023-11-27T15:32:41.384 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:32:41.384 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_items": 0, 2023-11-27T15:32:41.384 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_bytes": 0, 2023-11-27T15:32:41.384 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_items": 0, 2023-11-27T15:32:41.384 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:32:41.384 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:32:41.384 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_bytes": 0, 2023-11-27T15:32:41.384 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_items": 0, 2023-11-27T15:32:41.385 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_bytes": 928, 2023-11-27T15:32:41.385 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_items": 2, 2023-11-27T15:32:41.385 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_bytes": 0, 2023-11-27T15:32:41.385 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_items": 0, 2023-11-27T15:32:41.385 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_bytes": 0, 2023-11-27T15:32:41.385 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_items": 0 2023-11-27T15:32:41.385 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:41.386 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:32:41.386 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2023-11-27T15:32:41.386 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 104857600, 2023-11-27T15:32:41.386 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2023-11-27T15:32:41.386 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 159, 2023-11-27T15:32:41.386 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 432224, 2023-11-27T15:32:41.386 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:41.386 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 159, 2023-11-27T15:32:41.387 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 0, 2023-11-27T15:32:41.387 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 0, 2023-11-27T15:32:41.387 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 159, 2023-11-27T15:32:41.387 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 432224, 2023-11-27T15:32:41.387 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2023-11-27T15:32:41.387 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:41.387 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:41.388 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:41.388 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:41.388 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:41.388 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:32:41.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:41 smithi060 ceph-mon[131518]: pgmap v565: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-27T15:32:41.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:41 smithi008 ceph-mon[128788]: pgmap v565: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-27T15:32:41.880 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-11-27T15:32:41.880 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-11-27T15:32:41.880 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-11-27T15:32:41.881 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-11-27T15:32:41.881 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-11-27T15:32:41.881 DEBUG:tasks.check_counter:Getting stats from j 2023-11-27T15:32:41.881 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.j perf dump 2023-11-27T15:32:41.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:41 smithi029 ceph-mon[131166]: pgmap v565: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-27T15:32:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:32:43.573 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:43 smithi008 ceph-mon[128788]: pgmap v566: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-27T15:32:43.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:43 smithi060 ceph-mon[131518]: pgmap v566: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-27T15:32:43.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:43 smithi029 ceph-mon[131166]: pgmap v566: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-27T15:32:44.347 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2023-11-27T15:32:44.347 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:32:44.347 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 0, 2023-11-27T15:32:44.347 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 2, 2023-11-27T15:32:44.347 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:32:44.348 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 0, 2023-11-27T15:32:44.348 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 4, 2023-11-27T15:32:44.348 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 0, 2023-11-27T15:32:44.348 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.003261651, 2023-11-27T15:32:44.348 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:32:44.348 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.000558845, 2023-11-27T15:32:44.348 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:32:44.348 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:44.349 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:44.349 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:44.349 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:44.349 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.349 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:44.349 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:44.349 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:44.349 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:44.350 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.350 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:32:44.350 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:32:44.350 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.350 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:32:44.350 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 168, 2023-11-27T15:32:44.350 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 163, 2023-11-27T15:32:44.350 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 459168, 2023-11-27T15:32:44.351 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 91456, 2023-11-27T15:32:44.351 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 2, 2023-11-27T15:32:44.351 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 1, 2023-11-27T15:32:44.351 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.029855615, 2023-11-27T15:32:44.351 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.009820705, 2023-11-27T15:32:44.351 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.018766119, 2023-11-27T15:32:44.351 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.004145570, 2023-11-27T15:32:44.351 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:44.352 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 163, 2023-11-27T15:32:44.352 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.013155983, 2023-11-27T15:32:44.352 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000080711 2023-11-27T15:32:44.352 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.352 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:44.352 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:44.352 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:44.352 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:44.353 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.353 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 459168, 2023-11-27T15:32:44.353 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 91456 2023-11-27T15:32:44.353 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.353 INFO:teuthology.orchestra.run.smithi008.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:32:44.353 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_messages": 1, 2023-11-27T15:32:44.353 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages": 121, 2023-11-27T15:32:44.353 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_bytes": 160, 2023-11-27T15:32:44.354 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_bytes": 97056, 2023-11-27T15:32:44.354 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_created_connections": 3, 2023-11-27T15:32:44.354 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_active_connections": 2, 2023-11-27T15:32:44.354 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_total_time": 0.012786244, 2023-11-27T15:32:44.354 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_send_time": 0.009815278, 2023-11-27T15:32:44.354 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_recv_time": 0.001453760, 2023-11-27T15:32:44.354 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:32:44.354 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:44.355 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 119, 2023-11-27T15:32:44.355 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.008922195, 2023-11-27T15:32:44.355 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000074976 2023-11-27T15:32:44.355 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.355 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:44.355 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:44.355 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:44.355 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:44.356 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.356 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:32:44.356 INFO:teuthology.orchestra.run.smithi008.stdout: "msgr_send_encrypted_bytes": 97056 2023-11-27T15:32:44.356 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.356 INFO:teuthology.orchestra.run.smithi008.stdout: "cct": { 2023-11-27T15:32:44.356 INFO:teuthology.orchestra.run.smithi008.stdout: "total_workers": 0, 2023-11-27T15:32:44.356 INFO:teuthology.orchestra.run.smithi008.stdout: "unhealthy_workers": 0 2023-11-27T15:32:44.356 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.357 INFO:teuthology.orchestra.run.smithi008.stdout: "mempool": { 2023-11-27T15:32:44.357 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:32:44.357 INFO:teuthology.orchestra.run.smithi008.stdout: "bloom_filter_items": 0, 2023-11-27T15:32:44.357 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:32:44.357 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:32:44.357 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:32:44.357 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:32:44.357 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:32:44.358 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:32:44.358 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:32:44.358 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:32:44.358 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:32:44.358 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:32:44.358 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:32:44.358 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:32:44.358 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:32:44.359 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_extent_items": 0, 2023-11-27T15:32:44.359 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:32:44.359 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_blob_items": 0, 2023-11-27T15:32:44.359 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:32:44.359 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:32:44.359 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:32:44.359 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:32:44.359 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:32:44.360 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:32:44.360 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:32:44.360 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_txc_items": 0, 2023-11-27T15:32:44.360 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:32:44.360 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:32:44.360 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:32:44.360 INFO:teuthology.orchestra.run.smithi008.stdout: "bluestore_writing_items": 0, 2023-11-27T15:32:44.360 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_bytes": 0, 2023-11-27T15:32:44.361 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_items": 0, 2023-11-27T15:32:44.361 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:32:44.361 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:32:44.361 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:32:44.361 INFO:teuthology.orchestra.run.smithi008.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:32:44.361 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_bytes": 148967, 2023-11-27T15:32:44.361 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_anon_items": 29, 2023-11-27T15:32:44.361 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:32:44.362 INFO:teuthology.orchestra.run.smithi008.stdout: "buffer_meta_items": 0, 2023-11-27T15:32:44.362 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_bytes": 0, 2023-11-27T15:32:44.362 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_items": 0, 2023-11-27T15:32:44.362 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:32:44.362 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_mapbl_items": 0, 2023-11-27T15:32:44.362 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:32:44.362 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_pglog_items": 0, 2023-11-27T15:32:44.362 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_bytes": 0, 2023-11-27T15:32:44.363 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_items": 0, 2023-11-27T15:32:44.363 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:32:44.363 INFO:teuthology.orchestra.run.smithi008.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:32:44.363 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_bytes": 0, 2023-11-27T15:32:44.363 INFO:teuthology.orchestra.run.smithi008.stdout: "pgmap_items": 0, 2023-11-27T15:32:44.363 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_bytes": 928, 2023-11-27T15:32:44.363 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_co_items": 2, 2023-11-27T15:32:44.363 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_bytes": 0, 2023-11-27T15:32:44.364 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_1_items": 0, 2023-11-27T15:32:44.364 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_bytes": 0, 2023-11-27T15:32:44.364 INFO:teuthology.orchestra.run.smithi008.stdout: "unittest_2_items": 0 2023-11-27T15:32:44.364 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2023-11-27T15:32:44.364 INFO:teuthology.orchestra.run.smithi008.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:32:44.364 INFO:teuthology.orchestra.run.smithi008.stdout: "val": 0, 2023-11-27T15:32:44.364 INFO:teuthology.orchestra.run.smithi008.stdout: "max": 104857600, 2023-11-27T15:32:44.364 INFO:teuthology.orchestra.run.smithi008.stdout: "get_started": 0, 2023-11-27T15:32:44.365 INFO:teuthology.orchestra.run.smithi008.stdout: "get": 165, 2023-11-27T15:32:44.365 INFO:teuthology.orchestra.run.smithi008.stdout: "get_sum": 433004, 2023-11-27T15:32:44.365 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:44.365 INFO:teuthology.orchestra.run.smithi008.stdout: "get_or_fail_success": 165, 2023-11-27T15:32:44.365 INFO:teuthology.orchestra.run.smithi008.stdout: "take": 0, 2023-11-27T15:32:44.365 INFO:teuthology.orchestra.run.smithi008.stdout: "take_sum": 0, 2023-11-27T15:32:44.365 INFO:teuthology.orchestra.run.smithi008.stdout: "put": 165, 2023-11-27T15:32:44.365 INFO:teuthology.orchestra.run.smithi008.stdout: "put_sum": 433004, 2023-11-27T15:32:44.366 INFO:teuthology.orchestra.run.smithi008.stdout: "wait": { 2023-11-27T15:32:44.366 INFO:teuthology.orchestra.run.smithi008.stdout: "avgcount": 0, 2023-11-27T15:32:44.366 INFO:teuthology.orchestra.run.smithi008.stdout: "sum": 0.000000000, 2023-11-27T15:32:44.366 INFO:teuthology.orchestra.run.smithi008.stdout: "avgtime": 0.000000000 2023-11-27T15:32:44.366 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:44.366 INFO:teuthology.orchestra.run.smithi008.stdout: } 2023-11-27T15:32:44.366 INFO:teuthology.orchestra.run.smithi008.stdout:} 2023-11-27T15:32:44.944 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-11-27T15:32:44.944 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-11-27T15:32:44.944 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-11-27T15:32:44.944 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-11-27T15:32:44.944 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-11-27T15:32:44.944 DEBUG:tasks.check_counter:Getting stats from b 2023-11-27T15:32:44.945 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.b perf dump 2023-11-27T15:32:45.625 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:32:45.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:45 smithi060 ceph-mon[131518]: pgmap v567: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-27T15:32:45.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:45 smithi008 ceph-mon[128788]: pgmap v567: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-27T15:32:45.863 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:45 smithi029 ceph-mon[131166]: pgmap v567: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 682 B/s wr, 6 op/s 2023-11-27T15:32:46.984 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2023-11-27T15:32:46.984 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:32:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 34879, 2023-11-27T15:32:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 23712, 2023-11-27T15:32:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 45022996, 2023-11-27T15:32:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 99768153, 2023-11-27T15:32:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 11, 2023-11-27T15:32:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 5, 2023-11-27T15:32:46.985 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 2.304183627, 2023-11-27T15:32:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.856083028, 2023-11-27T15:32:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.951650511, 2023-11-27T15:32:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.647895264, 2023-11-27T15:32:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 23710, 2023-11-27T15:32:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 3.922176337, 2023-11-27T15:32:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000165422 2023-11-27T15:32:46.986 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 30705, 2023-11-27T15:32:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.004967099, 2023-11-27T15:32:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000161 2023-11-27T15:32:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 1536, 2023-11-27T15:32:46.987 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 1392 2023-11-27T15:32:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:32:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 25489, 2023-11-27T15:32:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 30628, 2023-11-27T15:32:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 27145920, 2023-11-27T15:32:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 71600732, 2023-11-27T15:32:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 7, 2023-11-27T15:32:46.988 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 5, 2023-11-27T15:32:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 1.972200515, 2023-11-27T15:32:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.693477100, 2023-11-27T15:32:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.764284404, 2023-11-27T15:32:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.537026877, 2023-11-27T15:32:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 30630, 2023-11-27T15:32:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 5.257619189, 2023-11-27T15:32:46.989 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000171649 2023-11-27T15:32:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 26493, 2023-11-27T15:32:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.005610063, 2023-11-27T15:32:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000211 2023-11-27T15:32:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 536656, 2023-11-27T15:32:46.990 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 94560 2023-11-27T15:32:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:32:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 101700, 2023-11-27T15:32:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 208071, 2023-11-27T15:32:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 40985313, 2023-11-27T15:32:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 187508176, 2023-11-27T15:32:46.991 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 13, 2023-11-27T15:32:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 10, 2023-11-27T15:32:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 8.362060745, 2023-11-27T15:32:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 3.820819370, 2023-11-27T15:32:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 6.217227037, 2023-11-27T15:32:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.578825105, 2023-11-27T15:32:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 208067, 2023-11-27T15:32:46.992 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 16.679964795, 2023-11-27T15:32:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000080166 2023-11-27T15:32:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 216359, 2023-11-27T15:32:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.048467477, 2023-11-27T15:32:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000224 2023-11-27T15:32:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.993 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 33230080, 2023-11-27T15:32:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 117040208 2023-11-27T15:32:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2023-11-27T15:32:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 1, 2023-11-27T15:32:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2023-11-27T15:32:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-MDSRank": { 2023-11-27T15:32:46.994 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2023-11-27T15:32:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2023-11-27T15:32:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 3633, 2023-11-27T15:32:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 99.192521524, 2023-11-27T15:32:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.027303198 2023-11-27T15:32:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-PurgeQueue": { 2023-11-27T15:32:46.995 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2023-11-27T15:32:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2023-11-27T15:32:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 32053, 2023-11-27T15:32:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 22.055277342, 2023-11-27T15:32:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000688087 2023-11-27T15:32:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": { 2023-11-27T15:32:46.996 INFO:teuthology.orchestra.run.smithi029.stdout: "request": 68662, 2023-11-27T15:32:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "reply": 60369, 2023-11-27T15:32:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "reply_latency": { 2023-11-27T15:32:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 60369, 2023-11-27T15:32:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 886.971374641, 2023-11-27T15:32:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.014692497 2023-11-27T15:32:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_reply": 0, 2023-11-27T15:32:46.997 INFO:teuthology.orchestra.run.smithi029.stdout: "forward": 8293, 2023-11-27T15:32:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_complete": 320, 2023-11-27T15:32:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_keys": 0, 2023-11-27T15:32:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_commit": 1103, 2023-11-27T15:32:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_split": 132, 2023-11-27T15:32:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_merge": 36, 2023-11-27T15:32:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes": 19, 2023-11-27T15:32:46.998 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_top": 2, 2023-11-27T15:32:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_bottom": 7, 2023-11-27T15:32:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pin_tail": 10, 2023-11-27T15:32:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pinned": 15, 2023-11-27T15:32:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_expired": 43787, 2023-11-27T15:32:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_with_caps": 1, 2023-11-27T15:32:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": 1, 2023-11-27T15:32:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "subtrees": 3, 2023-11-27T15:32:46.999 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse": 89560, 2023-11-27T15:32:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_hit": 80205, 2023-11-27T15:32:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_forward": 7425, 2023-11-27T15:32:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_discover": 7, 2023-11-27T15:32:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_dir_fetch": 311, 2023-11-27T15:32:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_remote_ino": 0, 2023-11-27T15:32:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_lock": 0, 2023-11-27T15:32:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "load_cent": 0, 2023-11-27T15:32:47.000 INFO:teuthology.orchestra.run.smithi029.stdout: "q": 0, 2023-11-27T15:32:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "exported": 797, 2023-11-27T15:32:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "exported_inodes": 32862, 2023-11-27T15:32:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "imported": 759, 2023-11-27T15:32:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "imported_inodes": 0, 2023-11-27T15:32:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_dir_fetch": 0, 2023-11-27T15:32:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:32:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_peer_discover": 0, 2023-11-27T15:32:47.001 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rfiles": 4, 2023-11-27T15:32:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rbytes": 246, 2023-11-27T15:32:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rsnaps": 0, 2023-11-27T15:32:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:32:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_set_tag": 0, 2023-11-27T15:32:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:32:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:32:47.002 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:32:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:32:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:32:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_file_inodes": 0, 2023-11-27T15:32:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_inode_file_caps": 4, 2023-11-27T15:32:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_revoke": 266, 2023-11-27T15:32:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_grant": 40133, 2023-11-27T15:32:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:32:47.003 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:32:47.004 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:32:47.004 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps": 14692, 2023-11-27T15:32:47.004 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps_dirty": 14386, 2023-11-27T15:32:47.004 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_cap_release": 407, 2023-11-27T15:32:47.004 INFO:teuthology.orchestra.run.smithi029.stdout: "process_request_cap_release": 39926 2023-11-27T15:32:47.004 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.004 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache": { 2023-11-27T15:32:47.004 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update": 609, 2023-11-27T15:32:47.005 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update_receipt": 700, 2023-11-27T15:32:47.005 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_try_discover": 352, 2023-11-27T15:32:47.005 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_send_discover": 355, 2023-11-27T15:32:47.005 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_handle_discover": 612, 2023-11-27T15:32:47.005 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays": 0, 2023-11-27T15:32:47.005 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_delayed": 0, 2023-11-27T15:32:47.005 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:32:47.005 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_created": 19665, 2023-11-27T15:32:47.006 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_enqueued": 19665, 2023-11-27T15:32:47.006 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_reintegrated": 0, 2023-11-27T15:32:47.006 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_migrated": 0, 2023-11-27T15:32:47.006 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_processing": 0, 2023-11-27T15:32:47.006 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:32:47.006 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:32:47.006 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_started": 0, 2023-11-27T15:32:47.006 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_completed": 0, 2023-11-27T15:32:47.007 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:32:47.007 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_exportdir": 833, 2023-11-27T15:32:47.007 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_flush": 0, 2023-11-27T15:32:47.007 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragmentdir": 170, 2023-11-27T15:32:47.007 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragstats": 0, 2023-11-27T15:32:47.007 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_inodestats": 0 2023-11-27T15:32:47.007 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.007 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log": { 2023-11-27T15:32:47.007 INFO:teuthology.orchestra.run.smithi029.stdout: "evadd": 86307, 2023-11-27T15:32:47.008 INFO:teuthology.orchestra.run.smithi029.stdout: "evex": 0, 2023-11-27T15:32:47.008 INFO:teuthology.orchestra.run.smithi029.stdout: "evtrm": 0, 2023-11-27T15:32:47.008 INFO:teuthology.orchestra.run.smithi029.stdout: "ev": 86307, 2023-11-27T15:32:47.008 INFO:teuthology.orchestra.run.smithi029.stdout: "evexg": 0, 2023-11-27T15:32:47.008 INFO:teuthology.orchestra.run.smithi029.stdout: "evexd": 0, 2023-11-27T15:32:47.008 INFO:teuthology.orchestra.run.smithi029.stdout: "segadd": 108, 2023-11-27T15:32:47.008 INFO:teuthology.orchestra.run.smithi029.stdout: "segex": 0, 2023-11-27T15:32:47.008 INFO:teuthology.orchestra.run.smithi029.stdout: "segtrm": 0, 2023-11-27T15:32:47.009 INFO:teuthology.orchestra.run.smithi029.stdout: "seg": 108, 2023-11-27T15:32:47.009 INFO:teuthology.orchestra.run.smithi029.stdout: "segexg": 0, 2023-11-27T15:32:47.009 INFO:teuthology.orchestra.run.smithi029.stdout: "segexd": 0, 2023-11-27T15:32:47.009 INFO:teuthology.orchestra.run.smithi029.stdout: "expos": 4194304, 2023-11-27T15:32:47.009 INFO:teuthology.orchestra.run.smithi029.stdout: "wrpos": 209516717, 2023-11-27T15:32:47.009 INFO:teuthology.orchestra.run.smithi029.stdout: "rdpos": 0, 2023-11-27T15:32:47.009 INFO:teuthology.orchestra.run.smithi029.stdout: "jlat": { 2023-11-27T15:32:47.009 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 2855, 2023-11-27T15:32:47.010 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 225.285038006, 2023-11-27T15:32:47.010 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.078908945 2023-11-27T15:32:47.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.010 INFO:teuthology.orchestra.run.smithi029.stdout: "replayed": 0 2023-11-27T15:32:47.010 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.010 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_mem": { 2023-11-27T15:32:47.010 INFO:teuthology.orchestra.run.smithi029.stdout: "ino": 24, 2023-11-27T15:32:47.011 INFO:teuthology.orchestra.run.smithi029.stdout: "ino+": 42090, 2023-11-27T15:32:47.011 INFO:teuthology.orchestra.run.smithi029.stdout: "ino-": 42066, 2023-11-27T15:32:47.011 INFO:teuthology.orchestra.run.smithi029.stdout: "dir": 21, 2023-11-27T15:32:47.011 INFO:teuthology.orchestra.run.smithi029.stdout: "dir+": 3127, 2023-11-27T15:32:47.011 INFO:teuthology.orchestra.run.smithi029.stdout: "dir-": 3106, 2023-11-27T15:32:47.011 INFO:teuthology.orchestra.run.smithi029.stdout: "dn": 19, 2023-11-27T15:32:47.011 INFO:teuthology.orchestra.run.smithi029.stdout: "dn+": 67680, 2023-11-27T15:32:47.011 INFO:teuthology.orchestra.run.smithi029.stdout: "dn-": 67661, 2023-11-27T15:32:47.012 INFO:teuthology.orchestra.run.smithi029.stdout: "cap": 1, 2023-11-27T15:32:47.012 INFO:teuthology.orchestra.run.smithi029.stdout: "cap+": 57646, 2023-11-27T15:32:47.012 INFO:teuthology.orchestra.run.smithi029.stdout: "cap-": 57645, 2023-11-27T15:32:47.012 INFO:teuthology.orchestra.run.smithi029.stdout: "rss": 161048, 2023-11-27T15:32:47.012 INFO:teuthology.orchestra.run.smithi029.stdout: "heap": 198964 2023-11-27T15:32:47.012 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.012 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_server": { 2023-11-27T15:32:47.012 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_client_request": 71292, 2023-11-27T15:32:47.013 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_server_request": 32, 2023-11-27T15:32:47.013 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_request": 68662, 2023-11-27T15:32:47.013 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_session": 143, 2023-11-27T15:32:47.013 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_peer_request": 336, 2023-11-27T15:32:47.013 INFO:teuthology.orchestra.run.smithi029.stdout: "req_create_latency": { 2023-11-27T15:32:47.013 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 20435, 2023-11-27T15:32:47.013 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 688.034336106, 2023-11-27T15:32:47.013 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.033669407 2023-11-27T15:32:47.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.014 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getattr_latency": { 2023-11-27T15:32:47.014 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 82, 2023-11-27T15:32:47.014 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.165925110, 2023-11-27T15:32:47.014 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.002023476 2023-11-27T15:32:47.014 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.014 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getfilelock_latency": { 2023-11-27T15:32:47.014 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.015 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.015 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.015 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.015 INFO:teuthology.orchestra.run.smithi029.stdout: "req_link_latency": { 2023-11-27T15:32:47.015 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.015 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.015 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.016 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookup_latency": { 2023-11-27T15:32:47.016 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 19397, 2023-11-27T15:32:47.016 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 69.556991758, 2023-11-27T15:32:47.016 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.003585966 2023-11-27T15:32:47.016 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.016 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookuphash_latency": { 2023-11-27T15:32:47.016 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.017 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.017 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.017 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupino_latency": { 2023-11-27T15:32:47.017 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1, 2023-11-27T15:32:47.017 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000175976, 2023-11-27T15:32:47.017 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000175976 2023-11-27T15:32:47.017 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.018 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupname_latency": { 2023-11-27T15:32:47.018 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.018 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.018 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.018 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.018 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupparent_latency": { 2023-11-27T15:32:47.018 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.018 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.019 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.019 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupsnap_latency": { 2023-11-27T15:32:47.019 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.019 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.019 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.019 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.019 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lssnap_latency": { 2023-11-27T15:32:47.020 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.020 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.020 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.020 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.020 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mkdir_latency": { 2023-11-27T15:32:47.020 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.020 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.021 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.021 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mknod_latency": { 2023-11-27T15:32:47.021 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.021 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.021 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.021 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.021 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mksnap_latency": { 2023-11-27T15:32:47.022 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.022 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.022 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.022 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.022 INFO:teuthology.orchestra.run.smithi029.stdout: "req_open_latency": { 2023-11-27T15:32:47.022 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.022 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.022 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.023 INFO:teuthology.orchestra.run.smithi029.stdout: "req_readdir_latency": { 2023-11-27T15:32:47.023 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 789, 2023-11-27T15:32:47.023 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 2.917241047, 2023-11-27T15:32:47.023 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.003697390 2023-11-27T15:32:47.023 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.023 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rename_latency": { 2023-11-27T15:32:47.024 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.024 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.024 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.024 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.024 INFO:teuthology.orchestra.run.smithi029.stdout: "req_renamesnap_latency": { 2023-11-27T15:32:47.024 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.024 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.024 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.025 INFO:teuthology.orchestra.run.smithi029.stdout: "req_snapdiff_latency": { 2023-11-27T15:32:47.025 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.025 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.025 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.025 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.025 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmdir_latency": { 2023-11-27T15:32:47.025 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1, 2023-11-27T15:32:47.026 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.058203168, 2023-11-27T15:32:47.026 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.058203168 2023-11-27T15:32:47.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.026 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmsnap_latency": { 2023-11-27T15:32:47.026 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.026 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.026 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.026 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.027 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmxattr_latency": { 2023-11-27T15:32:47.027 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.027 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.027 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.027 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.027 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setattr_latency": { 2023-11-27T15:32:47.027 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.027 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.028 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.028 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setdirlayout_latency": { 2023-11-27T15:32:47.028 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.028 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.028 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.028 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.029 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setfilelock_latency": { 2023-11-27T15:32:47.029 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.029 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.029 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.029 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.029 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setlayout_latency": { 2023-11-27T15:32:47.029 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.029 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.030 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.030 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setxattr_latency": { 2023-11-27T15:32:47.030 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.030 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.030 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.030 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.030 INFO:teuthology.orchestra.run.smithi029.stdout: "req_symlink_latency": { 2023-11-27T15:32:47.031 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.031 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.031 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.031 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.031 INFO:teuthology.orchestra.run.smithi029.stdout: "req_unlink_latency": { 2023-11-27T15:32:47.031 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 19664, 2023-11-27T15:32:47.031 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 126.238501476, 2023-11-27T15:32:47.031 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.006419777 2023-11-27T15:32:47.032 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.032 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:32:47.032 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:32:47.032 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getvxattr_latency": { 2023-11-27T15:32:47.032 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.032 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.032 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.032 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:47.033 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.033 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sessions": { 2023-11-27T15:32:47.033 INFO:teuthology.orchestra.run.smithi029.stdout: "session_count": 1, 2023-11-27T15:32:47.033 INFO:teuthology.orchestra.run.smithi029.stdout: "session_add": 2, 2023-11-27T15:32:47.033 INFO:teuthology.orchestra.run.smithi029.stdout: "session_remove": 1, 2023-11-27T15:32:47.033 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_open": 1, 2023-11-27T15:32:47.033 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_stale": 0, 2023-11-27T15:32:47.034 INFO:teuthology.orchestra.run.smithi029.stdout: "total_load": 19853, 2023-11-27T15:32:47.034 INFO:teuthology.orchestra.run.smithi029.stdout: "average_load": 19853, 2023-11-27T15:32:47.034 INFO:teuthology.orchestra.run.smithi029.stdout: "avg_session_uptime": 424 2023-11-27T15:32:47.034 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.034 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2023-11-27T15:32:47.034 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:32:47.034 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2023-11-27T15:32:47.035 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:32:47.035 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:32:47.035 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:32:47.035 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:32:47.035 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:32:47.035 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:32:47.035 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:32:47.035 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:32:47.036 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:32:47.036 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:32:47.036 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:32:47.036 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:32:47.036 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:32:47.036 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2023-11-27T15:32:47.036 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:32:47.036 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2023-11-27T15:32:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:32:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:32:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:32:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:32:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:32:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:32:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:32:47.037 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2023-11-27T15:32:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:32:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:32:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:32:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2023-11-27T15:32:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2023-11-27T15:32:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2023-11-27T15:32:47.038 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:32:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:32:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:32:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:32:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 283995, 2023-11-27T15:32:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 120, 2023-11-27T15:32:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 440, 2023-11-27T15:32:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 5, 2023-11-27T15:32:47.039 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2023-11-27T15:32:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2023-11-27T15:32:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:32:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2023-11-27T15:32:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:32:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2023-11-27T15:32:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 8792, 2023-11-27T15:32:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 168, 2023-11-27T15:32:47.040 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:32:47.041 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:32:47.041 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2023-11-27T15:32:47.041 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2023-11-27T15:32:47.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 95687, 2023-11-27T15:32:47.041 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 549, 2023-11-27T15:32:47.041 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2023-11-27T15:32:47.041 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2023-11-27T15:32:47.041 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2023-11-27T15:32:47.042 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2023-11-27T15:32:47.042 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.042 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter": { 2023-11-27T15:32:47.042 INFO:teuthology.orchestra.run.smithi029.stdout: "op_active": 0, 2023-11-27T15:32:47.042 INFO:teuthology.orchestra.run.smithi029.stdout: "op_laggy": 0, 2023-11-27T15:32:47.042 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send": 45356, 2023-11-27T15:32:47.042 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send_bytes": 219990498, 2023-11-27T15:32:47.042 INFO:teuthology.orchestra.run.smithi029.stdout: "op_resend": 27, 2023-11-27T15:32:47.043 INFO:teuthology.orchestra.run.smithi029.stdout: "op_reply": 45356, 2023-11-27T15:32:47.043 INFO:teuthology.orchestra.run.smithi029.stdout: "op_latency": { 2023-11-27T15:32:47.043 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 45356, 2023-11-27T15:32:47.043 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 447.912398248, 2023-11-27T15:32:47.043 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.009875482 2023-11-27T15:32:47.043 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.043 INFO:teuthology.orchestra.run.smithi029.stdout: "op_inflight": 0, 2023-11-27T15:32:47.044 INFO:teuthology.orchestra.run.smithi029.stdout: "oplen_avg": { 2023-11-27T15:32:47.044 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 45356, 2023-11-27T15:32:47.044 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 46848 2023-11-27T15:32:47.044 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.044 INFO:teuthology.orchestra.run.smithi029.stdout: "op": 45356, 2023-11-27T15:32:47.044 INFO:teuthology.orchestra.run.smithi029.stdout: "op_r": 718, 2023-11-27T15:32:47.044 INFO:teuthology.orchestra.run.smithi029.stdout: "op_w": 44638, 2023-11-27T15:32:47.044 INFO:teuthology.orchestra.run.smithi029.stdout: "op_rmw": 0, 2023-11-27T15:32:47.045 INFO:teuthology.orchestra.run.smithi029.stdout: "op_pg": 0, 2023-11-27T15:32:47.045 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_stat": 0, 2023-11-27T15:32:47.045 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_create": 10, 2023-11-27T15:32:47.045 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_read": 398, 2023-11-27T15:32:47.045 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_write": 3253, 2023-11-27T15:32:47.045 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writefull": 152, 2023-11-27T15:32:47.045 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writesame": 0, 2023-11-27T15:32:47.045 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_append": 0, 2023-11-27T15:32:47.046 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_zero": 0, 2023-11-27T15:32:47.046 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_truncate": 0, 2023-11-27T15:32:47.046 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_delete": 40069, 2023-11-27T15:32:47.046 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_mapext": 0, 2023-11-27T15:32:47.046 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_sparse_read": 0, 2023-11-27T15:32:47.046 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_clonerange": 0, 2023-11-27T15:32:47.046 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_getxattr": 0, 2023-11-27T15:32:47.046 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_setxattr": 20, 2023-11-27T15:32:47.047 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:32:47.047 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_rmxattr": 0, 2023-11-27T15:32:47.047 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:32:47.047 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_call": 0, 2023-11-27T15:32:47.047 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_watch": 0, 2023-11-27T15:32:47.047 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_notify": 0, 2023-11-27T15:32:47.047 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:32:47.048 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls": 0, 2023-11-27T15:32:47.048 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:32:47.048 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_other": 0, 2023-11-27T15:32:47.048 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_active": 0, 2023-11-27T15:32:47.048 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_send": 0, 2023-11-27T15:32:47.048 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_resend": 0, 2023-11-27T15:32:47.048 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_ping": 0, 2023-11-27T15:32:47.048 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_active": 0, 2023-11-27T15:32:47.049 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_send": 0, 2023-11-27T15:32:47.049 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_resend": 0, 2023-11-27T15:32:47.049 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_active": 0, 2023-11-27T15:32:47.049 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_send": 0, 2023-11-27T15:32:47.049 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_resend": 0, 2023-11-27T15:32:47.049 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_active": 0, 2023-11-27T15:32:47.049 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_send": 0, 2023-11-27T15:32:47.050 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_resend": 0, 2023-11-27T15:32:47.050 INFO:teuthology.orchestra.run.smithi029.stdout: "command_active": 0, 2023-11-27T15:32:47.050 INFO:teuthology.orchestra.run.smithi029.stdout: "command_send": 0, 2023-11-27T15:32:47.050 INFO:teuthology.orchestra.run.smithi029.stdout: "command_resend": 0, 2023-11-27T15:32:47.050 INFO:teuthology.orchestra.run.smithi029.stdout: "map_epoch": 94, 2023-11-27T15:32:47.050 INFO:teuthology.orchestra.run.smithi029.stdout: "map_full": 0, 2023-11-27T15:32:47.050 INFO:teuthology.orchestra.run.smithi029.stdout: "map_inc": 6, 2023-11-27T15:32:47.050 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_sessions": 12, 2023-11-27T15:32:47.051 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_open": 12, 2023-11-27T15:32:47.051 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_close": 0, 2023-11-27T15:32:47.051 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_laggy": 0, 2023-11-27T15:32:47.051 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_wr": 2261, 2023-11-27T15:32:47.051 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_rd": 640, 2023-11-27T15:32:47.051 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_del": 45 2023-11-27T15:32:47.051 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.051 INFO:teuthology.orchestra.run.smithi029.stdout: "oft": { 2023-11-27T15:32:47.052 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_objs": 1, 2023-11-27T15:32:47.052 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:32:47.052 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_updates": 0, 2023-11-27T15:32:47.052 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_removes": 0 2023-11-27T15:32:47.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.052 INFO:teuthology.orchestra.run.smithi029.stdout: "osd-slow-ops": { 2023-11-27T15:32:47.052 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_ops_count": 69696 2023-11-27T15:32:47.052 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.053 INFO:teuthology.orchestra.run.smithi029.stdout: "purge_queue": { 2023-11-27T15:32:47.053 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops": 0, 2023-11-27T15:32:47.053 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops_high_water": 16, 2023-11-27T15:32:47.053 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing": 0, 2023-11-27T15:32:47.053 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_high_water": 8, 2023-11-27T15:32:47.053 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed": 19665, 2023-11-27T15:32:47.053 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_item_in_journal": 0 2023-11-27T15:32:47.054 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.054 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:32:47.054 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:47.054 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2023-11-27T15:32:47.054 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:47.054 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 162064, 2023-11-27T15:32:47.054 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 94450727, 2023-11-27T15:32:47.054 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:47.055 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 162064, 2023-11-27T15:32:47.055 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2023-11-27T15:32:47.055 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2023-11-27T15:32:47.055 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 162064, 2023-11-27T15:32:47.055 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 94450727, 2023-11-27T15:32:47.055 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:47.055 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.055 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.056 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.056 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:47.056 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.056 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_bytes": { 2023-11-27T15:32:47.056 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:47.056 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2023-11-27T15:32:47.056 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:47.056 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2023-11-27T15:32:47.057 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2023-11-27T15:32:47.057 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:47.057 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:47.057 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 45356, 2023-11-27T15:32:47.057 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 222128927, 2023-11-27T15:32:47.057 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 45356, 2023-11-27T15:32:47.057 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 222128927, 2023-11-27T15:32:47.057 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:47.058 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.058 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.058 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.058 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:47.058 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.058 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_ops": { 2023-11-27T15:32:47.058 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:47.058 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 1024, 2023-11-27T15:32:47.059 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:47.059 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2023-11-27T15:32:47.059 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2023-11-27T15:32:47.059 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:47.059 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:47.059 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 45356, 2023-11-27T15:32:47.059 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 45356, 2023-11-27T15:32:47.060 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 45356, 2023-11-27T15:32:47.060 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 45356, 2023-11-27T15:32:47.060 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:47.060 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.060 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.060 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.060 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:47.060 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.061 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:32:47.061 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:47.061 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2023-11-27T15:32:47.061 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:47.061 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 19665, 2023-11-27T15:32:47.061 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 2143549, 2023-11-27T15:32:47.061 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:47.061 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 19665, 2023-11-27T15:32:47.062 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2023-11-27T15:32:47.062 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2023-11-27T15:32:47.062 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 398, 2023-11-27T15:32:47.062 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 2143549, 2023-11-27T15:32:47.062 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:47.062 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.062 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.062 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.063 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:47.063 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:47.063 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle-0x556179fabcc0": { 2023-11-27T15:32:47.063 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:47.063 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2023-11-27T15:32:47.063 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:47.063 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 86307, 2023-11-27T15:32:47.063 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 205322413, 2023-11-27T15:32:47.064 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:47.064 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 86307, 2023-11-27T15:32:47.064 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2023-11-27T15:32:47.064 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2023-11-27T15:32:47.064 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 2855, 2023-11-27T15:32:47.064 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 205322413, 2023-11-27T15:32:47.064 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:47.064 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:47.065 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:47.065 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:47.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:47.065 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:47.065 INFO:teuthology.orchestra.run.smithi029.stdout:} 2023-11-27T15:32:47.483 INFO:tasks.check_counter:Daemon mds.b mds.exported=797 2023-11-27T15:32:47.484 INFO:tasks.check_counter:Daemon mds.b mds.imported=759 2023-11-27T15:32:47.484 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=609 2023-11-27T15:32:47.484 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=700 2023-11-27T15:32:47.484 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=132 2023-11-27T15:32:47.484 DEBUG:tasks.check_counter:Getting stats from e 2023-11-27T15:32:47.485 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.e perf dump 2023-11-27T15:32:47.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:47 smithi060 ceph-mon[131518]: pgmap v568: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2023-11-27T15:32:47.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:47 smithi008 ceph-mon[128788]: pgmap v568: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2023-11-27T15:32:47.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:47 smithi029 ceph-mon[131166]: pgmap v568: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 6 op/s 2023-11-27T15:32:48.139 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:32:49.510 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2023-11-27T15:32:49.510 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:32:49.510 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 3, 2023-11-27T15:32:49.510 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 4, 2023-11-27T15:32:49.511 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 2656, 2023-11-27T15:32:49.511 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 448, 2023-11-27T15:32:49.511 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 4, 2023-11-27T15:32:49.511 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 0, 2023-11-27T15:32:49.511 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.002640880, 2023-11-27T15:32:49.511 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.000052097, 2023-11-27T15:32:49.511 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.000766155, 2023-11-27T15:32:49.512 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:32:49.512 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:49.512 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 3, 2023-11-27T15:32:49.512 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.002241443, 2023-11-27T15:32:49.512 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000747147 2023-11-27T15:32:49.512 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.512 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:49.512 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:49.513 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:49.513 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:49.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.513 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 2656, 2023-11-27T15:32:49.513 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 448 2023-11-27T15:32:49.513 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.513 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:32:49.513 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 1, 2023-11-27T15:32:49.514 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 21, 2023-11-27T15:32:49.514 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 160, 2023-11-27T15:32:49.514 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 79456, 2023-11-27T15:32:49.514 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 3, 2023-11-27T15:32:49.514 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 2, 2023-11-27T15:32:49.514 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.002277832, 2023-11-27T15:32:49.514 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.000839857, 2023-11-27T15:32:49.515 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.001122735, 2023-11-27T15:32:49.515 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:32:49.515 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:49.515 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 19, 2023-11-27T15:32:49.515 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.001079653, 2023-11-27T15:32:49.515 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000056823 2023-11-27T15:32:49.515 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.515 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:49.516 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:49.516 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:49.516 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:49.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.516 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:32:49.516 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 79456 2023-11-27T15:32:49.516 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.517 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:32:49.517 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 31, 2023-11-27T15:32:49.517 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 30, 2023-11-27T15:32:49.517 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 146688, 2023-11-27T15:32:49.517 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 15360, 2023-11-27T15:32:49.517 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 2, 2023-11-27T15:32:49.517 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 1, 2023-11-27T15:32:49.518 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.006177844, 2023-11-27T15:32:49.518 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.001183882, 2023-11-27T15:32:49.518 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.005023970, 2023-11-27T15:32:49.518 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000930454, 2023-11-27T15:32:49.518 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:49.518 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 29, 2023-11-27T15:32:49.518 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.006375575, 2023-11-27T15:32:49.519 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000219847 2023-11-27T15:32:49.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.519 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:49.519 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:49.519 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:49.519 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:49.519 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.519 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 146688, 2023-11-27T15:32:49.520 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 15360 2023-11-27T15:32:49.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.520 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2023-11-27T15:32:49.520 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 0, 2023-11-27T15:32:49.520 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2023-11-27T15:32:49.520 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.520 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2023-11-27T15:32:49.520 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:32:49.521 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2023-11-27T15:32:49.521 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:32:49.521 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:32:49.521 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:32:49.521 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:32:49.521 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:32:49.521 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:32:49.521 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:32:49.522 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:32:49.522 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:32:49.522 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:32:49.522 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:32:49.522 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:32:49.522 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:32:49.522 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2023-11-27T15:32:49.523 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:32:49.523 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2023-11-27T15:32:49.523 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:32:49.523 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:32:49.523 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:32:49.523 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:32:49.523 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:32:49.523 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:32:49.524 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:32:49.524 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2023-11-27T15:32:49.524 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:32:49.524 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:32:49.524 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:32:49.524 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2023-11-27T15:32:49.524 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2023-11-27T15:32:49.524 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2023-11-27T15:32:49.525 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:32:49.525 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:32:49.525 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:32:49.525 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:32:49.525 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 148007, 2023-11-27T15:32:49.525 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 24, 2023-11-27T15:32:49.525 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:32:49.526 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 0, 2023-11-27T15:32:49.526 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2023-11-27T15:32:49.526 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2023-11-27T15:32:49.526 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:32:49.526 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2023-11-27T15:32:49.526 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:32:49.526 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2023-11-27T15:32:49.526 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 0, 2023-11-27T15:32:49.527 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 0, 2023-11-27T15:32:49.527 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:32:49.527 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:32:49.527 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2023-11-27T15:32:49.527 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2023-11-27T15:32:49.527 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 928, 2023-11-27T15:32:49.527 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 2, 2023-11-27T15:32:49.528 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2023-11-27T15:32:49.528 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2023-11-27T15:32:49.528 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2023-11-27T15:32:49.528 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2023-11-27T15:32:49.528 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:49.528 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:32:49.528 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:49.528 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2023-11-27T15:32:49.529 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:49.529 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 32, 2023-11-27T15:32:49.529 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 142353, 2023-11-27T15:32:49.529 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:49.529 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 32, 2023-11-27T15:32:49.529 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2023-11-27T15:32:49.529 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2023-11-27T15:32:49.529 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 32, 2023-11-27T15:32:49.530 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 142353, 2023-11-27T15:32:49.530 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:49.530 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:49.530 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:49.530 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:49.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:49.530 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:49.531 INFO:teuthology.orchestra.run.smithi029.stdout:} 2023-11-27T15:32:49.533 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:49 smithi029 ceph-mon[131166]: pgmap v569: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:32:49.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:49 smithi060 ceph-mon[131518]: pgmap v569: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:32:49.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:49 smithi008 ceph-mon[128788]: pgmap v569: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:32:49.904 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2023-11-27T15:32:49.905 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2023-11-27T15:32:49.905 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2023-11-27T15:32:49.905 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2023-11-27T15:32:49.905 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2023-11-27T15:32:49.905 DEBUG:tasks.check_counter:Getting stats from h 2023-11-27T15:32:49.906 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.h perf dump 2023-11-27T15:32:50.558 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:32:51.817 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2023-11-27T15:32:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:32:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 87372, 2023-11-27T15:32:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 159802, 2023-11-27T15:32:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 33534204, 2023-11-27T15:32:51.818 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 136457800, 2023-11-27T15:32:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 12, 2023-11-27T15:32:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 6, 2023-11-27T15:32:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 6.974814891, 2023-11-27T15:32:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 3.162205914, 2023-11-27T15:32:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 4.954231800, 2023-11-27T15:32:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.471695518, 2023-11-27T15:32:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:51.819 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 159801, 2023-11-27T15:32:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 15.577005833, 2023-11-27T15:32:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000097477 2023-11-27T15:32:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 177301, 2023-11-27T15:32:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.035046505, 2023-11-27T15:32:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000197 2023-11-27T15:32:51.820 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 30339584, 2023-11-27T15:32:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 99410384 2023-11-27T15:32:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:32:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 23462, 2023-11-27T15:32:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 34750, 2023-11-27T15:32:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 21316022, 2023-11-27T15:32:51.821 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 94654827, 2023-11-27T15:32:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 8, 2023-11-27T15:32:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 6, 2023-11-27T15:32:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 2.114874721, 2023-11-27T15:32:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.851963586, 2023-11-27T15:32:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.733180999, 2023-11-27T15:32:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.502156973, 2023-11-27T15:32:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:51.822 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 34748, 2023-11-27T15:32:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 5.191879312, 2023-11-27T15:32:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000149415 2023-11-27T15:32:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 29847, 2023-11-27T15:32:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.006804824, 2023-11-27T15:32:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000227 2023-11-27T15:32:51.823 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:32:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 271744 2023-11-27T15:32:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:32:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 23325, 2023-11-27T15:32:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 22334, 2023-11-27T15:32:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 12229080, 2023-11-27T15:32:51.824 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 104985100, 2023-11-27T15:32:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 11, 2023-11-27T15:32:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 6, 2023-11-27T15:32:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 1.974938199, 2023-11-27T15:32:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.600928072, 2023-11-27T15:32:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.510411163, 2023-11-27T15:32:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.828260610, 2023-11-27T15:32:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 22333, 2023-11-27T15:32:51.825 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 3.409164580, 2023-11-27T15:32:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000152651 2023-11-27T15:32:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 10218, 2023-11-27T15:32:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.002053473, 2023-11-27T15:32:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000200 2023-11-27T15:32:51.826 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-27T15:32:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 448 2023-11-27T15:32:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2023-11-27T15:32:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 1, 2023-11-27T15:32:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2023-11-27T15:32:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.827 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-MDSRank": { 2023-11-27T15:32:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2023-11-27T15:32:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2023-11-27T15:32:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1540, 2023-11-27T15:32:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 109.227973657, 2023-11-27T15:32:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.070927255 2023-11-27T15:32:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.828 INFO:teuthology.orchestra.run.smithi029.stdout: "finisher-PurgeQueue": { 2023-11-27T15:32:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "queue_len": 0, 2023-11-27T15:32:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "complete_latency": { 2023-11-27T15:32:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 33003, 2023-11-27T15:32:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 26.107692359, 2023-11-27T15:32:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000791070 2023-11-27T15:32:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:51.829 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "mds": { 2023-11-27T15:32:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "request": 54995, 2023-11-27T15:32:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "reply": 48052, 2023-11-27T15:32:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "reply_latency": { 2023-11-27T15:32:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 48052, 2023-11-27T15:32:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 276.383883889, 2023-11-27T15:32:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.005751766 2023-11-27T15:32:51.830 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_reply": 0, 2023-11-27T15:32:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "forward": 6943, 2023-11-27T15:32:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_complete": 0, 2023-11-27T15:32:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_fetch_keys": 0, 2023-11-27T15:32:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_commit": 881, 2023-11-27T15:32:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_split": 100, 2023-11-27T15:32:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_merge": 70, 2023-11-27T15:32:51.831 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes": 20, 2023-11-27T15:32:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_top": 2, 2023-11-27T15:32:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_bottom": 8, 2023-11-27T15:32:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pin_tail": 10, 2023-11-27T15:32:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_pinned": 15, 2023-11-27T15:32:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_expired": 3448, 2023-11-27T15:32:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "inodes_with_caps": 0, 2023-11-27T15:32:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "caps": 0, 2023-11-27T15:32:51.832 INFO:teuthology.orchestra.run.smithi029.stdout: "subtrees": 4, 2023-11-27T15:32:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse": 73952, 2023-11-27T15:32:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_hit": 66778, 2023-11-27T15:32:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_forward": 6790, 2023-11-27T15:32:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_discover": 9, 2023-11-27T15:32:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:32:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_remote_ino": 0, 2023-11-27T15:32:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "traverse_lock": 0, 2023-11-27T15:32:51.833 INFO:teuthology.orchestra.run.smithi029.stdout: "load_cent": 0, 2023-11-27T15:32:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "q": 0, 2023-11-27T15:32:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "exported": 161, 2023-11-27T15:32:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "exported_inodes": 5063, 2023-11-27T15:32:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "imported": 142, 2023-11-27T15:32:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "imported_inodes": 0, 2023-11-27T15:32:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_dir_fetch": 0, 2023-11-27T15:32:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:32:51.834 INFO:teuthology.orchestra.run.smithi029.stdout: "openino_peer_discover": 0, 2023-11-27T15:32:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rfiles": 4, 2023-11-27T15:32:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rbytes": 246, 2023-11-27T15:32:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "root_rsnaps": 0, 2023-11-27T15:32:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:32:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_set_tag": 0, 2023-11-27T15:32:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:32:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:32:51.835 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:32:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:32:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:32:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "scrub_file_inodes": 0, 2023-11-27T15:32:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_inode_file_caps": 6, 2023-11-27T15:32:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_revoke": 84, 2023-11-27T15:32:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_grant": 29897, 2023-11-27T15:32:51.836 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:32:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:32:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:32:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps": 19050, 2023-11-27T15:32:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_caps_dirty": 18882, 2023-11-27T15:32:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_cap_release": 376, 2023-11-27T15:32:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "process_request_cap_release": 38391 2023-11-27T15:32:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_cache": { 2023-11-27T15:32:51.837 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update": 0, 2023-11-27T15:32:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_update_receipt": 2761, 2023-11-27T15:32:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_try_discover": 1383, 2023-11-27T15:32:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_send_discover": 1604, 2023-11-27T15:32:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "dir_handle_discover": 1, 2023-11-27T15:32:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays": 0, 2023-11-27T15:32:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_delayed": 0, 2023-11-27T15:32:51.838 INFO:teuthology.orchestra.run.smithi029.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:32:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_created": 19897, 2023-11-27T15:32:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_enqueued": 19897, 2023-11-27T15:32:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_reintegrated": 0, 2023-11-27T15:32:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "strays_migrated": 0, 2023-11-27T15:32:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_processing": 0, 2023-11-27T15:32:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:32:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:32:51.839 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_started": 0, 2023-11-27T15:32:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "recovery_completed": 0, 2023-11-27T15:32:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:32:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_exportdir": 161, 2023-11-27T15:32:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_flush": 0, 2023-11-27T15:32:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragmentdir": 172, 2023-11-27T15:32:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_fragstats": 0, 2023-11-27T15:32:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: "ireq_inodestats": 0 2023-11-27T15:32:51.840 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_log": { 2023-11-27T15:32:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "evadd": 72305, 2023-11-27T15:32:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "evex": 0, 2023-11-27T15:32:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "evtrm": 0, 2023-11-27T15:32:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "ev": 72305, 2023-11-27T15:32:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "evexg": 0, 2023-11-27T15:32:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "evexd": 0, 2023-11-27T15:32:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "segadd": 88, 2023-11-27T15:32:51.841 INFO:teuthology.orchestra.run.smithi029.stdout: "segex": 0, 2023-11-27T15:32:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "segtrm": 0, 2023-11-27T15:32:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "seg": 88, 2023-11-27T15:32:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "segexg": 0, 2023-11-27T15:32:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "segexd": 0, 2023-11-27T15:32:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "expos": 4194304, 2023-11-27T15:32:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "wrpos": 177838144, 2023-11-27T15:32:51.842 INFO:teuthology.orchestra.run.smithi029.stdout: "rdpos": 0, 2023-11-27T15:32:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "jlat": { 2023-11-27T15:32:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 893, 2023-11-27T15:32:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 122.496293315, 2023-11-27T15:32:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.137173900 2023-11-27T15:32:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "replayed": 0 2023-11-27T15:32:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.843 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_mem": { 2023-11-27T15:32:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "ino": 25, 2023-11-27T15:32:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "ino+": 22673, 2023-11-27T15:32:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "ino-": 22648, 2023-11-27T15:32:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "dir": 23, 2023-11-27T15:32:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "dir+": 2919, 2023-11-27T15:32:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "dir-": 2896, 2023-11-27T15:32:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "dn": 20, 2023-11-27T15:32:51.844 INFO:teuthology.orchestra.run.smithi029.stdout: "dn+": 43239, 2023-11-27T15:32:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "dn-": 43219, 2023-11-27T15:32:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "cap": 0, 2023-11-27T15:32:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "cap+": 51139, 2023-11-27T15:32:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "cap-": 51139, 2023-11-27T15:32:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "rss": 157396, 2023-11-27T15:32:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "heap": 198964 2023-11-27T15:32:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_server": { 2023-11-27T15:32:51.845 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_client_request": 56315, 2023-11-27T15:32:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "dispatch_server_request": 15, 2023-11-27T15:32:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_request": 54995, 2023-11-27T15:32:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_client_session": 150, 2023-11-27T15:32:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "handle_peer_request": 72, 2023-11-27T15:32:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "req_create_latency": { 2023-11-27T15:32:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 9813, 2023-11-27T15:32:51.846 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 91.404780817, 2023-11-27T15:32:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.009314662 2023-11-27T15:32:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getattr_latency": { 2023-11-27T15:32:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 55, 2023-11-27T15:32:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.343112130, 2023-11-27T15:32:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.006238402 2023-11-27T15:32:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.847 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getfilelock_latency": { 2023-11-27T15:32:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "req_link_latency": { 2023-11-27T15:32:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.848 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookup_latency": { 2023-11-27T15:32:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 17218, 2023-11-27T15:32:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 48.361079067, 2023-11-27T15:32:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.002808751 2023-11-27T15:32:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookuphash_latency": { 2023-11-27T15:32:51.849 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupino_latency": { 2023-11-27T15:32:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.850 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupname_latency": { 2023-11-27T15:32:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupparent_latency": { 2023-11-27T15:32:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.851 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lookupsnap_latency": { 2023-11-27T15:32:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.852 INFO:teuthology.orchestra.run.smithi029.stdout: "req_lssnap_latency": { 2023-11-27T15:32:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mkdir_latency": { 2023-11-27T15:32:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.853 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mknod_latency": { 2023-11-27T15:32:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.854 INFO:teuthology.orchestra.run.smithi029.stdout: "req_mksnap_latency": { 2023-11-27T15:32:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "req_open_latency": { 2023-11-27T15:32:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.855 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "req_readdir_latency": { 2023-11-27T15:32:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 1069, 2023-11-27T15:32:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 16.514886577, 2023-11-27T15:32:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.015448911 2023-11-27T15:32:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rename_latency": { 2023-11-27T15:32:51.856 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "req_renamesnap_latency": { 2023-11-27T15:32:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.857 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "req_snapdiff_latency": { 2023-11-27T15:32:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmdir_latency": { 2023-11-27T15:32:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 2, 2023-11-27T15:32:51.858 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.167671550, 2023-11-27T15:32:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.083835775 2023-11-27T15:32:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmsnap_latency": { 2023-11-27T15:32:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.859 INFO:teuthology.orchestra.run.smithi029.stdout: "req_rmxattr_latency": { 2023-11-27T15:32:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setattr_latency": { 2023-11-27T15:32:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.860 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setdirlayout_latency": { 2023-11-27T15:32:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.861 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setfilelock_latency": { 2023-11-27T15:32:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setlayout_latency": { 2023-11-27T15:32:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.862 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "req_setxattr_latency": { 2023-11-27T15:32:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.863 INFO:teuthology.orchestra.run.smithi029.stdout: "req_symlink_latency": { 2023-11-27T15:32:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "req_unlink_latency": { 2023-11-27T15:32:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 19895, 2023-11-27T15:32:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 119.592353748, 2023-11-27T15:32:51.864 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.006011176 2023-11-27T15:32:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:32:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:32:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "req_getvxattr_latency": { 2023-11-27T15:32:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.865 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_sessions": { 2023-11-27T15:32:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "session_count": 1, 2023-11-27T15:32:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "session_add": 2, 2023-11-27T15:32:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "session_remove": 1, 2023-11-27T15:32:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_open": 1, 2023-11-27T15:32:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "sessions_stale": 0, 2023-11-27T15:32:51.866 INFO:teuthology.orchestra.run.smithi029.stdout: "total_load": 20733, 2023-11-27T15:32:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "average_load": 20733, 2023-11-27T15:32:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "avg_session_uptime": 379 2023-11-27T15:32:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2023-11-27T15:32:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:32:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2023-11-27T15:32:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:32:51.867 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:32:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:32:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:32:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:32:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:32:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:32:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:32:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:32:51.868 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:32:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:32:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:32:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:32:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2023-11-27T15:32:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:32:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2023-11-27T15:32:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:32:51.869 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:32:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:32:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:32:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:32:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:32:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:32:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2023-11-27T15:32:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:32:51.870 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:32:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:32:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2023-11-27T15:32:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2023-11-27T15:32:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2023-11-27T15:32:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:32:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:32:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:32:51.871 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:32:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 300507, 2023-11-27T15:32:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 125, 2023-11-27T15:32:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 176, 2023-11-27T15:32:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 2, 2023-11-27T15:32:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2023-11-27T15:32:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2023-11-27T15:32:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:32:51.872 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2023-11-27T15:32:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:32:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2023-11-27T15:32:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 8792, 2023-11-27T15:32:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 168, 2023-11-27T15:32:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:32:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:32:51.873 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2023-11-27T15:32:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2023-11-27T15:32:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 99759, 2023-11-27T15:32:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 539, 2023-11-27T15:32:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2023-11-27T15:32:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2023-11-27T15:32:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2023-11-27T15:32:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2023-11-27T15:32:51.874 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "objecter": { 2023-11-27T15:32:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "op_active": 0, 2023-11-27T15:32:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "op_laggy": 0, 2023-11-27T15:32:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send": 43581, 2023-11-27T15:32:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "op_send_bytes": 184285672, 2023-11-27T15:32:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "op_resend": 27, 2023-11-27T15:32:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "op_reply": 43581, 2023-11-27T15:32:51.875 INFO:teuthology.orchestra.run.smithi029.stdout: "op_latency": { 2023-11-27T15:32:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 43581, 2023-11-27T15:32:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 407.977077371, 2023-11-27T15:32:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.009361351 2023-11-27T15:32:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "op_inflight": 0, 2023-11-27T15:32:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "oplen_avg": { 2023-11-27T15:32:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 43581, 2023-11-27T15:32:51.876 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 44493 2023-11-27T15:32:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: "op": 43581, 2023-11-27T15:32:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: "op_r": 297, 2023-11-27T15:32:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: "op_w": 43284, 2023-11-27T15:32:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: "op_rmw": 0, 2023-11-27T15:32:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: "op_pg": 0, 2023-11-27T15:32:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_stat": 0, 2023-11-27T15:32:51.877 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_create": 10, 2023-11-27T15:32:51.878 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_read": 297, 2023-11-27T15:32:51.878 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_write": 1190, 2023-11-27T15:32:51.878 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writefull": 117, 2023-11-27T15:32:51.878 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_writesame": 0, 2023-11-27T15:32:51.878 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_append": 0, 2023-11-27T15:32:51.878 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_zero": 0, 2023-11-27T15:32:51.878 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_truncate": 0, 2023-11-27T15:32:51.879 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_delete": 41038, 2023-11-27T15:32:51.879 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_mapext": 0, 2023-11-27T15:32:51.879 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_sparse_read": 0, 2023-11-27T15:32:51.879 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_clonerange": 0, 2023-11-27T15:32:51.879 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_getxattr": 0, 2023-11-27T15:32:51.879 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_setxattr": 20, 2023-11-27T15:32:51.879 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:32:51.879 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_rmxattr": 0, 2023-11-27T15:32:51.880 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:32:51.880 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_call": 0, 2023-11-27T15:32:51.880 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_watch": 0, 2023-11-27T15:32:51.880 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_notify": 0, 2023-11-27T15:32:51.880 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:32:51.880 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls": 0, 2023-11-27T15:32:51.880 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:32:51.880 INFO:teuthology.orchestra.run.smithi029.stdout: "osdop_other": 0, 2023-11-27T15:32:51.881 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_active": 0, 2023-11-27T15:32:51.881 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_send": 0, 2023-11-27T15:32:51.881 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_resend": 0, 2023-11-27T15:32:51.881 INFO:teuthology.orchestra.run.smithi029.stdout: "linger_ping": 0, 2023-11-27T15:32:51.881 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_active": 0, 2023-11-27T15:32:51.881 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_send": 0, 2023-11-27T15:32:51.881 INFO:teuthology.orchestra.run.smithi029.stdout: "poolop_resend": 0, 2023-11-27T15:32:51.881 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_active": 0, 2023-11-27T15:32:51.882 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_send": 0, 2023-11-27T15:32:51.882 INFO:teuthology.orchestra.run.smithi029.stdout: "poolstat_resend": 0, 2023-11-27T15:32:51.882 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_active": 0, 2023-11-27T15:32:51.882 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_send": 0, 2023-11-27T15:32:51.882 INFO:teuthology.orchestra.run.smithi029.stdout: "statfs_resend": 0, 2023-11-27T15:32:51.882 INFO:teuthology.orchestra.run.smithi029.stdout: "command_active": 0, 2023-11-27T15:32:51.882 INFO:teuthology.orchestra.run.smithi029.stdout: "command_send": 0, 2023-11-27T15:32:51.882 INFO:teuthology.orchestra.run.smithi029.stdout: "command_resend": 0, 2023-11-27T15:32:51.883 INFO:teuthology.orchestra.run.smithi029.stdout: "map_epoch": 94, 2023-11-27T15:32:51.883 INFO:teuthology.orchestra.run.smithi029.stdout: "map_full": 0, 2023-11-27T15:32:51.883 INFO:teuthology.orchestra.run.smithi029.stdout: "map_inc": 6, 2023-11-27T15:32:51.883 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_sessions": 12, 2023-11-27T15:32:51.883 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_open": 12, 2023-11-27T15:32:51.883 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_session_close": 0, 2023-11-27T15:32:51.883 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_laggy": 0, 2023-11-27T15:32:51.883 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_wr": 1778, 2023-11-27T15:32:51.884 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_rd": 0, 2023-11-27T15:32:51.884 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_del": 43 2023-11-27T15:32:51.884 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.884 INFO:teuthology.orchestra.run.smithi029.stdout: "oft": { 2023-11-27T15:32:51.884 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_objs": 1, 2023-11-27T15:32:51.884 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:32:51.884 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_updates": 0, 2023-11-27T15:32:51.884 INFO:teuthology.orchestra.run.smithi029.stdout: "omap_total_removes": 0 2023-11-27T15:32:51.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.885 INFO:teuthology.orchestra.run.smithi029.stdout: "osd-slow-ops": { 2023-11-27T15:32:51.885 INFO:teuthology.orchestra.run.smithi029.stdout: "slow_ops_count": 55343 2023-11-27T15:32:51.885 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.885 INFO:teuthology.orchestra.run.smithi029.stdout: "purge_queue": { 2023-11-27T15:32:51.885 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops": 0, 2023-11-27T15:32:51.885 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_ops_high_water": 49, 2023-11-27T15:32:51.885 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing": 0, 2023-11-27T15:32:51.886 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executing_high_water": 8, 2023-11-27T15:32:51.886 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_executed": 19897, 2023-11-27T15:32:51.886 INFO:teuthology.orchestra.run.smithi029.stdout: "pq_item_in_journal": 0 2023-11-27T15:32:51.886 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.886 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:32:51.886 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:51.886 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2023-11-27T15:32:51.886 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:51.887 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 134155, 2023-11-27T15:32:51.887 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 51437527, 2023-11-27T15:32:51.887 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:51.887 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 134155, 2023-11-27T15:32:51.887 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2023-11-27T15:32:51.887 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2023-11-27T15:32:51.887 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 134155, 2023-11-27T15:32:51.887 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 51437527, 2023-11-27T15:32:51.888 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:51.888 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.888 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.888 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.888 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:51.888 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.888 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_bytes": { 2023-11-27T15:32:51.888 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:51.889 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2023-11-27T15:32:51.889 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:51.889 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2023-11-27T15:32:51.889 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2023-11-27T15:32:51.889 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:51.889 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:51.889 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 43581, 2023-11-27T15:32:51.889 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 186454549, 2023-11-27T15:32:51.890 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 43581, 2023-11-27T15:32:51.890 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 186454549, 2023-11-27T15:32:51.890 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:51.890 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.890 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.890 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.890 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:51.891 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.891 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-objecter_ops": { 2023-11-27T15:32:51.891 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:51.891 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 1024, 2023-11-27T15:32:51.891 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:51.891 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 0, 2023-11-27T15:32:51.891 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 0, 2023-11-27T15:32:51.891 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:51.892 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:51.892 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 43581, 2023-11-27T15:32:51.892 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 43581, 2023-11-27T15:32:51.892 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 43581, 2023-11-27T15:32:51.892 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 43581, 2023-11-27T15:32:51.892 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:51.892 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.892 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.893 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.893 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:51.893 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.893 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:32:51.893 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:51.893 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2023-11-27T15:32:51.893 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:51.893 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 19897, 2023-11-27T15:32:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 2168877, 2023-11-27T15:32:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 19897, 2023-11-27T15:32:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2023-11-27T15:32:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2023-11-27T15:32:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 297, 2023-11-27T15:32:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 2168877, 2023-11-27T15:32:51.894 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:51.895 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.895 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.895 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.895 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:51.895 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:51.895 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-write_buf_throttle-0x564d200d1cc0": { 2023-11-27T15:32:51.895 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:51.895 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 3758096384, 2023-11-27T15:32:51.896 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:51.896 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 72305, 2023-11-27T15:32:51.896 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 173643840, 2023-11-27T15:32:51.896 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:51.896 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 72305, 2023-11-27T15:32:51.896 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2023-11-27T15:32:51.896 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2023-11-27T15:32:51.896 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 893, 2023-11-27T15:32:51.897 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 173643840, 2023-11-27T15:32:51.897 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:51.897 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:51.897 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:51.897 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:51.897 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:51.897 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:51.897 INFO:teuthology.orchestra.run.smithi029.stdout:} 2023-11-27T15:32:51.898 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:51 smithi008 ceph-mon[128788]: pgmap v570: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:51.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:51 smithi060 ceph-mon[131518]: pgmap v570: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:51.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:51 smithi029 ceph-mon[131166]: pgmap v570: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:52.218 INFO:tasks.check_counter:Daemon mds.h mds.exported=161 2023-11-27T15:32:52.218 INFO:tasks.check_counter:Daemon mds.h mds.imported=142 2023-11-27T15:32:52.218 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-11-27T15:32:52.218 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=2761 2023-11-27T15:32:52.219 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=100 2023-11-27T15:32:52.219 DEBUG:tasks.check_counter:Getting stats from k 2023-11-27T15:32:52.219 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.k perf dump 2023-11-27T15:32:52.874 INFO:teuthology.orchestra.run.smithi029.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:32:53.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:53 smithi060 ceph-mon[131518]: pgmap v571: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:53.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:53 smithi008 ceph-mon[128788]: pgmap v571: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:53.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:53 smithi029 ceph-mon[131166]: pgmap v571: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:54.267 INFO:teuthology.orchestra.run.smithi029.stdout:{ 2023-11-27T15:32:54.268 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:32:54.268 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 167, 2023-11-27T15:32:54.268 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 164, 2023-11-27T15:32:54.268 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 456992, 2023-11-27T15:32:54.268 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 92768, 2023-11-27T15:32:54.269 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 4, 2023-11-27T15:32:54.269 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 0, 2023-11-27T15:32:54.269 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.028368117, 2023-11-27T15:32:54.269 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.009088801, 2023-11-27T15:32:54.269 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.014485582, 2023-11-27T15:32:54.269 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.004945713, 2023-11-27T15:32:54.270 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:54.270 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 163, 2023-11-27T15:32:54.270 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.009322104, 2023-11-27T15:32:54.270 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000057190 2023-11-27T15:32:54.270 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.271 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:54.271 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:54.271 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:54.271 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:54.271 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.272 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 456992, 2023-11-27T15:32:54.272 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 92768 2023-11-27T15:32:54.272 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.272 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:32:54.272 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 1, 2023-11-27T15:32:54.272 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 124, 2023-11-27T15:32:54.273 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 160, 2023-11-27T15:32:54.273 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 97584, 2023-11-27T15:32:54.273 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 3, 2023-11-27T15:32:54.273 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 2, 2023-11-27T15:32:54.273 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.007388239, 2023-11-27T15:32:54.274 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.004823278, 2023-11-27T15:32:54.274 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.001153763, 2023-11-27T15:32:54.274 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:32:54.274 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:54.274 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 122, 2023-11-27T15:32:54.274 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.007418664, 2023-11-27T15:32:54.275 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000060808 2023-11-27T15:32:54.275 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.275 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:54.275 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:54.275 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:54.276 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:54.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.276 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:32:54.276 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 97584 2023-11-27T15:32:54.276 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.276 INFO:teuthology.orchestra.run.smithi029.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:32:54.277 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_messages": 4, 2023-11-27T15:32:54.277 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages": 4, 2023-11-27T15:32:54.277 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_bytes": 2992, 2023-11-27T15:32:54.277 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_bytes": 448, 2023-11-27T15:32:54.277 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_created_connections": 2, 2023-11-27T15:32:54.278 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_active_connections": 1, 2023-11-27T15:32:54.278 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_total_time": 0.000855436, 2023-11-27T15:32:54.278 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_send_time": 0.000047327, 2023-11-27T15:32:54.278 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_recv_time": 0.000770852, 2023-11-27T15:32:54.278 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:32:54.278 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:54.279 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 3, 2023-11-27T15:32:54.279 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000908615, 2023-11-27T15:32:54.279 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000302871 2023-11-27T15:32:54.279 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.279 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:54.280 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:54.280 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:54.280 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:54.280 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.280 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-27T15:32:54.280 INFO:teuthology.orchestra.run.smithi029.stdout: "msgr_send_encrypted_bytes": 448 2023-11-27T15:32:54.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.281 INFO:teuthology.orchestra.run.smithi029.stdout: "cct": { 2023-11-27T15:32:54.281 INFO:teuthology.orchestra.run.smithi029.stdout: "total_workers": 0, 2023-11-27T15:32:54.281 INFO:teuthology.orchestra.run.smithi029.stdout: "unhealthy_workers": 0 2023-11-27T15:32:54.281 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.281 INFO:teuthology.orchestra.run.smithi029.stdout: "mempool": { 2023-11-27T15:32:54.282 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:32:54.282 INFO:teuthology.orchestra.run.smithi029.stdout: "bloom_filter_items": 0, 2023-11-27T15:32:54.282 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:32:54.282 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:32:54.282 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:32:54.283 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:32:54.283 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:32:54.283 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:32:54.283 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:32:54.283 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:32:54.283 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:32:54.284 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:32:54.284 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:32:54.284 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:32:54.284 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:32:54.284 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_extent_items": 0, 2023-11-27T15:32:54.284 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:32:54.285 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_blob_items": 0, 2023-11-27T15:32:54.285 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:32:54.285 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:32:54.285 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:32:54.285 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:32:54.286 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:32:54.286 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:32:54.286 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:32:54.286 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_txc_items": 0, 2023-11-27T15:32:54.286 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:32:54.286 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:32:54.287 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:32:54.287 INFO:teuthology.orchestra.run.smithi029.stdout: "bluestore_writing_items": 0, 2023-11-27T15:32:54.287 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_bytes": 0, 2023-11-27T15:32:54.287 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_items": 0, 2023-11-27T15:32:54.287 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:32:54.288 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:32:54.288 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:32:54.288 INFO:teuthology.orchestra.run.smithi029.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:32:54.288 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_bytes": 148967, 2023-11-27T15:32:54.288 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_anon_items": 29, 2023-11-27T15:32:54.288 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:32:54.289 INFO:teuthology.orchestra.run.smithi029.stdout: "buffer_meta_items": 0, 2023-11-27T15:32:54.289 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_bytes": 0, 2023-11-27T15:32:54.289 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_items": 0, 2023-11-27T15:32:54.289 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:32:54.289 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_mapbl_items": 0, 2023-11-27T15:32:54.290 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:32:54.290 INFO:teuthology.orchestra.run.smithi029.stdout: "osd_pglog_items": 0, 2023-11-27T15:32:54.290 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_bytes": 0, 2023-11-27T15:32:54.290 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_items": 0, 2023-11-27T15:32:54.290 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:32:54.290 INFO:teuthology.orchestra.run.smithi029.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:32:54.291 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_bytes": 0, 2023-11-27T15:32:54.291 INFO:teuthology.orchestra.run.smithi029.stdout: "pgmap_items": 0, 2023-11-27T15:32:54.291 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_bytes": 928, 2023-11-27T15:32:54.291 INFO:teuthology.orchestra.run.smithi029.stdout: "mds_co_items": 2, 2023-11-27T15:32:54.291 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_bytes": 0, 2023-11-27T15:32:54.291 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_1_items": 0, 2023-11-27T15:32:54.292 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_bytes": 0, 2023-11-27T15:32:54.292 INFO:teuthology.orchestra.run.smithi029.stdout: "unittest_2_items": 0 2023-11-27T15:32:54.292 INFO:teuthology.orchestra.run.smithi029.stdout: }, 2023-11-27T15:32:54.292 INFO:teuthology.orchestra.run.smithi029.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:32:54.292 INFO:teuthology.orchestra.run.smithi029.stdout: "val": 0, 2023-11-27T15:32:54.292 INFO:teuthology.orchestra.run.smithi029.stdout: "max": 104857600, 2023-11-27T15:32:54.293 INFO:teuthology.orchestra.run.smithi029.stdout: "get_started": 0, 2023-11-27T15:32:54.293 INFO:teuthology.orchestra.run.smithi029.stdout: "get": 168, 2023-11-27T15:32:54.293 INFO:teuthology.orchestra.run.smithi029.stdout: "get_sum": 433394, 2023-11-27T15:32:54.293 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:54.293 INFO:teuthology.orchestra.run.smithi029.stdout: "get_or_fail_success": 168, 2023-11-27T15:32:54.293 INFO:teuthology.orchestra.run.smithi029.stdout: "take": 0, 2023-11-27T15:32:54.294 INFO:teuthology.orchestra.run.smithi029.stdout: "take_sum": 0, 2023-11-27T15:32:54.294 INFO:teuthology.orchestra.run.smithi029.stdout: "put": 168, 2023-11-27T15:32:54.294 INFO:teuthology.orchestra.run.smithi029.stdout: "put_sum": 433394, 2023-11-27T15:32:54.294 INFO:teuthology.orchestra.run.smithi029.stdout: "wait": { 2023-11-27T15:32:54.294 INFO:teuthology.orchestra.run.smithi029.stdout: "avgcount": 0, 2023-11-27T15:32:54.294 INFO:teuthology.orchestra.run.smithi029.stdout: "sum": 0.000000000, 2023-11-27T15:32:54.295 INFO:teuthology.orchestra.run.smithi029.stdout: "avgtime": 0.000000000 2023-11-27T15:32:54.295 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:54.295 INFO:teuthology.orchestra.run.smithi029.stdout: } 2023-11-27T15:32:54.295 INFO:teuthology.orchestra.run.smithi029.stdout:} 2023-11-27T15:32:54.723 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-11-27T15:32:54.723 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-11-27T15:32:54.723 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-11-27T15:32:54.723 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-11-27T15:32:54.723 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-11-27T15:32:54.724 DEBUG:tasks.check_counter:Getting stats from c 2023-11-27T15:32:54.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.c perf dump 2023-11-27T15:32:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:32:55.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:55 smithi060 ceph-mon[131518]: pgmap v572: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:55.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:55 smithi008 ceph-mon[128788]: pgmap v572: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:55.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:55 smithi029 ceph-mon[131166]: pgmap v572: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:56.846 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-27T15:32:56.847 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:32:56.847 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1306, 2023-11-27T15:32:56.847 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 1308, 2023-11-27T15:32:56.847 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 71488682, 2023-11-27T15:32:56.847 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 406173, 2023-11-27T15:32:56.848 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 8, 2023-11-27T15:32:56.848 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2023-11-27T15:32:56.848 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.170164583, 2023-11-27T15:32:56.848 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.040319355, 2023-11-27T15:32:56.848 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.068408615, 2023-11-27T15:32:56.849 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.048526276, 2023-11-27T15:32:56.849 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:56.849 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1306, 2023-11-27T15:32:56.849 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.070532460, 2023-11-27T15:32:56.849 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000054006 2023-11-27T15:32:56.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.850 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:56.850 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.850 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.850 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.851 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:32:56.851 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:32:56.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.851 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:32:56.851 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 529, 2023-11-27T15:32:56.851 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 509, 2023-11-27T15:32:56.852 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 67858055, 2023-11-27T15:32:56.852 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 197575, 2023-11-27T15:32:56.852 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 6, 2023-11-27T15:32:56.852 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 5, 2023-11-27T15:32:56.852 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.102396319, 2023-11-27T15:32:56.852 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.019526779, 2023-11-27T15:32:56.853 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.065819412, 2023-11-27T15:32:56.853 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.015427617, 2023-11-27T15:32:56.853 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:56.853 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 509, 2023-11-27T15:32:56.853 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.031069763, 2023-11-27T15:32:56.854 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000061040 2023-11-27T15:32:56.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.854 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:56.854 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.854 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.854 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.855 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 532384, 2023-11-27T15:32:56.855 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 93328 2023-11-27T15:32:56.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.855 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:32:56.855 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 304, 2023-11-27T15:32:56.856 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 424, 2023-11-27T15:32:56.856 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 67193052, 2023-11-27T15:32:56.856 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 366153, 2023-11-27T15:32:56.856 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 6, 2023-11-27T15:32:56.856 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 5, 2023-11-27T15:32:56.857 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.104911601, 2023-11-27T15:32:56.857 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.011447614, 2023-11-27T15:32:56.857 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.041974718, 2023-11-27T15:32:56.857 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.047691934, 2023-11-27T15:32:56.857 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:56.857 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 422, 2023-11-27T15:32:56.858 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.023471206, 2023-11-27T15:32:56.858 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000055618 2023-11-27T15:32:56.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.858 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:56.858 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.859 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.859 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.859 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:32:56.859 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 271920 2023-11-27T15:32:56.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.860 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2023-11-27T15:32:56.860 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2023-11-27T15:32:56.860 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2023-11-27T15:32:56.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.860 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2023-11-27T15:32:56.861 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-11-27T15:32:56.861 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-11-27T15:32:56.861 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3150, 2023-11-27T15:32:56.861 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.100524963, 2023-11-27T15:32:56.861 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000031912 2023-11-27T15:32:56.861 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:56.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.862 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2023-11-27T15:32:56.862 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-11-27T15:32:56.862 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-11-27T15:32:56.862 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.862 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.863 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:56.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.863 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2023-11-27T15:32:56.863 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 0, 2023-11-27T15:32:56.864 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 0, 2023-11-27T15:32:56.864 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2023-11-27T15:32:56.864 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.864 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.864 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.865 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2023-11-27T15:32:56.865 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 0, 2023-11-27T15:32:56.865 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 0, 2023-11-27T15:32:56.865 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 0, 2023-11-27T15:32:56.865 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 0, 2023-11-27T15:32:56.865 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 0, 2023-11-27T15:32:56.866 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2023-11-27T15:32:56.866 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 19675, 2023-11-27T15:32:56.866 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 0, 2023-11-27T15:32:56.866 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 0, 2023-11-27T15:32:56.866 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 19675, 2023-11-27T15:32:56.866 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 19675, 2023-11-27T15:32:56.867 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 1, 2023-11-27T15:32:56.867 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 0, 2023-11-27T15:32:56.867 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 0, 2023-11-27T15:32:56.867 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 1, 2023-11-27T15:32:56.867 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 0, 2023-11-27T15:32:56.867 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 0, 2023-11-27T15:32:56.868 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 0, 2023-11-27T15:32:56.868 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 0, 2023-11-27T15:32:56.868 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:32:56.868 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2023-11-27T15:32:56.868 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 0, 2023-11-27T15:32:56.869 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 0, 2023-11-27T15:32:56.869 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2023-11-27T15:32:56.869 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 0, 2023-11-27T15:32:56.869 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 0, 2023-11-27T15:32:56.869 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 0, 2023-11-27T15:32:56.869 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 0, 2023-11-27T15:32:56.869 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2023-11-27T15:32:56.870 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:32:56.870 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2023-11-27T15:32:56.870 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 0, 2023-11-27T15:32:56.870 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 0, 2023-11-27T15:32:56.870 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2023-11-27T15:32:56.870 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:32:56.871 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2023-11-27T15:32:56.871 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:32:56.871 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:32:56.871 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:32:56.871 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:32:56.871 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:32:56.872 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 0, 2023-11-27T15:32:56.872 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 0, 2023-11-27T15:32:56.872 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 0, 2023-11-27T15:32:56.872 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 0, 2023-11-27T15:32:56.872 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:32:56.872 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:32:56.873 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:32:56.873 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 0, 2023-11-27T15:32:56.873 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 0, 2023-11-27T15:32:56.873 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 0, 2023-11-27T15:32:56.873 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 0 2023-11-27T15:32:56.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.873 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2023-11-27T15:32:56.874 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2023-11-27T15:32:56.874 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2023-11-27T15:32:56.874 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2023-11-27T15:32:56.874 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 0, 2023-11-27T15:32:56.874 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 0, 2023-11-27T15:32:56.874 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 0, 2023-11-27T15:32:56.875 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 0, 2023-11-27T15:32:56.875 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:32:56.875 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 0, 2023-11-27T15:32:56.875 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 0, 2023-11-27T15:32:56.875 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 0, 2023-11-27T15:32:56.875 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2023-11-27T15:32:56.876 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2023-11-27T15:32:56.876 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:32:56.876 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:32:56.876 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2023-11-27T15:32:56.876 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2023-11-27T15:32:56.876 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:32:56.877 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 0, 2023-11-27T15:32:56.877 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2023-11-27T15:32:56.877 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 0, 2023-11-27T15:32:56.877 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2023-11-27T15:32:56.877 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2023-11-27T15:32:56.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.878 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2023-11-27T15:32:56.878 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 0, 2023-11-27T15:32:56.878 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 0, 2023-11-27T15:32:56.878 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 0, 2023-11-27T15:32:56.878 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 0, 2023-11-27T15:32:56.878 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2023-11-27T15:32:56.879 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 0, 2023-11-27T15:32:56.879 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 0, 2023-11-27T15:32:56.879 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 0, 2023-11-27T15:32:56.879 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 0, 2023-11-27T15:32:56.879 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 108, 2023-11-27T15:32:56.879 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2023-11-27T15:32:56.879 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 0, 2023-11-27T15:32:56.880 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 4194304, 2023-11-27T15:32:56.880 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 0, 2023-11-27T15:32:56.880 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 209516118, 2023-11-27T15:32:56.880 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2023-11-27T15:32:56.880 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.880 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.881 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.881 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 86307 2023-11-27T15:32:56.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.881 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2023-11-27T15:32:56.881 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 13, 2023-11-27T15:32:56.881 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 38075, 2023-11-27T15:32:56.882 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 38062, 2023-11-27T15:32:56.882 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 11, 2023-11-27T15:32:56.882 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 2209, 2023-11-27T15:32:56.882 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 2198, 2023-11-27T15:32:56.882 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 19675, 2023-11-27T15:32:56.882 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 64347, 2023-11-27T15:32:56.883 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 44672, 2023-11-27T15:32:56.883 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 0, 2023-11-27T15:32:56.883 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 0, 2023-11-27T15:32:56.883 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 0, 2023-11-27T15:32:56.883 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 106248, 2023-11-27T15:32:56.883 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 207156 2023-11-27T15:32:56.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.884 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2023-11-27T15:32:56.884 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 0, 2023-11-27T15:32:56.884 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 0, 2023-11-27T15:32:56.884 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 0, 2023-11-27T15:32:56.884 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 0, 2023-11-27T15:32:56.884 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 0, 2023-11-27T15:32:56.885 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2023-11-27T15:32:56.885 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.885 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.885 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.885 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2023-11-27T15:32:56.885 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.886 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.886 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.886 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2023-11-27T15:32:56.886 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.886 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.887 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.887 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2023-11-27T15:32:56.887 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.887 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.887 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.888 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2023-11-27T15:32:56.888 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.888 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.888 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.889 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2023-11-27T15:32:56.889 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.889 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.889 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.889 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2023-11-27T15:32:56.889 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.890 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.890 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.890 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2023-11-27T15:32:56.890 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.890 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.891 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.891 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2023-11-27T15:32:56.891 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.891 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.891 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.892 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2023-11-27T15:32:56.892 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.892 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.892 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.892 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2023-11-27T15:32:56.892 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.893 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.893 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.893 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2023-11-27T15:32:56.893 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.893 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.893 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.894 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2023-11-27T15:32:56.894 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.894 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.894 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.895 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2023-11-27T15:32:56.895 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.895 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.895 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.895 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2023-11-27T15:32:56.895 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.896 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.896 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.896 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2023-11-27T15:32:56.896 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.896 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.897 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.897 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2023-11-27T15:32:56.897 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.897 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.897 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.898 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2023-11-27T15:32:56.898 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.898 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.898 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.898 INFO:teuthology.orchestra.run.smithi060.stdout: "req_snapdiff_latency": { 2023-11-27T15:32:56.899 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.899 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.899 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.899 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2023-11-27T15:32:56.899 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.899 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.900 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.900 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2023-11-27T15:32:56.900 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.900 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.900 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.901 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2023-11-27T15:32:56.901 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.901 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.901 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.901 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2023-11-27T15:32:56.902 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.902 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.902 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.902 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2023-11-27T15:32:56.902 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.902 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.903 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.903 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2023-11-27T15:32:56.903 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.903 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.903 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.904 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2023-11-27T15:32:56.904 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.904 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.904 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.904 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2023-11-27T15:32:56.905 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.905 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.905 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.905 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2023-11-27T15:32:56.905 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.906 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.906 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.906 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2023-11-27T15:32:56.906 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.906 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.906 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.907 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:32:56.907 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:32:56.907 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2023-11-27T15:32:56.907 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.907 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.907 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.908 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:56.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.908 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2023-11-27T15:32:56.908 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2023-11-27T15:32:56.908 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 2, 2023-11-27T15:32:56.908 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 1, 2023-11-27T15:32:56.909 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 1, 2023-11-27T15:32:56.909 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2023-11-27T15:32:56.909 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 0, 2023-11-27T15:32:56.909 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 0, 2023-11-27T15:32:56.909 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 2460 2023-11-27T15:32:56.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.909 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2023-11-27T15:32:56.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:32:56.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2023-11-27T15:32:56.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:32:56.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:32:56.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:32:56.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:32:56.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:32:56.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:32:56.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:32:56.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:32:56.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:32:56.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:32:56.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:32:56.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:32:56.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:32:56.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2023-11-27T15:32:56.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:32:56.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2023-11-27T15:32:56.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:32:56.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:32:56.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:32:56.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:32:56.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:32:56.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:32:56.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:32:56.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2023-11-27T15:32:56.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:32:56.913 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:32:56.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:32:56.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2023-11-27T15:32:56.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2023-11-27T15:32:56.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2023-11-27T15:32:56.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:32:56.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:32:56.914 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:32:56.915 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:32:56.915 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 239511, 2023-11-27T15:32:56.915 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 80, 2023-11-27T15:32:56.915 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:32:56.915 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2023-11-27T15:32:56.915 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2023-11-27T15:32:56.915 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2023-11-27T15:32:56.916 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:32:56.916 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2023-11-27T15:32:56.916 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:32:56.916 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2023-11-27T15:32:56.916 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 8792, 2023-11-27T15:32:56.916 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 168, 2023-11-27T15:32:56.916 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:32:56.917 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:32:56.917 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2023-11-27T15:32:56.917 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2023-11-27T15:32:56.917 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 13893960, 2023-11-27T15:32:56.917 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 78828, 2023-11-27T15:32:56.917 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2023-11-27T15:32:56.917 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2023-11-27T15:32:56.918 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2023-11-27T15:32:56.918 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2023-11-27T15:32:56.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.918 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2023-11-27T15:32:56.918 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2023-11-27T15:32:56.918 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2023-11-27T15:32:56.919 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 1944, 2023-11-27T15:32:56.919 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 21, 2023-11-27T15:32:56.919 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 3, 2023-11-27T15:32:56.919 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 1944, 2023-11-27T15:32:56.919 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2023-11-27T15:32:56.919 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1944, 2023-11-27T15:32:56.919 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 3.993961045, 2023-11-27T15:32:56.920 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002054506 2023-11-27T15:32:56.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.920 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2023-11-27T15:32:56.920 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2023-11-27T15:32:56.920 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1944, 2023-11-27T15:32:56.920 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1945 2023-11-27T15:32:56.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.920 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 1944, 2023-11-27T15:32:56.921 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 1944, 2023-11-27T15:32:56.921 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 0, 2023-11-27T15:32:56.921 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2023-11-27T15:32:56.921 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2023-11-27T15:32:56.921 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 1172, 2023-11-27T15:32:56.921 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 0, 2023-11-27T15:32:56.921 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 770, 2023-11-27T15:32:56.922 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 0, 2023-11-27T15:32:56.922 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 0, 2023-11-27T15:32:56.922 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2023-11-27T15:32:56.922 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2023-11-27T15:32:56.922 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 0, 2023-11-27T15:32:56.922 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 0, 2023-11-27T15:32:56.923 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 0, 2023-11-27T15:32:56.923 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2023-11-27T15:32:56.923 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2023-11-27T15:32:56.923 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2023-11-27T15:32:56.923 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 1, 2023-11-27T15:32:56.923 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 0, 2023-11-27T15:32:56.923 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:32:56.924 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2023-11-27T15:32:56.924 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:32:56.924 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2023-11-27T15:32:56.924 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2023-11-27T15:32:56.924 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2023-11-27T15:32:56.924 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:32:56.924 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2023-11-27T15:32:56.925 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:32:56.925 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 0, 2023-11-27T15:32:56.925 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2023-11-27T15:32:56.925 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2023-11-27T15:32:56.925 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2023-11-27T15:32:56.925 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2023-11-27T15:32:56.925 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2023-11-27T15:32:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2023-11-27T15:32:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2023-11-27T15:32:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2023-11-27T15:32:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2023-11-27T15:32:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2023-11-27T15:32:56.926 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2023-11-27T15:32:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2023-11-27T15:32:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2023-11-27T15:32:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2023-11-27T15:32:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2023-11-27T15:32:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2023-11-27T15:32:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 94, 2023-11-27T15:32:56.927 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2023-11-27T15:32:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 6, 2023-11-27T15:32:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 11, 2023-11-27T15:32:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 11, 2023-11-27T15:32:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 0, 2023-11-27T15:32:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2023-11-27T15:32:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 0, 2023-11-27T15:32:56.928 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 2, 2023-11-27T15:32:56.929 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 0 2023-11-27T15:32:56.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.929 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2023-11-27T15:32:56.929 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 0, 2023-11-27T15:32:56.929 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:32:56.929 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2023-11-27T15:32:56.929 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2023-11-27T15:32:56.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.930 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2023-11-27T15:32:56.930 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 0 2023-11-27T15:32:56.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.930 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2023-11-27T15:32:56.930 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2023-11-27T15:32:56.930 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 0, 2023-11-27T15:32:56.931 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2023-11-27T15:32:56.931 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 0, 2023-11-27T15:32:56.931 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 0, 2023-11-27T15:32:56.931 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2023-11-27T15:32:56.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.931 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:32:56.931 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:32:56.932 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-11-27T15:32:56.932 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:32:56.932 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 2135, 2023-11-27T15:32:56.932 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 206334985, 2023-11-27T15:32:56.932 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:56.932 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 2135, 2023-11-27T15:32:56.932 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-11-27T15:32:56.932 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-11-27T15:32:56.933 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 2135, 2023-11-27T15:32:56.933 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 206334985, 2023-11-27T15:32:56.933 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:32:56.933 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.933 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.933 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:56.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.934 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2023-11-27T15:32:56.934 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:32:56.934 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-11-27T15:32:56.934 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:32:56.934 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:32:56.934 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:32:56.934 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:56.935 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:56.935 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 1944, 2023-11-27T15:32:56.935 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 205467303, 2023-11-27T15:32:56.935 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 1944, 2023-11-27T15:32:56.935 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 205467303, 2023-11-27T15:32:56.935 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:32:56.935 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.936 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.936 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.936 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:56.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.936 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2023-11-27T15:32:56.936 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:32:56.936 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2023-11-27T15:32:56.936 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:32:56.937 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:32:56.937 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:32:56.937 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:56.937 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:56.937 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 1944, 2023-11-27T15:32:56.937 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 1944, 2023-11-27T15:32:56.937 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 1944, 2023-11-27T15:32:56.938 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 1944, 2023-11-27T15:32:56.938 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:32:56.938 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.938 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.938 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:56.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.939 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:32:56.939 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:32:56.939 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-11-27T15:32:56.939 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:32:56.939 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:32:56.939 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:32:56.939 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:56.940 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:56.940 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-11-27T15:32:56.940 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-11-27T15:32:56.940 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-11-27T15:32:56.940 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-11-27T15:32:56.940 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:32:56.940 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.941 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.941 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.941 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:56.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:56.941 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x5632e5f6c000": { 2023-11-27T15:32:56.941 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:32:56.941 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-11-27T15:32:56.941 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:32:56.942 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:32:56.942 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:32:56.942 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:56.942 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:56.942 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-11-27T15:32:56.942 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-11-27T15:32:56.943 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-11-27T15:32:56.943 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-11-27T15:32:56.943 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:32:56.943 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:56.943 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:56.943 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:56.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:56.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:56.944 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-27T15:32:57.250 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-11-27T15:32:57.250 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-11-27T15:32:57.250 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-11-27T15:32:57.251 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-11-27T15:32:57.251 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-11-27T15:32:57.251 DEBUG:tasks.check_counter:Getting stats from f 2023-11-27T15:32:57.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.f perf dump 2023-11-27T15:32:57.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:57 smithi060 ceph-mon[131518]: pgmap v573: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:57.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:57 smithi008 ceph-mon[128788]: pgmap v573: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:57.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:57 smithi029 ceph-mon[131166]: pgmap v573: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:32:57.905 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:32:59.209 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-27T15:32:59.209 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:32:59.209 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 413, 2023-11-27T15:32:59.209 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 413, 2023-11-27T15:32:59.210 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 33670354, 2023-11-27T15:32:59.210 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 127711, 2023-11-27T15:32:59.210 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 8, 2023-11-27T15:32:59.210 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2023-11-27T15:32:59.210 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.058965461, 2023-11-27T15:32:59.211 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.011481316, 2023-11-27T15:32:59.211 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.028469552, 2023-11-27T15:32:59.211 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.013920369, 2023-11-27T15:32:59.211 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:59.211 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 412, 2023-11-27T15:32:59.211 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.021420563, 2023-11-27T15:32:59.212 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000051991 2023-11-27T15:32:59.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.212 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:59.212 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.212 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.213 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.213 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-27T15:32:59.213 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 448 2023-11-27T15:32:59.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.213 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:32:59.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 639, 2023-11-27T15:32:59.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 760, 2023-11-27T15:32:59.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 73341620, 2023-11-27T15:32:59.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 470519, 2023-11-27T15:32:59.214 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 7, 2023-11-27T15:32:59.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 6, 2023-11-27T15:32:59.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.104779205, 2023-11-27T15:32:59.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.018994743, 2023-11-27T15:32:59.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.056404206, 2023-11-27T15:32:59.215 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.022731897, 2023-11-27T15:32:59.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:59.216 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 758, 2023-11-27T15:32:59.216 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.048352757, 2023-11-27T15:32:59.216 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000063789 2023-11-27T15:32:59.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.216 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:59.217 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.217 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.217 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.217 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:32:59.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 272096 2023-11-27T15:32:59.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.218 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:32:59.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1021, 2023-11-27T15:32:59.218 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 1006, 2023-11-27T15:32:59.219 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 67905132, 2023-11-27T15:32:59.219 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 352535, 2023-11-27T15:32:59.219 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 5, 2023-11-27T15:32:59.219 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2023-11-27T15:32:59.219 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.149738223, 2023-11-27T15:32:59.220 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.034337585, 2023-11-27T15:32:59.220 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.071697844, 2023-11-27T15:32:59.220 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.036198938, 2023-11-27T15:32:59.220 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:32:59.220 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1005, 2023-11-27T15:32:59.220 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.057080272, 2023-11-27T15:32:59.221 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000056796 2023-11-27T15:32:59.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.221 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:32:59.221 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.221 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.222 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.222 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 520080, 2023-11-27T15:32:59.222 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 93472 2023-11-27T15:32:59.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.222 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2023-11-27T15:32:59.223 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2023-11-27T15:32:59.223 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2023-11-27T15:32:59.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.223 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2023-11-27T15:32:59.223 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-11-27T15:32:59.223 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-11-27T15:32:59.224 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3056, 2023-11-27T15:32:59.224 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.101962242, 2023-11-27T15:32:59.224 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000033364 2023-11-27T15:32:59.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:59.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.225 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2023-11-27T15:32:59.225 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-11-27T15:32:59.225 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-11-27T15:32:59.225 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.225 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.225 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.226 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:59.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.226 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2023-11-27T15:32:59.226 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 0, 2023-11-27T15:32:59.226 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 0, 2023-11-27T15:32:59.226 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2023-11-27T15:32:59.227 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.227 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.227 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.227 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2023-11-27T15:32:59.228 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 0, 2023-11-27T15:32:59.228 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 0, 2023-11-27T15:32:59.228 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 0, 2023-11-27T15:32:59.228 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 0, 2023-11-27T15:32:59.228 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 0, 2023-11-27T15:32:59.228 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2023-11-27T15:32:59.229 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 19907, 2023-11-27T15:32:59.229 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 0, 2023-11-27T15:32:59.229 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 0, 2023-11-27T15:32:59.229 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 19907, 2023-11-27T15:32:59.229 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 19907, 2023-11-27T15:32:59.229 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 1, 2023-11-27T15:32:59.230 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 0, 2023-11-27T15:32:59.230 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 0, 2023-11-27T15:32:59.230 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 1, 2023-11-27T15:32:59.230 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 0, 2023-11-27T15:32:59.230 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 0, 2023-11-27T15:32:59.231 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 0, 2023-11-27T15:32:59.231 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 0, 2023-11-27T15:32:59.231 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:32:59.231 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2023-11-27T15:32:59.231 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 0, 2023-11-27T15:32:59.231 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 0, 2023-11-27T15:32:59.232 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2023-11-27T15:32:59.232 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 0, 2023-11-27T15:32:59.232 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 0, 2023-11-27T15:32:59.232 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 0, 2023-11-27T15:32:59.232 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 0, 2023-11-27T15:32:59.232 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2023-11-27T15:32:59.233 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:32:59.233 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2023-11-27T15:32:59.233 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 0, 2023-11-27T15:32:59.233 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 0, 2023-11-27T15:32:59.233 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2023-11-27T15:32:59.233 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:32:59.234 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2023-11-27T15:32:59.234 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:32:59.234 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:32:59.234 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:32:59.234 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:32:59.234 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:32:59.235 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 0, 2023-11-27T15:32:59.235 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 0, 2023-11-27T15:32:59.235 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 0, 2023-11-27T15:32:59.235 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 0, 2023-11-27T15:32:59.235 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:32:59.235 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:32:59.236 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:32:59.236 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 0, 2023-11-27T15:32:59.236 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 0, 2023-11-27T15:32:59.236 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 0, 2023-11-27T15:32:59.236 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 0 2023-11-27T15:32:59.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.237 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2023-11-27T15:32:59.237 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2023-11-27T15:32:59.237 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2023-11-27T15:32:59.237 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2023-11-27T15:32:59.237 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 0, 2023-11-27T15:32:59.237 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 0, 2023-11-27T15:32:59.238 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 0, 2023-11-27T15:32:59.238 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 0, 2023-11-27T15:32:59.238 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:32:59.238 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 0, 2023-11-27T15:32:59.238 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 0, 2023-11-27T15:32:59.239 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 0, 2023-11-27T15:32:59.239 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2023-11-27T15:32:59.239 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2023-11-27T15:32:59.239 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:32:59.239 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:32:59.239 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2023-11-27T15:32:59.240 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2023-11-27T15:32:59.240 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:32:59.240 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 0, 2023-11-27T15:32:59.240 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2023-11-27T15:32:59.240 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 0, 2023-11-27T15:32:59.241 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2023-11-27T15:32:59.241 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2023-11-27T15:32:59.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.241 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2023-11-27T15:32:59.241 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 0, 2023-11-27T15:32:59.241 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 0, 2023-11-27T15:32:59.242 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 0, 2023-11-27T15:32:59.242 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 0, 2023-11-27T15:32:59.242 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2023-11-27T15:32:59.242 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 0, 2023-11-27T15:32:59.242 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 0, 2023-11-27T15:32:59.242 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 0, 2023-11-27T15:32:59.243 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 0, 2023-11-27T15:32:59.243 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 88, 2023-11-27T15:32:59.243 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2023-11-27T15:32:59.243 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 0, 2023-11-27T15:32:59.243 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 4194304, 2023-11-27T15:32:59.243 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 0, 2023-11-27T15:32:59.244 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 177837006, 2023-11-27T15:32:59.244 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2023-11-27T15:32:59.244 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.244 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.244 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.245 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 72305 2023-11-27T15:32:59.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.245 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2023-11-27T15:32:59.245 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 13, 2023-11-27T15:32:59.245 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 22647, 2023-11-27T15:32:59.245 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 22634, 2023-11-27T15:32:59.245 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 11, 2023-11-27T15:32:59.246 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 1401, 2023-11-27T15:32:59.246 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 1390, 2023-11-27T15:32:59.246 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 19907, 2023-11-27T15:32:59.246 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 44901, 2023-11-27T15:32:59.246 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 24994, 2023-11-27T15:32:59.246 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 0, 2023-11-27T15:32:59.247 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 0, 2023-11-27T15:32:59.247 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 0, 2023-11-27T15:32:59.247 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 119252, 2023-11-27T15:32:59.247 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 207156 2023-11-27T15:32:59.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.248 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2023-11-27T15:32:59.248 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 0, 2023-11-27T15:32:59.248 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 0, 2023-11-27T15:32:59.248 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 0, 2023-11-27T15:32:59.248 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 0, 2023-11-27T15:32:59.248 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 0, 2023-11-27T15:32:59.249 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2023-11-27T15:32:59.249 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.249 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.249 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.249 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2023-11-27T15:32:59.249 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.250 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.250 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.250 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2023-11-27T15:32:59.250 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.250 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.251 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.251 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2023-11-27T15:32:59.251 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.251 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.251 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.252 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2023-11-27T15:32:59.252 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.252 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.252 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.253 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2023-11-27T15:32:59.253 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.253 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.253 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.253 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2023-11-27T15:32:59.253 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.254 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.254 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.254 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2023-11-27T15:32:59.254 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.254 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.255 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.255 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2023-11-27T15:32:59.255 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.255 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.255 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.256 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2023-11-27T15:32:59.256 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.256 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.256 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.256 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2023-11-27T15:32:59.257 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.257 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.257 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.257 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2023-11-27T15:32:59.257 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.258 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.258 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.258 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2023-11-27T15:32:59.258 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.258 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.259 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.259 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2023-11-27T15:32:59.259 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.259 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.259 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.260 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2023-11-27T15:32:59.260 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.260 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.260 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.261 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2023-11-27T15:32:59.261 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.261 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.261 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.261 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2023-11-27T15:32:59.262 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.262 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.262 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.262 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2023-11-27T15:32:59.262 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.262 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.263 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.263 INFO:teuthology.orchestra.run.smithi060.stdout: "req_snapdiff_latency": { 2023-11-27T15:32:59.263 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.263 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.263 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.264 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2023-11-27T15:32:59.264 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.264 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.264 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.264 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2023-11-27T15:32:59.265 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.265 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.265 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.265 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2023-11-27T15:32:59.265 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.266 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.266 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.266 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2023-11-27T15:32:59.266 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.266 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.266 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.267 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2023-11-27T15:32:59.267 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.267 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.267 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.268 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2023-11-27T15:32:59.268 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.268 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.268 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.268 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2023-11-27T15:32:59.269 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.269 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.269 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.269 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2023-11-27T15:32:59.269 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.269 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.270 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.270 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2023-11-27T15:32:59.270 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.270 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.270 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.271 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2023-11-27T15:32:59.271 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.271 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.271 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.271 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:32:59.272 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:32:59.272 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2023-11-27T15:32:59.272 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.272 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.272 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.272 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:59.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.273 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2023-11-27T15:32:59.273 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2023-11-27T15:32:59.273 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 2, 2023-11-27T15:32:59.273 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 1, 2023-11-27T15:32:59.273 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 1, 2023-11-27T15:32:59.273 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2023-11-27T15:32:59.274 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 0, 2023-11-27T15:32:59.274 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 0, 2023-11-27T15:32:59.274 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 2463 2023-11-27T15:32:59.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.274 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2023-11-27T15:32:59.274 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:32:59.274 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2023-11-27T15:32:59.275 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:32:59.275 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:32:59.275 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:32:59.275 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:32:59.275 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:32:59.275 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:32:59.275 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:32:59.276 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:32:59.276 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:32:59.276 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:32:59.276 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:32:59.276 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:32:59.276 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:32:59.276 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2023-11-27T15:32:59.277 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:32:59.277 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2023-11-27T15:32:59.277 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:32:59.277 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:32:59.277 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:32:59.277 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:32:59.277 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:32:59.278 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:32:59.278 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:32:59.278 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2023-11-27T15:32:59.278 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:32:59.278 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:32:59.278 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:32:59.279 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2023-11-27T15:32:59.279 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2023-11-27T15:32:59.279 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2023-11-27T15:32:59.279 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:32:59.279 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:32:59.279 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:32:59.279 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:32:59.280 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 239448, 2023-11-27T15:32:59.280 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 79, 2023-11-27T15:32:59.280 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:32:59.280 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2023-11-27T15:32:59.280 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2023-11-27T15:32:59.280 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2023-11-27T15:32:59.280 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:32:59.281 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2023-11-27T15:32:59.281 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:32:59.281 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2023-11-27T15:32:59.281 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 8792, 2023-11-27T15:32:59.281 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 168, 2023-11-27T15:32:59.281 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:32:59.282 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:32:59.282 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2023-11-27T15:32:59.282 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2023-11-27T15:32:59.282 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 14057288, 2023-11-27T15:32:59.282 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 79756, 2023-11-27T15:32:59.282 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2023-11-27T15:32:59.282 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2023-11-27T15:32:59.283 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2023-11-27T15:32:59.283 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2023-11-27T15:32:59.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.283 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2023-11-27T15:32:59.283 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2023-11-27T15:32:59.283 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2023-11-27T15:32:59.283 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 1880, 2023-11-27T15:32:59.284 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 21, 2023-11-27T15:32:59.284 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 3, 2023-11-27T15:32:59.284 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 1880, 2023-11-27T15:32:59.284 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2023-11-27T15:32:59.284 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1880, 2023-11-27T15:32:59.284 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 4.457956498, 2023-11-27T15:32:59.284 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.002371253 2023-11-27T15:32:59.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.285 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2023-11-27T15:32:59.285 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2023-11-27T15:32:59.285 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 1880, 2023-11-27T15:32:59.285 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 1881 2023-11-27T15:32:59.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.285 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 1880, 2023-11-27T15:32:59.286 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 1880, 2023-11-27T15:32:59.286 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 0, 2023-11-27T15:32:59.286 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2023-11-27T15:32:59.286 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2023-11-27T15:32:59.286 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 1139, 2023-11-27T15:32:59.286 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 0, 2023-11-27T15:32:59.286 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 739, 2023-11-27T15:32:59.287 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 0, 2023-11-27T15:32:59.287 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 0, 2023-11-27T15:32:59.287 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2023-11-27T15:32:59.287 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2023-11-27T15:32:59.287 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 0, 2023-11-27T15:32:59.287 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 0, 2023-11-27T15:32:59.287 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 0, 2023-11-27T15:32:59.287 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2023-11-27T15:32:59.288 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2023-11-27T15:32:59.288 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2023-11-27T15:32:59.288 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 1, 2023-11-27T15:32:59.288 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 0, 2023-11-27T15:32:59.288 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:32:59.288 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2023-11-27T15:32:59.289 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:32:59.289 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2023-11-27T15:32:59.289 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2023-11-27T15:32:59.289 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2023-11-27T15:32:59.289 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:32:59.289 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2023-11-27T15:32:59.289 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:32:59.290 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 0, 2023-11-27T15:32:59.290 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2023-11-27T15:32:59.290 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2023-11-27T15:32:59.290 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2023-11-27T15:32:59.290 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2023-11-27T15:32:59.290 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2023-11-27T15:32:59.290 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2023-11-27T15:32:59.291 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2023-11-27T15:32:59.291 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2023-11-27T15:32:59.291 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2023-11-27T15:32:59.291 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2023-11-27T15:32:59.291 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2023-11-27T15:32:59.291 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2023-11-27T15:32:59.291 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2023-11-27T15:32:59.292 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2023-11-27T15:32:59.292 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2023-11-27T15:32:59.292 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2023-11-27T15:32:59.292 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 94, 2023-11-27T15:32:59.292 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2023-11-27T15:32:59.292 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 6, 2023-11-27T15:32:59.292 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 11, 2023-11-27T15:32:59.293 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 11, 2023-11-27T15:32:59.293 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 0, 2023-11-27T15:32:59.293 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2023-11-27T15:32:59.293 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 0, 2023-11-27T15:32:59.293 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 2, 2023-11-27T15:32:59.293 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 0 2023-11-27T15:32:59.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.294 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2023-11-27T15:32:59.294 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 0, 2023-11-27T15:32:59.294 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:32:59.294 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2023-11-27T15:32:59.294 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2023-11-27T15:32:59.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.294 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2023-11-27T15:32:59.295 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 0 2023-11-27T15:32:59.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.295 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2023-11-27T15:32:59.295 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2023-11-27T15:32:59.295 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 0, 2023-11-27T15:32:59.295 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2023-11-27T15:32:59.295 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 0, 2023-11-27T15:32:59.296 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 0, 2023-11-27T15:32:59.296 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2023-11-27T15:32:59.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.296 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:32:59.296 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:32:59.296 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-11-27T15:32:59.296 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:32:59.297 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 2069, 2023-11-27T15:32:59.297 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 174718329, 2023-11-27T15:32:59.297 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:59.297 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 2069, 2023-11-27T15:32:59.297 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-11-27T15:32:59.297 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-11-27T15:32:59.297 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 2069, 2023-11-27T15:32:59.297 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 174718329, 2023-11-27T15:32:59.298 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:32:59.298 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.298 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.298 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:59.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.299 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2023-11-27T15:32:59.299 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:32:59.299 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-11-27T15:32:59.299 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:32:59.299 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:32:59.299 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:32:59.299 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:59.299 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:59.300 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 1880, 2023-11-27T15:32:59.300 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 173870264, 2023-11-27T15:32:59.300 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 1880, 2023-11-27T15:32:59.300 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 173870264, 2023-11-27T15:32:59.300 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:32:59.300 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.300 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.301 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.301 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:59.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.301 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2023-11-27T15:32:59.301 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:32:59.301 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2023-11-27T15:32:59.301 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:32:59.302 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:32:59.302 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:32:59.302 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:59.302 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:59.302 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 1880, 2023-11-27T15:32:59.302 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 1880, 2023-11-27T15:32:59.302 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 1880, 2023-11-27T15:32:59.303 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 1880, 2023-11-27T15:32:59.303 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:32:59.303 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.303 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.303 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:59.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.303 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:32:59.304 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:32:59.304 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-11-27T15:32:59.304 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:32:59.304 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:32:59.304 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:32:59.304 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:59.304 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:59.305 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-11-27T15:32:59.305 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-11-27T15:32:59.305 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-11-27T15:32:59.305 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-11-27T15:32:59.305 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:32:59.305 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.305 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.305 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.306 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:59.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:32:59.306 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x563938000000": { 2023-11-27T15:32:59.306 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:32:59.306 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-11-27T15:32:59.306 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:32:59.306 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:32:59.307 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:32:59.307 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:32:59.307 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:32:59.307 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-11-27T15:32:59.307 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-11-27T15:32:59.307 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-11-27T15:32:59.307 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-11-27T15:32:59.308 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:32:59.308 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:32:59.308 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:32:59.308 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:32:59.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:59.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:32:59.308 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-27T15:32:59.617 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:32:59 smithi060 ceph-mon[131518]: pgmap v574: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:32:59.653 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-11-27T15:32:59.654 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-11-27T15:32:59.654 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-11-27T15:32:59.654 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-11-27T15:32:59.654 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-11-27T15:32:59.654 DEBUG:tasks.check_counter:Getting stats from i 2023-11-27T15:32:59.655 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.i perf dump 2023-11-27T15:32:59.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:32:59 smithi008 ceph-mon[128788]: pgmap v574: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:32:59.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:32:59 smithi029 ceph-mon[131166]: pgmap v574: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:33:00.307 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:33:01.622 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-27T15:33:01.622 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:33:01.622 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 141, 2023-11-27T15:33:01.623 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 143, 2023-11-27T15:33:01.623 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 4239159, 2023-11-27T15:33:01.623 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 43861, 2023-11-27T15:33:01.623 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 7, 2023-11-27T15:33:01.623 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 3, 2023-11-27T15:33:01.623 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.022291513, 2023-11-27T15:33:01.624 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.004541195, 2023-11-27T15:33:01.624 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.008503800, 2023-11-27T15:33:01.624 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.006361392, 2023-11-27T15:33:01.624 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:33:01.624 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 141, 2023-11-27T15:33:01.625 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.013174028, 2023-11-27T15:33:01.625 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000093432 2023-11-27T15:33:01.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.625 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:33:01.625 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.626 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.626 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.626 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:33:01.626 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:33:01.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.627 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:33:01.627 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 138, 2023-11-27T15:33:01.627 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 137, 2023-11-27T15:33:01.627 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 2098677, 2023-11-27T15:33:01.628 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 54002, 2023-11-27T15:33:01.628 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2023-11-27T15:33:01.628 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 3, 2023-11-27T15:33:01.628 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.018847491, 2023-11-27T15:33:01.628 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.005740282, 2023-11-27T15:33:01.629 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.008559932, 2023-11-27T15:33:01.629 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.004249135, 2023-11-27T15:33:01.629 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:33:01.629 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 136, 2023-11-27T15:33:01.629 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.014867105, 2023-11-27T15:33:01.630 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000109316 2023-11-27T15:33:01.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.630 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:33:01.630 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.630 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.631 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.631 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 166416, 2023-11-27T15:33:01.631 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 29376 2023-11-27T15:33:01.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.632 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:33:01.632 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 141, 2023-11-27T15:33:01.632 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 177, 2023-11-27T15:33:01.632 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 5900072, 2023-11-27T15:33:01.632 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 166770, 2023-11-27T15:33:01.632 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 5, 2023-11-27T15:33:01.633 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 4, 2023-11-27T15:33:01.633 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.020043647, 2023-11-27T15:33:01.633 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.004999724, 2023-11-27T15:33:01.633 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.009263622, 2023-11-27T15:33:01.633 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.003995758, 2023-11-27T15:33:01.634 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:33:01.634 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 176, 2023-11-27T15:33:01.634 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.017553519, 2023-11-27T15:33:01.634 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000099735 2023-11-27T15:33:01.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.634 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:33:01.635 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.635 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.635 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.635 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-27T15:33:01.636 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 124464 2023-11-27T15:33:01.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.636 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2023-11-27T15:33:01.636 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 1, 2023-11-27T15:33:01.636 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2023-11-27T15:33:01.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.637 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-MDSRank": { 2023-11-27T15:33:01.637 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-11-27T15:33:01.637 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-11-27T15:33:01.637 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 594, 2023-11-27T15:33:01.637 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.136132639, 2023-11-27T15:33:01.637 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000229179 2023-11-27T15:33:01.638 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:01.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.638 INFO:teuthology.orchestra.run.smithi060.stdout: "finisher-PurgeQueue": { 2023-11-27T15:33:01.638 INFO:teuthology.orchestra.run.smithi060.stdout: "queue_len": 0, 2023-11-27T15:33:01.638 INFO:teuthology.orchestra.run.smithi060.stdout: "complete_latency": { 2023-11-27T15:33:01.639 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.639 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.639 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.639 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:01.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.640 INFO:teuthology.orchestra.run.smithi060.stdout: "mds": { 2023-11-27T15:33:01.640 INFO:teuthology.orchestra.run.smithi060.stdout: "request": 0, 2023-11-27T15:33:01.640 INFO:teuthology.orchestra.run.smithi060.stdout: "reply": 0, 2023-11-27T15:33:01.640 INFO:teuthology.orchestra.run.smithi060.stdout: "reply_latency": { 2023-11-27T15:33:01.640 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.640 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.641 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.641 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_reply": 0, 2023-11-27T15:33:01.641 INFO:teuthology.orchestra.run.smithi060.stdout: "forward": 0, 2023-11-27T15:33:01.641 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_complete": 0, 2023-11-27T15:33:01.641 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_fetch_keys": 0, 2023-11-27T15:33:01.642 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_commit": 0, 2023-11-27T15:33:01.642 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_split": 0, 2023-11-27T15:33:01.642 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_merge": 0, 2023-11-27T15:33:01.642 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes": 1824, 2023-11-27T15:33:01.642 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_top": 1262, 2023-11-27T15:33:01.643 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_bottom": 551, 2023-11-27T15:33:01.643 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pin_tail": 11, 2023-11-27T15:33:01.643 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_pinned": 20, 2023-11-27T15:33:01.643 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_expired": 5609, 2023-11-27T15:33:01.643 INFO:teuthology.orchestra.run.smithi060.stdout: "inodes_with_caps": 0, 2023-11-27T15:33:01.644 INFO:teuthology.orchestra.run.smithi060.stdout: "caps": 0, 2023-11-27T15:33:01.644 INFO:teuthology.orchestra.run.smithi060.stdout: "subtrees": 2, 2023-11-27T15:33:01.644 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse": 0, 2023-11-27T15:33:01.644 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_hit": 0, 2023-11-27T15:33:01.644 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_forward": 0, 2023-11-27T15:33:01.644 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_discover": 0, 2023-11-27T15:33:01.645 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:33:01.645 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_remote_ino": 0, 2023-11-27T15:33:01.645 INFO:teuthology.orchestra.run.smithi060.stdout: "traverse_lock": 0, 2023-11-27T15:33:01.645 INFO:teuthology.orchestra.run.smithi060.stdout: "load_cent": 0, 2023-11-27T15:33:01.645 INFO:teuthology.orchestra.run.smithi060.stdout: "q": 0, 2023-11-27T15:33:01.646 INFO:teuthology.orchestra.run.smithi060.stdout: "exported": 0, 2023-11-27T15:33:01.646 INFO:teuthology.orchestra.run.smithi060.stdout: "exported_inodes": 0, 2023-11-27T15:33:01.646 INFO:teuthology.orchestra.run.smithi060.stdout: "imported": 0, 2023-11-27T15:33:01.646 INFO:teuthology.orchestra.run.smithi060.stdout: "imported_inodes": 0, 2023-11-27T15:33:01.646 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_dir_fetch": 0, 2023-11-27T15:33:01.647 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:33:01.647 INFO:teuthology.orchestra.run.smithi060.stdout: "openino_peer_discover": 0, 2023-11-27T15:33:01.647 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rfiles": 4, 2023-11-27T15:33:01.647 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rbytes": 246, 2023-11-27T15:33:01.647 INFO:teuthology.orchestra.run.smithi060.stdout: "root_rsnaps": 0, 2023-11-27T15:33:01.647 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:33:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_set_tag": 0, 2023-11-27T15:33:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:33:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:33:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:33:01.648 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:33:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:33:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: "scrub_file_inodes": 0, 2023-11-27T15:33:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_inode_file_caps": 0, 2023-11-27T15:33:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_revoke": 0, 2023-11-27T15:33:01.649 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_grant": 0, 2023-11-27T15:33:01.650 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:33:01.650 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:33:01.650 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:33:01.650 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps": 0, 2023-11-27T15:33:01.650 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_caps_dirty": 0, 2023-11-27T15:33:01.651 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_cap_release": 0, 2023-11-27T15:33:01.651 INFO:teuthology.orchestra.run.smithi060.stdout: "process_request_cap_release": 0 2023-11-27T15:33:01.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.651 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache": { 2023-11-27T15:33:01.651 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update": 0, 2023-11-27T15:33:01.651 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_update_receipt": 0, 2023-11-27T15:33:01.652 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_try_discover": 0, 2023-11-27T15:33:01.652 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_send_discover": 0, 2023-11-27T15:33:01.652 INFO:teuthology.orchestra.run.smithi060.stdout: "dir_handle_discover": 0, 2023-11-27T15:33:01.652 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays": 0, 2023-11-27T15:33:01.652 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_delayed": 0, 2023-11-27T15:33:01.652 INFO:teuthology.orchestra.run.smithi060.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:33:01.653 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_created": 0, 2023-11-27T15:33:01.653 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_enqueued": 0, 2023-11-27T15:33:01.653 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_reintegrated": 0, 2023-11-27T15:33:01.653 INFO:teuthology.orchestra.run.smithi060.stdout: "strays_migrated": 0, 2023-11-27T15:33:01.653 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_processing": 0, 2023-11-27T15:33:01.653 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:33:01.654 INFO:teuthology.orchestra.run.smithi060.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:33:01.654 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_started": 0, 2023-11-27T15:33:01.654 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_completed": 0, 2023-11-27T15:33:01.654 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:33:01.654 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_exportdir": 0, 2023-11-27T15:33:01.654 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_flush": 0, 2023-11-27T15:33:01.655 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragmentdir": 0, 2023-11-27T15:33:01.655 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_fragstats": 0, 2023-11-27T15:33:01.655 INFO:teuthology.orchestra.run.smithi060.stdout: "ireq_inodestats": 0 2023-11-27T15:33:01.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.655 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log": { 2023-11-27T15:33:01.655 INFO:teuthology.orchestra.run.smithi060.stdout: "evadd": 0, 2023-11-27T15:33:01.656 INFO:teuthology.orchestra.run.smithi060.stdout: "evex": 0, 2023-11-27T15:33:01.656 INFO:teuthology.orchestra.run.smithi060.stdout: "evtrm": 0, 2023-11-27T15:33:01.656 INFO:teuthology.orchestra.run.smithi060.stdout: "ev": 0, 2023-11-27T15:33:01.656 INFO:teuthology.orchestra.run.smithi060.stdout: "evexg": 0, 2023-11-27T15:33:01.656 INFO:teuthology.orchestra.run.smithi060.stdout: "evexd": 0, 2023-11-27T15:33:01.656 INFO:teuthology.orchestra.run.smithi060.stdout: "segadd": 0, 2023-11-27T15:33:01.657 INFO:teuthology.orchestra.run.smithi060.stdout: "segex": 0, 2023-11-27T15:33:01.657 INFO:teuthology.orchestra.run.smithi060.stdout: "segtrm": 0, 2023-11-27T15:33:01.657 INFO:teuthology.orchestra.run.smithi060.stdout: "seg": 2, 2023-11-27T15:33:01.657 INFO:teuthology.orchestra.run.smithi060.stdout: "segexg": 0, 2023-11-27T15:33:01.657 INFO:teuthology.orchestra.run.smithi060.stdout: "segexd": 0, 2023-11-27T15:33:01.657 INFO:teuthology.orchestra.run.smithi060.stdout: "expos": 169355884, 2023-11-27T15:33:01.658 INFO:teuthology.orchestra.run.smithi060.stdout: "wrpos": 0, 2023-11-27T15:33:01.658 INFO:teuthology.orchestra.run.smithi060.stdout: "rdpos": 169437395, 2023-11-27T15:33:01.658 INFO:teuthology.orchestra.run.smithi060.stdout: "jlat": { 2023-11-27T15:33:01.658 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.658 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.659 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.659 INFO:teuthology.orchestra.run.smithi060.stdout: "replayed": 5536 2023-11-27T15:33:01.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.659 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mem": { 2023-11-27T15:33:01.659 INFO:teuthology.orchestra.run.smithi060.stdout: "ino": 22, 2023-11-27T15:33:01.659 INFO:teuthology.orchestra.run.smithi060.stdout: "ino+": 2461, 2023-11-27T15:33:01.660 INFO:teuthology.orchestra.run.smithi060.stdout: "ino-": 2439, 2023-11-27T15:33:01.660 INFO:teuthology.orchestra.run.smithi060.stdout: "dir": 30, 2023-11-27T15:33:01.660 INFO:teuthology.orchestra.run.smithi060.stdout: "dir+": 340, 2023-11-27T15:33:01.660 INFO:teuthology.orchestra.run.smithi060.stdout: "dir-": 310, 2023-11-27T15:33:01.660 INFO:teuthology.orchestra.run.smithi060.stdout: "dn": 1824, 2023-11-27T15:33:01.660 INFO:teuthology.orchestra.run.smithi060.stdout: "dn+": 7589, 2023-11-27T15:33:01.661 INFO:teuthology.orchestra.run.smithi060.stdout: "dn-": 5765, 2023-11-27T15:33:01.661 INFO:teuthology.orchestra.run.smithi060.stdout: "cap": 0, 2023-11-27T15:33:01.661 INFO:teuthology.orchestra.run.smithi060.stdout: "cap+": 0, 2023-11-27T15:33:01.661 INFO:teuthology.orchestra.run.smithi060.stdout: "cap-": 0, 2023-11-27T15:33:01.661 INFO:teuthology.orchestra.run.smithi060.stdout: "rss": 54576, 2023-11-27T15:33:01.661 INFO:teuthology.orchestra.run.smithi060.stdout: "heap": 207156 2023-11-27T15:33:01.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.662 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_server": { 2023-11-27T15:33:01.662 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_client_request": 0, 2023-11-27T15:33:01.662 INFO:teuthology.orchestra.run.smithi060.stdout: "dispatch_server_request": 0, 2023-11-27T15:33:01.662 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_request": 0, 2023-11-27T15:33:01.662 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_client_session": 0, 2023-11-27T15:33:01.663 INFO:teuthology.orchestra.run.smithi060.stdout: "handle_peer_request": 0, 2023-11-27T15:33:01.663 INFO:teuthology.orchestra.run.smithi060.stdout: "req_create_latency": { 2023-11-27T15:33:01.663 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.663 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.663 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.664 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getattr_latency": { 2023-11-27T15:33:01.664 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.664 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.664 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.664 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getfilelock_latency": { 2023-11-27T15:33:01.665 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.665 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.665 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.665 INFO:teuthology.orchestra.run.smithi060.stdout: "req_link_latency": { 2023-11-27T15:33:01.665 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.665 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.666 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.666 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookup_latency": { 2023-11-27T15:33:01.666 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.666 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.666 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.667 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookuphash_latency": { 2023-11-27T15:33:01.667 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.667 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.667 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.667 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupino_latency": { 2023-11-27T15:33:01.668 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.668 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.668 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.668 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupname_latency": { 2023-11-27T15:33:01.668 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.669 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.669 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.669 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupparent_latency": { 2023-11-27T15:33:01.669 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.669 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.670 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.670 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lookupsnap_latency": { 2023-11-27T15:33:01.670 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.670 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.670 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.671 INFO:teuthology.orchestra.run.smithi060.stdout: "req_lssnap_latency": { 2023-11-27T15:33:01.671 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.671 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.671 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.671 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mkdir_latency": { 2023-11-27T15:33:01.672 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.672 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.672 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.672 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mknod_latency": { 2023-11-27T15:33:01.672 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.673 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.673 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.673 INFO:teuthology.orchestra.run.smithi060.stdout: "req_mksnap_latency": { 2023-11-27T15:33:01.673 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.673 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.674 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.674 INFO:teuthology.orchestra.run.smithi060.stdout: "req_open_latency": { 2023-11-27T15:33:01.674 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.674 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.674 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.675 INFO:teuthology.orchestra.run.smithi060.stdout: "req_readdir_latency": { 2023-11-27T15:33:01.675 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.675 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.675 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.675 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rename_latency": { 2023-11-27T15:33:01.676 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.676 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.676 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.676 INFO:teuthology.orchestra.run.smithi060.stdout: "req_renamesnap_latency": { 2023-11-27T15:33:01.676 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.676 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.677 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.677 INFO:teuthology.orchestra.run.smithi060.stdout: "req_snapdiff_latency": { 2023-11-27T15:33:01.677 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.677 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.677 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.678 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmdir_latency": { 2023-11-27T15:33:01.678 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.678 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.678 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.679 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmsnap_latency": { 2023-11-27T15:33:01.679 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.679 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.679 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.679 INFO:teuthology.orchestra.run.smithi060.stdout: "req_rmxattr_latency": { 2023-11-27T15:33:01.680 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.680 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.680 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.680 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setattr_latency": { 2023-11-27T15:33:01.680 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.680 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.681 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.681 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setdirlayout_latency": { 2023-11-27T15:33:01.681 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.681 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.681 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.682 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setfilelock_latency": { 2023-11-27T15:33:01.682 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.682 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.682 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.683 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setlayout_latency": { 2023-11-27T15:33:01.683 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.683 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.683 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.683 INFO:teuthology.orchestra.run.smithi060.stdout: "req_setxattr_latency": { 2023-11-27T15:33:01.683 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.684 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.684 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.684 INFO:teuthology.orchestra.run.smithi060.stdout: "req_symlink_latency": { 2023-11-27T15:33:01.684 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.684 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.684 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.685 INFO:teuthology.orchestra.run.smithi060.stdout: "req_unlink_latency": { 2023-11-27T15:33:01.685 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.685 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.685 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.685 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:33:01.686 INFO:teuthology.orchestra.run.smithi060.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:33:01.686 INFO:teuthology.orchestra.run.smithi060.stdout: "req_getvxattr_latency": { 2023-11-27T15:33:01.686 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.686 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.686 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.686 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:01.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.687 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessions": { 2023-11-27T15:33:01.687 INFO:teuthology.orchestra.run.smithi060.stdout: "session_count": 1, 2023-11-27T15:33:01.687 INFO:teuthology.orchestra.run.smithi060.stdout: "session_add": 1, 2023-11-27T15:33:01.687 INFO:teuthology.orchestra.run.smithi060.stdout: "session_remove": 0, 2023-11-27T15:33:01.687 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_open": 0, 2023-11-27T15:33:01.687 INFO:teuthology.orchestra.run.smithi060.stdout: "sessions_stale": 0, 2023-11-27T15:33:01.688 INFO:teuthology.orchestra.run.smithi060.stdout: "total_load": 0, 2023-11-27T15:33:01.688 INFO:teuthology.orchestra.run.smithi060.stdout: "average_load": 0, 2023-11-27T15:33:01.688 INFO:teuthology.orchestra.run.smithi060.stdout: "avg_session_uptime": 2463 2023-11-27T15:33:01.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.688 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2023-11-27T15:33:01.688 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:33:01.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2023-11-27T15:33:01.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:33:01.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:33:01.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:33:01.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:33:01.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:33:01.689 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:33:01.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:33:01.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:33:01.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:33:01.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:33:01.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:33:01.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:33:01.690 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:33:01.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2023-11-27T15:33:01.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:33:01.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2023-11-27T15:33:01.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:33:01.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:33:01.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:33:01.691 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:33:01.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:33:01.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:33:01.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:33:01.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2023-11-27T15:33:01.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:33:01.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:33:01.692 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:33:01.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2023-11-27T15:33:01.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2023-11-27T15:33:01.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2023-11-27T15:33:01.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:33:01.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:33:01.693 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:33:01.694 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:33:01.694 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 204769, 2023-11-27T15:33:01.694 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 58, 2023-11-27T15:33:01.694 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:33:01.694 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2023-11-27T15:33:01.694 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2023-11-27T15:33:01.694 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2023-11-27T15:33:01.695 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:33:01.695 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2023-11-27T15:33:01.695 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:33:01.695 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2023-11-27T15:33:01.695 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 8792, 2023-11-27T15:33:01.695 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 168, 2023-11-27T15:33:01.695 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:33:01.696 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:33:01.696 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2023-11-27T15:33:01.696 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2023-11-27T15:33:01.696 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 1362858, 2023-11-27T15:33:01.696 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 6598, 2023-11-27T15:33:01.696 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2023-11-27T15:33:01.696 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2023-11-27T15:33:01.697 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2023-11-27T15:33:01.697 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2023-11-27T15:33:01.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.697 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter": { 2023-11-27T15:33:01.697 INFO:teuthology.orchestra.run.smithi060.stdout: "op_active": 0, 2023-11-27T15:33:01.697 INFO:teuthology.orchestra.run.smithi060.stdout: "op_laggy": 0, 2023-11-27T15:33:01.697 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send": 356, 2023-11-27T15:33:01.698 INFO:teuthology.orchestra.run.smithi060.stdout: "op_send_bytes": 26, 2023-11-27T15:33:01.698 INFO:teuthology.orchestra.run.smithi060.stdout: "op_resend": 4, 2023-11-27T15:33:01.698 INFO:teuthology.orchestra.run.smithi060.stdout: "op_reply": 356, 2023-11-27T15:33:01.698 INFO:teuthology.orchestra.run.smithi060.stdout: "op_latency": { 2023-11-27T15:33:01.698 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 356, 2023-11-27T15:33:01.698 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.464995493, 2023-11-27T15:33:01.698 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.001306167 2023-11-27T15:33:01.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.699 INFO:teuthology.orchestra.run.smithi060.stdout: "op_inflight": 0, 2023-11-27T15:33:01.699 INFO:teuthology.orchestra.run.smithi060.stdout: "oplen_avg": { 2023-11-27T15:33:01.699 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 356, 2023-11-27T15:33:01.699 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 357 2023-11-27T15:33:01.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.699 INFO:teuthology.orchestra.run.smithi060.stdout: "op": 356, 2023-11-27T15:33:01.700 INFO:teuthology.orchestra.run.smithi060.stdout: "op_r": 356, 2023-11-27T15:33:01.700 INFO:teuthology.orchestra.run.smithi060.stdout: "op_w": 0, 2023-11-27T15:33:01.700 INFO:teuthology.orchestra.run.smithi060.stdout: "op_rmw": 0, 2023-11-27T15:33:01.700 INFO:teuthology.orchestra.run.smithi060.stdout: "op_pg": 0, 2023-11-27T15:33:01.700 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_stat": 204, 2023-11-27T15:33:01.700 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_create": 0, 2023-11-27T15:33:01.700 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_read": 149, 2023-11-27T15:33:01.701 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_write": 0, 2023-11-27T15:33:01.701 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writefull": 0, 2023-11-27T15:33:01.701 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_writesame": 0, 2023-11-27T15:33:01.701 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_append": 0, 2023-11-27T15:33:01.701 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_zero": 0, 2023-11-27T15:33:01.701 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_truncate": 0, 2023-11-27T15:33:01.701 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_delete": 0, 2023-11-27T15:33:01.702 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_mapext": 0, 2023-11-27T15:33:01.702 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_sparse_read": 0, 2023-11-27T15:33:01.702 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_clonerange": 0, 2023-11-27T15:33:01.702 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_getxattr": 2, 2023-11-27T15:33:01.702 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_setxattr": 0, 2023-11-27T15:33:01.702 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:33:01.702 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_rmxattr": 0, 2023-11-27T15:33:01.703 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:33:01.703 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_call": 0, 2023-11-27T15:33:01.703 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_watch": 0, 2023-11-27T15:33:01.703 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_notify": 0, 2023-11-27T15:33:01.703 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:33:01.703 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls": 0, 2023-11-27T15:33:01.703 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:33:01.704 INFO:teuthology.orchestra.run.smithi060.stdout: "osdop_other": 0, 2023-11-27T15:33:01.704 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_active": 0, 2023-11-27T15:33:01.704 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_send": 0, 2023-11-27T15:33:01.704 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_resend": 0, 2023-11-27T15:33:01.704 INFO:teuthology.orchestra.run.smithi060.stdout: "linger_ping": 0, 2023-11-27T15:33:01.704 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_active": 0, 2023-11-27T15:33:01.705 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_send": 0, 2023-11-27T15:33:01.705 INFO:teuthology.orchestra.run.smithi060.stdout: "poolop_resend": 0, 2023-11-27T15:33:01.705 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_active": 0, 2023-11-27T15:33:01.705 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_send": 0, 2023-11-27T15:33:01.705 INFO:teuthology.orchestra.run.smithi060.stdout: "poolstat_resend": 0, 2023-11-27T15:33:01.705 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_active": 0, 2023-11-27T15:33:01.705 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_send": 0, 2023-11-27T15:33:01.706 INFO:teuthology.orchestra.run.smithi060.stdout: "statfs_resend": 0, 2023-11-27T15:33:01.706 INFO:teuthology.orchestra.run.smithi060.stdout: "command_active": 0, 2023-11-27T15:33:01.706 INFO:teuthology.orchestra.run.smithi060.stdout: "command_send": 0, 2023-11-27T15:33:01.706 INFO:teuthology.orchestra.run.smithi060.stdout: "command_resend": 0, 2023-11-27T15:33:01.706 INFO:teuthology.orchestra.run.smithi060.stdout: "map_epoch": 0, 2023-11-27T15:33:01.706 INFO:teuthology.orchestra.run.smithi060.stdout: "map_full": 0, 2023-11-27T15:33:01.707 INFO:teuthology.orchestra.run.smithi060.stdout: "map_inc": 0, 2023-11-27T15:33:01.707 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_sessions": 7, 2023-11-27T15:33:01.707 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_open": 7, 2023-11-27T15:33:01.707 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_session_close": 0, 2023-11-27T15:33:01.707 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_laggy": 0, 2023-11-27T15:33:01.707 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_wr": 0, 2023-11-27T15:33:01.707 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_rd": 2, 2023-11-27T15:33:01.708 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_del": 0 2023-11-27T15:33:01.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.708 INFO:teuthology.orchestra.run.smithi060.stdout: "oft": { 2023-11-27T15:33:01.708 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_objs": 0, 2023-11-27T15:33:01.708 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:33:01.708 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_updates": 0, 2023-11-27T15:33:01.709 INFO:teuthology.orchestra.run.smithi060.stdout: "omap_total_removes": 0 2023-11-27T15:33:01.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.709 INFO:teuthology.orchestra.run.smithi060.stdout: "osd-slow-ops": { 2023-11-27T15:33:01.709 INFO:teuthology.orchestra.run.smithi060.stdout: "slow_ops_count": 0 2023-11-27T15:33:01.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.709 INFO:teuthology.orchestra.run.smithi060.stdout: "purge_queue": { 2023-11-27T15:33:01.709 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops": 0, 2023-11-27T15:33:01.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_ops_high_water": 0, 2023-11-27T15:33:01.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing": 0, 2023-11-27T15:33:01.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executing_high_water": 0, 2023-11-27T15:33:01.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_executed": 0, 2023-11-27T15:33:01.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pq_item_in_journal": 0 2023-11-27T15:33:01.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.710 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:33:01.710 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:33:01.711 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-11-27T15:33:01.711 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:33:01.711 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 416, 2023-11-27T15:33:01.711 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 12194443, 2023-11-27T15:33:01.711 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:33:01.711 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 416, 2023-11-27T15:33:01.711 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-11-27T15:33:01.712 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-11-27T15:33:01.712 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 416, 2023-11-27T15:33:01.712 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 12194443, 2023-11-27T15:33:01.712 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:33:01.712 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.712 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.713 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.713 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:01.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.713 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_bytes": { 2023-11-27T15:33:01.713 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:33:01.713 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-11-27T15:33:01.713 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:33:01.713 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:33:01.714 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:33:01.714 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:33:01.714 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:33:01.714 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 356, 2023-11-27T15:33:01.714 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 11965004, 2023-11-27T15:33:01.714 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 356, 2023-11-27T15:33:01.715 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 11965004, 2023-11-27T15:33:01.715 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:33:01.715 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.715 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.715 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.715 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:01.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.715 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-objecter_ops": { 2023-11-27T15:33:01.716 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:33:01.716 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 1024, 2023-11-27T15:33:01.716 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:33:01.716 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:33:01.716 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:33:01.716 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:33:01.716 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:33:01.717 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 356, 2023-11-27T15:33:01.717 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 356, 2023-11-27T15:33:01.717 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 356, 2023-11-27T15:33:01.717 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 356, 2023-11-27T15:33:01.717 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:33:01.717 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.717 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.718 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.718 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:01.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.718 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:33:01.718 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:33:01.718 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-11-27T15:33:01.719 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:33:01.719 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:33:01.719 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:33:01.719 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:33:01.719 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:33:01.719 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-11-27T15:33:01.719 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-11-27T15:33:01.720 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-11-27T15:33:01.720 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-11-27T15:33:01.720 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:33:01.720 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.720 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.720 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.720 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:01.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:01.721 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-write_buf_throttle-0x556500670000": { 2023-11-27T15:33:01.721 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:33:01.721 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 3758096384, 2023-11-27T15:33:01.721 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:33:01.721 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 0, 2023-11-27T15:33:01.721 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 0, 2023-11-27T15:33:01.721 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:33:01.722 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 0, 2023-11-27T15:33:01.722 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-11-27T15:33:01.722 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-11-27T15:33:01.722 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 0, 2023-11-27T15:33:01.722 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 0, 2023-11-27T15:33:01.722 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:33:01.722 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:01.723 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:01.723 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:01.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:01.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:01.723 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-27T15:33:01.724 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:01 smithi060 ceph-mon[131518]: pgmap v575: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:33:01.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:01 smithi008 ceph-mon[128788]: pgmap v575: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:33:01.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:01 smithi029 ceph-mon[131166]: pgmap v575: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:33:01.969 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2023-11-27T15:33:01.969 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2023-11-27T15:33:01.969 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2023-11-27T15:33:01.970 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2023-11-27T15:33:01.970 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2023-11-27T15:33:01.970 DEBUG:tasks.check_counter:Getting stats from l 2023-11-27T15:33:01.970 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.l perf dump 2023-11-27T15:33:02.623 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:33:03.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:03 smithi060 ceph-mon[131518]: pgmap v576: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:33:03.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:03 smithi008 ceph-mon[128788]: pgmap v576: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:33:03.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:03 smithi029 ceph-mon[131166]: pgmap v576: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:33:03.898 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2023-11-27T15:33:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:33:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 170, 2023-11-27T15:33:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 167, 2023-11-27T15:33:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 457808, 2023-11-27T15:33:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 94544, 2023-11-27T15:33:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 4, 2023-11-27T15:33:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 0, 2023-11-27T15:33:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.029806776, 2023-11-27T15:33:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.010575398, 2023-11-27T15:33:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.013453560, 2023-11-27T15:33:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.004878161, 2023-11-27T15:33:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:33:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 166, 2023-11-27T15:33:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.010880225, 2023-11-27T15:33:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000065543 2023-11-27T15:33:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:33:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 457808, 2023-11-27T15:33:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 94544 2023-11-27T15:33:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:33:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 1, 2023-11-27T15:33:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 126, 2023-11-27T15:33:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 160, 2023-11-27T15:33:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 97936, 2023-11-27T15:33:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 3, 2023-11-27T15:33:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 2, 2023-11-27T15:33:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.008054747, 2023-11-27T15:33:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.005432742, 2023-11-27T15:33:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.000955183, 2023-11-27T15:33:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:33:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:33:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 124, 2023-11-27T15:33:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.007988661, 2023-11-27T15:33:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000064424 2023-11-27T15:33:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:33:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:33:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 97936 2023-11-27T15:33:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:33:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_messages": 4, 2023-11-27T15:33:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages": 4, 2023-11-27T15:33:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_bytes": 2992, 2023-11-27T15:33:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_bytes": 448, 2023-11-27T15:33:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_created_connections": 2, 2023-11-27T15:33:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_active_connections": 1, 2023-11-27T15:33:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_total_time": 0.000740685, 2023-11-27T15:33:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_send_time": 0.000042293, 2023-11-27T15:33:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_recv_time": 0.000660935, 2023-11-27T15:33:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:33:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:33:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 3, 2023-11-27T15:33:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000876291, 2023-11-27T15:33:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000292097 2023-11-27T15:33:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:33:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-27T15:33:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "msgr_send_encrypted_bytes": 448 2023-11-27T15:33:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "cct": { 2023-11-27T15:33:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "total_workers": 0, 2023-11-27T15:33:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "unhealthy_workers": 0 2023-11-27T15:33:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool": { 2023-11-27T15:33:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:33:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "bloom_filter_items": 0, 2023-11-27T15:33:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:33:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:33:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:33:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:33:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:33:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:33:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:33:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:33:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:33:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:33:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:33:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:33:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:33:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_items": 0, 2023-11-27T15:33:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:33:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blob_items": 0, 2023-11-27T15:33:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:33:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:33:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:33:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:33:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:33:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:33:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:33:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_txc_items": 0, 2023-11-27T15:33:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:33:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:33:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:33:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_writing_items": 0, 2023-11-27T15:33:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_bytes": 0, 2023-11-27T15:33:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_items": 0, 2023-11-27T15:33:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:33:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:33:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:33:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:33:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_bytes": 149007, 2023-11-27T15:33:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_anon_items": 30, 2023-11-27T15:33:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:33:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "buffer_meta_items": 0, 2023-11-27T15:33:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bytes": 0, 2023-11-27T15:33:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_items": 0, 2023-11-27T15:33:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:33:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mapbl_items": 0, 2023-11-27T15:33:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:33:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pglog_items": 0, 2023-11-27T15:33:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_bytes": 0, 2023-11-27T15:33:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_items": 0, 2023-11-27T15:33:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:33:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:33:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_bytes": 0, 2023-11-27T15:33:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "pgmap_items": 0, 2023-11-27T15:33:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_bytes": 928, 2023-11-27T15:33:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_co_items": 2, 2023-11-27T15:33:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_bytes": 0, 2023-11-27T15:33:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_1_items": 0, 2023-11-27T15:33:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_bytes": 0, 2023-11-27T15:33:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "unittest_2_items": 0 2023-11-27T15:33:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2023-11-27T15:33:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:33:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "val": 0, 2023-11-27T15:33:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "max": 104857600, 2023-11-27T15:33:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "get_started": 0, 2023-11-27T15:33:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "get": 171, 2023-11-27T15:33:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "get_sum": 433784, 2023-11-27T15:33:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_fail": 0, 2023-11-27T15:33:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "get_or_fail_success": 171, 2023-11-27T15:33:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "take": 0, 2023-11-27T15:33:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "take_sum": 0, 2023-11-27T15:33:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "put": 171, 2023-11-27T15:33:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "put_sum": 433784, 2023-11-27T15:33:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "wait": { 2023-11-27T15:33:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "avgcount": 0, 2023-11-27T15:33:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "sum": 0.000000000, 2023-11-27T15:33:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "avgtime": 0.000000000 2023-11-27T15:33:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: } 2023-11-27T15:33:03.917 INFO:teuthology.orchestra.run.smithi060.stdout:} 2023-11-27T15:33:04.284 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-11-27T15:33:04.284 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-11-27T15:33:04.284 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-11-27T15:33:04.284 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-11-27T15:33:04.284 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-11-27T15:33:04.285 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-27T15:33:04.294 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-27T15:33:04.295 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:33:04.295 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:33:04.359 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2023-11-27T15:33:04.360 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:33:04.407 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:33:04.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.657+0000 7f2529dff700 1 -- 172.21.15.8:0/972209481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25241014c0 msgr2=0x7f2524101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:04.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.657+0000 7f2529dff700 1 --2- 172.21.15.8:0/972209481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25241014c0 0x7f2524101910 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2518009990 tx=0x7f2518009ca0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.657+0000 7f2529dff700 1 -- 172.21.15.8:0/972209481 shutdown_connections 2023-11-27T15:33:04.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.657+0000 7f2529dff700 1 --2- 172.21.15.8:0/972209481 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f2524101e50 0x7f2524108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.657+0000 7f2529dff700 1 --2- 172.21.15.8:0/972209481 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25241002d0 0x7f25241006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.661 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.657+0000 7f2529dff700 1 --2- 172.21.15.8:0/972209481 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25241014c0 0x7f2524101910 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.657+0000 7f2529dff700 1 -- 172.21.15.8:0/972209481 >> 172.21.15.8:0/972209481 conn(0x7f25240fb8b0 msgr2=0x7f25240fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:04.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.657+0000 7f2529dff700 1 -- 172.21.15.8:0/972209481 shutdown_connections 2023-11-27T15:33:04.662 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.657+0000 7f2529dff700 1 -- 172.21.15.8:0/972209481 wait complete. 2023-11-27T15:33:04.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.658+0000 7f2529dff700 1 Processor -- start 2023-11-27T15:33:04.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2529dff700 1 -- start start 2023-11-27T15:33:04.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2529dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25241002d0 0x7f2524198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:04.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2529dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241014c0 0x7f2524198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:04.664 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2529dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2524101e50 0x7f252419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:04.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2529dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252419fdd0 con 0x7f2524101e50 2023-11-27T15:33:04.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2529dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252419ff10 con 0x7f25241002d0 2023-11-27T15:33:04.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2529dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25241a00a0 con 0x7f25241014c0 2023-11-27T15:33:04.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2523fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2524101e50 0x7f252419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:04.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2523fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2524101e50 0x7f252419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53118/0 (socket says 172.21.15.8:53118) 2023-11-27T15:33:04.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2523fff700 1 -- 172.21.15.8:0/1974713215 learned_addr learned my addr 172.21.15.8:0/1974713215 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:04.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f25237fe700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25241002d0 0x7f2524198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:04.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.659+0000 7f2522ffd700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241014c0 0x7f2524198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:04.665 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2523fff700 1 -- 172.21.15.8:0/1974713215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241014c0 msgr2=0x7f2524198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:04.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2523fff700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241014c0 0x7f2524198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2523fff700 1 -- 172.21.15.8:0/1974713215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25241002d0 msgr2=0x7f2524198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:04.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2523fff700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25241002d0 0x7f2524198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2523fff700 1 -- 172.21.15.8:0/1974713215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180095f0 con 0x7f2524101e50 2023-11-27T15:33:04.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2523fff700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2524101e50 0x7f252419f800 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f251c00ef30 tx=0x7f251c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:04.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2520ff9700 1 -- 172.21.15.8:0/1974713215 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c018070 con 0x7f2524101e50 2023-11-27T15:33:04.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2520ff9700 1 -- 172.21.15.8:0/1974713215 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f251c00f040 con 0x7f2524101e50 2023-11-27T15:33:04.666 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2529dff700 1 -- 172.21.15.8:0/1974713215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252410bc80 con 0x7f2524101e50 2023-11-27T15:33:04.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2520ff9700 1 -- 172.21.15.8:0/1974713215 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c00cdb0 con 0x7f2524101e50 2023-11-27T15:33:04.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.660+0000 7f2529dff700 1 -- 172.21.15.8:0/1974713215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252410baf0 con 0x7f2524101e50 2023-11-27T15:33:04.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.661+0000 7f2529dff700 1 -- 172.21.15.8:0/1974713215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25241929e0 con 0x7f2524101e50 2023-11-27T15:33:04.667 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.663+0000 7f2520ff9700 1 -- 172.21.15.8:0/1974713215 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f251c003a00 con 0x7f2524101e50 2023-11-27T15:33:04.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.665+0000 7f2520ff9700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f250c0a11d0 0x7f250c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:04.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.665+0000 7f2520ff9700 1 -- 172.21.15.8:0/1974713215 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f251c0cc7d0 con 0x7f2524101e50 2023-11-27T15:33:04.668 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.665+0000 7f25237fe700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f250c0a11d0 0x7f250c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:04.669 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.666+0000 7f25237fe700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f250c0a11d0 0x7f250c0a3680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2524101320 tx=0x7f251400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:04.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.669+0000 7f2520ff9700 1 -- 172.21.15.8:0/1974713215 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f251c095c20 con 0x7f2524101e50 2023-11-27T15:33:04.883 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.879+0000 7f2529dff700 1 -- 172.21.15.8:0/1974713215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f252402cfe0 con 0x7f2524101e50 2023-11-27T15:33:04.884 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.881+0000 7f2520ff9700 1 -- 172.21.15.8:0/1974713215 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24392 (secure 0 0 0) 0x7f251c099240 con 0x7f2524101e50 2023-11-27T15:33:04.885 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:33:04.885 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":94,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:30:30.970908+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:33:04.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 -- 172.21.15.8:0/1974713215 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f250c0a11d0 msgr2=0x7f250c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:04.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f250c0a11d0 0x7f250c0a3680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2524101320 tx=0x7f251400b410 comp rx=0 tx=0).stop 2023-11-27T15:33:04.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 -- 172.21.15.8:0/1974713215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2524101e50 msgr2=0x7f252419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:04.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2524101e50 0x7f252419f800 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f251c00ef30 tx=0x7f251c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.887 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 -- 172.21.15.8:0/1974713215 shutdown_connections 2023-11-27T15:33:04.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2524101e50 0x7f252419f800 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f25241002d0 0x7f2524198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f250c0a11d0 0x7f250c0a3680 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 --2- 172.21.15.8:0/1974713215 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f25241014c0 0x7f2524198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:04.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 -- 172.21.15.8:0/1974713215 >> 172.21.15.8:0/1974713215 conn(0x7f25240fb8b0 msgr2=0x7f2524106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:04.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 -- 172.21.15.8:0/1974713215 shutdown_connections 2023-11-27T15:33:04.888 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:04.884+0000 7f2529dff700 1 -- 172.21.15.8:0/1974713215 wait complete. 2023-11-27T15:33:04.906 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:19:02.859722+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:23:25.166148+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:23:26.584539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:23:30.130965+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:33:04.907 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:33:05.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.112+0000 7f6fd39ff700 1 -- 172.21.15.8:0/171380050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fcc100290 msgr2=0x7f6fcc1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:05.115 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.112+0000 7f6fd39ff700 1 --2- 172.21.15.8:0/171380050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fcc100290 0x7f6fcc1006a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009990 tx=0x7f6fc8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.112+0000 7f6fd39ff700 1 -- 172.21.15.8:0/171380050 shutdown_connections 2023-11-27T15:33:05.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.112+0000 7f6fd39ff700 1 --2- 172.21.15.8:0/171380050 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fcc101e10 0x7f6fcc1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.112+0000 7f6fd39ff700 1 --2- 172.21.15.8:0/171380050 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fcc100290 0x7f6fcc1006a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.112+0000 7f6fd39ff700 1 --2- 172.21.15.8:0/171380050 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fcc101480 0x7f6fcc1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.112+0000 7f6fd39ff700 1 -- 172.21.15.8:0/171380050 >> 172.21.15.8:0/171380050 conn(0x7f6fcc0fb870 msgr2=0x7f6fcc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:05.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.112+0000 7f6fd39ff700 1 -- 172.21.15.8:0/171380050 shutdown_connections 2023-11-27T15:33:05.116 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.112+0000 7f6fd39ff700 1 -- 172.21.15.8:0/171380050 wait complete. 2023-11-27T15:33:05.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.113+0000 7f6fd39ff700 1 Processor -- start 2023-11-27T15:33:05.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.113+0000 7f6fd39ff700 1 -- start start 2023-11-27T15:33:05.119 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.113+0000 7f6fd39ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fcc100290 0x7f6fcc198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:05.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.113+0000 7f6fd39ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fcc101480 0x7f6fcc198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:05.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.113+0000 7f6fd39ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fcc101e10 0x7f6fcc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:05.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.113+0000 7f6fd39ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc19fda0 con 0x7f6fcc100290 2023-11-27T15:33:05.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.113+0000 7f6fd39ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc19fee0 con 0x7f6fcc101480 2023-11-27T15:33:05.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.113+0000 7f6fd39ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc1a0070 con 0x7f6fcc101e10 2023-11-27T15:33:05.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.113+0000 7f6fd179b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fcc100290 0x7f6fcc198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:05.120 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd179b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fcc100290 0x7f6fcc198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53134/0 (socket says 172.21.15.8:53134) 2023-11-27T15:33:05.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd179b700 1 -- 172.21.15.8:0/734434444 learned_addr learned my addr 172.21.15.8:0/734434444 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:05.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd0f9a700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fcc101480 0x7f6fcc198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:05.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd1f9c700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fcc101e10 0x7f6fcc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:05.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd179b700 1 -- 172.21.15.8:0/734434444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fcc101e10 msgr2=0x7f6fcc19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:05.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd179b700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fcc101e10 0x7f6fcc19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd179b700 1 -- 172.21.15.8:0/734434444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fcc101480 msgr2=0x7f6fcc198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:05.121 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd179b700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fcc101480 0x7f6fcc198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd179b700 1 -- 172.21.15.8:0/734434444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc80095f0 con 0x7f6fcc100290 2023-11-27T15:33:05.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd1f9c700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fcc101e10 0x7f6fcc19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:33:05.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd179b700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fcc100290 0x7f6fcc198220 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8004730 tx=0x7f6fc8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:05.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fc27fc700 1 -- 172.21.15.8:0/734434444 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc801d070 con 0x7f6fcc100290 2023-11-27T15:33:05.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fc27fc700 1 -- 172.21.15.8:0/734434444 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fc8021410 con 0x7f6fcc100290 2023-11-27T15:33:05.122 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fd39ff700 1 -- 172.21.15.8:0/734434444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc10bc40 con 0x7f6fcc100290 2023-11-27T15:33:05.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.114+0000 7f6fc27fc700 1 -- 172.21.15.8:0/734434444 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8017650 con 0x7f6fcc100290 2023-11-27T15:33:05.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.115+0000 7f6fd39ff700 1 -- 172.21.15.8:0/734434444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc10bab0 con 0x7f6fcc100290 2023-11-27T15:33:05.123 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.116+0000 7f6fd39ff700 1 -- 172.21.15.8:0/734434444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc067a50 con 0x7f6fcc100290 2023-11-27T15:33:05.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.118+0000 7f6fc27fc700 1 -- 172.21.15.8:0/734434444 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6fc8021810 con 0x7f6fcc100290 2023-11-27T15:33:05.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.119+0000 7f6fc27fc700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fb80a11d0 0x7f6fb80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:05.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.119+0000 7f6fc27fc700 1 -- 172.21.15.8:0/734434444 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6fc80ce790 con 0x7f6fcc100290 2023-11-27T15:33:05.124 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.119+0000 7f6fd0f9a700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fb80a11d0 0x7f6fb80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:05.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.120+0000 7f6fd0f9a700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fb80a11d0 0x7f6fb80a3680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009c60 tx=0x7f6fbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:05.125 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.121+0000 7f6fc27fc700 1 -- 172.21.15.8:0/734434444 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6fc8097be0 con 0x7f6fcc100290 2023-11-27T15:33:05.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.328+0000 7f6fd39ff700 1 -- 172.21.15.8:0/734434444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6fcc1a0cb0 con 0x7f6fcc100290 2023-11-27T15:33:05.331 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.328+0000 7f6fc27fc700 1 -- 172.21.15.8:0/734434444 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6fc809b200 con 0x7f6fcc100290 2023-11-27T15:33:05.332 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2023-11-27T15:33:05.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.330+0000 7f6fd39ff700 1 -- 172.21.15.8:0/734434444 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fb80a11d0 msgr2=0x7f6fb80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:05.334 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.330+0000 7f6fd39ff700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fb80a11d0 0x7f6fb80a3680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009c60 tx=0x7f6fbc009260 comp rx=0 tx=0).stop 2023-11-27T15:33:05.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.330+0000 7f6fd39ff700 1 -- 172.21.15.8:0/734434444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fcc100290 msgr2=0x7f6fcc198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:05.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.330+0000 7f6fd39ff700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fcc100290 0x7f6fcc198220 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8004730 tx=0x7f6fc8004760 comp rx=0 tx=0).stop 2023-11-27T15:33:05.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.331+0000 7f6fd39ff700 1 -- 172.21.15.8:0/734434444 shutdown_connections 2023-11-27T15:33:05.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.331+0000 7f6fd39ff700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6fb80a11d0 0x7f6fb80a3680 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.331+0000 7f6fd39ff700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6fcc101e10 0x7f6fcc19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.335 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.331+0000 7f6fd39ff700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6fcc100290 0x7f6fcc198220 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.331+0000 7f6fd39ff700 1 --2- 172.21.15.8:0/734434444 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6fcc101480 0x7f6fcc198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.331+0000 7f6fd39ff700 1 -- 172.21.15.8:0/734434444 >> 172.21.15.8:0/734434444 conn(0x7f6fcc0fb870 msgr2=0x7f6fcc0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:05.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.331+0000 7f6fd39ff700 1 -- 172.21.15.8:0/734434444 shutdown_connections 2023-11-27T15:33:05.336 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.331+0000 7f6fd39ff700 1 -- 172.21.15.8:0/734434444 wait complete. 2023-11-27T15:33:05.353 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:33:05.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.559+0000 7f91f23ff700 1 -- 172.21.15.8:0/233736119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91ec105760 msgr2=0x7f91ec107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:05.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.559+0000 7f91f23ff700 1 --2- 172.21.15.8:0/233736119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91ec105760 0x7f91ec107b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f91e00098f0 tx=0x7f91e0009c00 comp rx=0 tx=0).stop 2023-11-27T15:33:05.563 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.559+0000 7f91f23ff700 1 -- 172.21.15.8:0/233736119 shutdown_connections 2023-11-27T15:33:05.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.559+0000 7f91f23ff700 1 --2- 172.21.15.8:0/233736119 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91ec108080 0x7f91ec10a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.559+0000 7f91f23ff700 1 --2- 172.21.15.8:0/233736119 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91ec068ae0 0x7f91ec105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.559+0000 7f91f23ff700 1 --2- 172.21.15.8:0/233736119 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91ec105760 0x7f91ec107b40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.564 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.559+0000 7f91f23ff700 1 -- 172.21.15.8:0/233736119 >> 172.21.15.8:0/233736119 conn(0x7f91ec0fa750 msgr2=0x7f91ec0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:05.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.559+0000 7f91f23ff700 1 -- 172.21.15.8:0/233736119 shutdown_connections 2023-11-27T15:33:05.565 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.559+0000 7f91f23ff700 1 -- 172.21.15.8:0/233736119 wait complete. 2023-11-27T15:33:05.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.560+0000 7f91f23ff700 1 Processor -- start 2023-11-27T15:33:05.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91f23ff700 1 -- start start 2023-11-27T15:33:05.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91f23ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91ec068ae0 0x7f91ec1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:05.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91f23ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91ec105760 0x7f91ec1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:05.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91f23ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91ec108080 0x7f91ec19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:05.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91f23ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec19fe20 con 0x7f91ec105760 2023-11-27T15:33:05.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91f23ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec19ff60 con 0x7f91ec108080 2023-11-27T15:33:05.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91f23ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec1a00f0 con 0x7f91ec068ae0 2023-11-27T15:33:05.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91eb7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91ec105760 0x7f91ec1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:05.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91ebfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91ec068ae0 0x7f91ec1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:05.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91eb7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91ec105760 0x7f91ec1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53152/0 (socket says 172.21.15.8:53152) 2023-11-27T15:33:05.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91eb7fe700 1 -- 172.21.15.8:0/724856406 learned_addr learned my addr 172.21.15.8:0/724856406 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:05.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.561+0000 7f91f099c700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91ec108080 0x7f91ec19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:05.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.562+0000 7f91eb7fe700 1 -- 172.21.15.8:0/724856406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91ec068ae0 msgr2=0x7f91ec1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:05.570 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.562+0000 7f91eb7fe700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91ec068ae0 0x7f91ec1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.562+0000 7f91eb7fe700 1 -- 172.21.15.8:0/724856406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91ec108080 msgr2=0x7f91ec19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:05.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.562+0000 7f91eb7fe700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91ec108080 0x7f91ec19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.562+0000 7f91eb7fe700 1 -- 172.21.15.8:0/724856406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc0095f0 con 0x7f91ec105760 2023-11-27T15:33:05.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.562+0000 7f91f099c700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91ec108080 0x7f91ec19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:33:05.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.562+0000 7f91eb7fe700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91ec105760 0x7f91ec1987e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009fd0 tx=0x7f91e000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:05.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.563+0000 7f91e97fa700 1 -- 172.21.15.8:0/724856406 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e001c070 con 0x7f91ec105760 2023-11-27T15:33:05.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.563+0000 7f91f23ff700 1 -- 172.21.15.8:0/724856406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e0009520 con 0x7f91ec105760 2023-11-27T15:33:05.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.563+0000 7f91e97fa700 1 -- 172.21.15.8:0/724856406 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91e000f7d0 con 0x7f91ec105760 2023-11-27T15:33:05.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.563+0000 7f91e97fa700 1 -- 172.21.15.8:0/724856406 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0004ad0 con 0x7f91ec105760 2023-11-27T15:33:05.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.563+0000 7f91f23ff700 1 -- 172.21.15.8:0/724856406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ec10de40 con 0x7f91ec105760 2023-11-27T15:33:05.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.564+0000 7f91f23ff700 1 -- 172.21.15.8:0/724856406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ec10dfd0 con 0x7f91ec105760 2023-11-27T15:33:05.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.566+0000 7f91e97fa700 1 -- 172.21.15.8:0/724856406 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f91e0020070 con 0x7f91ec105760 2023-11-27T15:33:05.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.567+0000 7f91e97fa700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f91d40a1150 0x7f91d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:05.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.567+0000 7f91e97fa700 1 -- 172.21.15.8:0/724856406 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f91e00cd8b0 con 0x7f91ec105760 2023-11-27T15:33:05.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.567+0000 7f91ebfff700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f91d40a1150 0x7f91d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:05.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.568+0000 7f91ebfff700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f91d40a1150 0x7f91d40a3600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f91ec068800 tx=0x7f91dc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:05.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.571+0000 7f91e97fa700 1 -- 172.21.15.8:0/724856406 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f91e005c730 con 0x7f91ec105760 2023-11-27T15:33:05.612 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:05 smithi008 ceph-mon[128788]: pgmap v577: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:33:05.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:05 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1974713215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:33:05.613 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:05 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/734434444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:33:05.787 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.784+0000 7f91f23ff700 1 -- 172.21.15.8:0/724856406 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91ec02cfe0 con 0x7f91ec105760 2023-11-27T15:33:05.788 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.785+0000 7f91e97fa700 1 -- 172.21.15.8:0/724856406 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f91e0096d00 con 0x7f91ec105760 2023-11-27T15:33:05.789 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:33:05.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 -- 172.21.15.8:0/724856406 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f91d40a1150 msgr2=0x7f91d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:05.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f91d40a1150 0x7f91d40a3600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f91ec068800 tx=0x7f91dc009310 comp rx=0 tx=0).stop 2023-11-27T15:33:05.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 -- 172.21.15.8:0/724856406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91ec105760 msgr2=0x7f91ec1987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:05.791 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91ec105760 0x7f91ec1987e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009fd0 tx=0x7f91e000b790 comp rx=0 tx=0).stop 2023-11-27T15:33:05.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 -- 172.21.15.8:0/724856406 shutdown_connections 2023-11-27T15:33:05.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f91d40a1150 0x7f91d40a3600 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f91ec068ae0 0x7f91ec1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f91ec108080 0x7f91ec19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.792 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 --2- 172.21.15.8:0/724856406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f91ec105760 0x7f91ec1987e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:05.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 -- 172.21.15.8:0/724856406 >> 172.21.15.8:0/724856406 conn(0x7f91ec0fa750 msgr2=0x7f91ec1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:05.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 -- 172.21.15.8:0/724856406 shutdown_connections 2023-11-27T15:33:05.793 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:05.787+0000 7f91f23ff700 1 -- 172.21.15.8:0/724856406 wait complete. 2023-11-27T15:33:05.810 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:33:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:05 smithi060 ceph-mon[131518]: pgmap v577: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:33:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:05 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1974713215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:33:05.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:05 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/734434444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:33:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:05 smithi029 ceph-mon[131166]: pgmap v577: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:33:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:05 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1974713215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:33:05.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:05 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/734434444' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:33:06.019 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.015+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2761249679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6eac101480 msgr2=0x7f6eac1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.016+0000 7f6eb43ff700 1 --2- 172.21.15.8:0/2761249679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6eac101480 0x7f6eac1018d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009990 tx=0x7f6e9c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.016+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2761249679 shutdown_connections 2023-11-27T15:33:06.020 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.016+0000 7f6eb43ff700 1 --2- 172.21.15.8:0/2761249679 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eac101e10 0x7f6eac1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.016+0000 7f6eb43ff700 1 --2- 172.21.15.8:0/2761249679 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6eac100290 0x7f6eac1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.016+0000 7f6eb43ff700 1 --2- 172.21.15.8:0/2761249679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6eac101480 0x7f6eac1018d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.016+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2761249679 >> 172.21.15.8:0/2761249679 conn(0x7f6eac0fb870 msgr2=0x7f6eac0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.016+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2761249679 shutdown_connections 2023-11-27T15:33:06.021 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.016+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2761249679 wait complete. 2023-11-27T15:33:06.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.017+0000 7f6eb43ff700 1 Processor -- start 2023-11-27T15:33:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.017+0000 7f6eb43ff700 1 -- start start 2023-11-27T15:33:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.017+0000 7f6eb43ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eac100290 0x7f6eac198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.017+0000 7f6eb43ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6eac101480 0x7f6eac1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.017+0000 7f6eb43ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6eac101e10 0x7f6eac19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.017+0000 7f6eb43ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac19fde0 con 0x7f6eac101e10 2023-11-27T15:33:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.017+0000 7f6eb43ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac19ff20 con 0x7f6eac101480 2023-11-27T15:33:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.017+0000 7f6eb43ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1a00b0 con 0x7f6eac100290 2023-11-27T15:33:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb299c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6eac101e10 0x7f6eac19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb299c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6eac101e10 0x7f6eac19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53166/0 (socket says 172.21.15.8:53166) 2023-11-27T15:33:06.026 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb299c700 1 -- 172.21.15.8:0/2779838584 learned_addr learned my addr 172.21.15.8:0/2779838584 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb199a700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6eac101480 0x7f6eac1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb219b700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eac100290 0x7f6eac198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb299c700 1 -- 172.21.15.8:0/2779838584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eac100290 msgr2=0x7f6eac198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb299c700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eac100290 0x7f6eac198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb299c700 1 -- 172.21.15.8:0/2779838584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6eac101480 msgr2=0x7f6eac1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb299c700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6eac101480 0x7f6eac1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb299c700 1 -- 172.21.15.8:0/2779838584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c0095f0 con 0x7f6eac101e10 2023-11-27T15:33:06.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb199a700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6eac101480 0x7f6eac1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:33:06.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.018+0000 7f6eb299c700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6eac101e10 0x7f6eac19f810 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400ef30 tx=0x7f6ea400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:06.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.019+0000 7f6ea37fe700 1 -- 172.21.15.8:0/2779838584 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4018070 con 0x7f6eac101e10 2023-11-27T15:33:06.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.019+0000 7f6ea37fe700 1 -- 172.21.15.8:0/2779838584 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ea400f040 con 0x7f6eac101e10 2023-11-27T15:33:06.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.019+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2779838584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac10bc50 con 0x7f6eac101e10 2023-11-27T15:33:06.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.019+0000 7f6ea37fe700 1 -- 172.21.15.8:0/2779838584 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400cd00 con 0x7f6eac101e10 2023-11-27T15:33:06.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.019+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2779838584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac10bac0 con 0x7f6eac101e10 2023-11-27T15:33:06.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.020+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2779838584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac1928e0 con 0x7f6eac101e10 2023-11-27T15:33:06.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.022+0000 7f6ea37fe700 1 -- 172.21.15.8:0/2779838584 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6ea4003a00 con 0x7f6eac101e10 2023-11-27T15:33:06.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.024+0000 7f6ea37fe700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6e980a11d0 0x7f6e980a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.024+0000 7f6ea37fe700 1 -- 172.21.15.8:0/2779838584 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6ea40cc6b0 con 0x7f6eac101e10 2023-11-27T15:33:06.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.024+0000 7f6eb219b700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6e980a11d0 0x7f6e980a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.025+0000 7f6eb219b700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6e980a11d0 0x7f6e980a3680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6eac1012e0 tx=0x7f6ea8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:06.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.028+0000 7f6ea37fe700 1 -- 172.21.15.8:0/2779838584 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6ea4095b00 con 0x7f6eac101e10 2023-11-27T15:33:06.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.239+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2779838584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6eac061cc0 con 0x7f6eac101e10 2023-11-27T15:33:06.243 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.240+0000 7f6ea37fe700 1 -- 172.21.15.8:0/2779838584 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ea4099120 con 0x7f6eac101e10 2023-11-27T15:33:06.244 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 64 2023-11-27T15:33:06.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2779838584 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6e980a11d0 msgr2=0x7f6e980a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6e980a11d0 0x7f6e980a3680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6eac1012e0 tx=0x7f6ea8009260 comp rx=0 tx=0).stop 2023-11-27T15:33:06.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2779838584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6eac101e10 msgr2=0x7f6eac19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.246 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6eac101e10 0x7f6eac19f810 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400ef30 tx=0x7f6ea400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2779838584 shutdown_connections 2023-11-27T15:33:06.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f6e980a11d0 0x7f6e980a3680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6eac100290 0x7f6eac198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6eac101e10 0x7f6eac19f810 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.247 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 --2- 172.21.15.8:0/2779838584 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6eac101480 0x7f6eac1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2779838584 >> 172.21.15.8:0/2779838584 conn(0x7f6eac0fb870 msgr2=0x7f6eac106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:06.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2779838584 shutdown_connections 2023-11-27T15:33:06.248 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.242+0000 7f6eb43ff700 1 -- 172.21.15.8:0/2779838584 wait complete. 2023-11-27T15:33:06.265 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:33:06.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.470+0000 7f69601ff700 1 -- 172.21.15.8:0/2744402840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6958101480 msgr2=0x7f69581018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.470+0000 7f69601ff700 1 --2- 172.21.15.8:0/2744402840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6958101480 0x7f69581018d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6948009990 tx=0x7f6948009ca0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.475 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.470+0000 7f69601ff700 1 -- 172.21.15.8:0/2744402840 shutdown_connections 2023-11-27T15:33:06.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.470+0000 7f69601ff700 1 --2- 172.21.15.8:0/2744402840 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6958100290 0x7f69581006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.470+0000 7f69601ff700 1 --2- 172.21.15.8:0/2744402840 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6958101e10 0x7f69581086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.470+0000 7f69601ff700 1 --2- 172.21.15.8:0/2744402840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6958101480 0x7f69581018d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.470+0000 7f69601ff700 1 -- 172.21.15.8:0/2744402840 >> 172.21.15.8:0/2744402840 conn(0x7f69580fb870 msgr2=0x7f69580fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:06.476 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.471+0000 7f69601ff700 1 -- 172.21.15.8:0/2744402840 shutdown_connections 2023-11-27T15:33:06.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.471+0000 7f69601ff700 1 -- 172.21.15.8:0/2744402840 wait complete. 2023-11-27T15:33:06.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f69601ff700 1 Processor -- start 2023-11-27T15:33:06.477 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f69601ff700 1 -- start start 2023-11-27T15:33:06.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f69601ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6958100290 0x7f69580748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f69601ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6958101480 0x7f6958072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f69601ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6958101e10 0x7f6958073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.480 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f69601ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6958073bc0 con 0x7f6958100290 2023-11-27T15:33:06.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f69601ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6958073d30 con 0x7f6958101e10 2023-11-27T15:33:06.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f69601ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69581a0060 con 0x7f6958101480 2023-11-27T15:33:06.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f695df9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6958100290 0x7f69580748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f695df9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6958100290 0x7f69580748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53178/0 (socket says 172.21.15.8:53178) 2023-11-27T15:33:06.481 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f695df9b700 1 -- 172.21.15.8:0/1326140508 learned_addr learned my addr 172.21.15.8:0/1326140508 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:06.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f695d79a700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6958101480 0x7f6958072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.472+0000 7f695e79c700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6958101e10 0x7f6958073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f695df9b700 1 -- 172.21.15.8:0/1326140508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6958101480 msgr2=0x7f6958072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.482 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f695df9b700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6958101480 0x7f6958072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f695df9b700 1 -- 172.21.15.8:0/1326140508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6958101e10 msgr2=0x7f6958073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f695df9b700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6958101e10 0x7f6958073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f695df9b700 1 -- 172.21.15.8:0/1326140508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480095f0 con 0x7f6958100290 2023-11-27T15:33:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f695df9b700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6958100290 0x7f69580748a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f695400e970 tx=0x7f695400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f694effd700 1 -- 172.21.15.8:0/1326140508 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954009790 con 0x7f6958100290 2023-11-27T15:33:06.483 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f694effd700 1 -- 172.21.15.8:0/1326140508 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f695400c7e0 con 0x7f6958100290 2023-11-27T15:33:06.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f69601ff700 1 -- 172.21.15.8:0/1326140508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f695810bc50 con 0x7f6958100290 2023-11-27T15:33:06.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f694effd700 1 -- 172.21.15.8:0/1326140508 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954003c70 con 0x7f6958100290 2023-11-27T15:33:06.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.473+0000 7f69601ff700 1 -- 172.21.15.8:0/1326140508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695810bac0 con 0x7f6958100290 2023-11-27T15:33:06.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.475+0000 7f69601ff700 1 -- 172.21.15.8:0/1326140508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6958107870 con 0x7f6958100290 2023-11-27T15:33:06.484 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.477+0000 7f694effd700 1 -- 172.21.15.8:0/1326140508 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f695400cc50 con 0x7f6958100290 2023-11-27T15:33:06.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.478+0000 7f694effd700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f69440a1150 0x7f69440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.478+0000 7f694effd700 1 -- 172.21.15.8:0/1326140508 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f6954014070 con 0x7f6958100290 2023-11-27T15:33:06.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.478+0000 7f695d79a700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f69440a1150 0x7f69440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.479+0000 7f695d79a700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f69440a1150 0x7f69440a3600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f694800b2e0 tx=0x7f6948005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:06.486 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.482+0000 7f694effd700 1 -- 172.21.15.8:0/1326140508 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f69540961f0 con 0x7f6958100290 2023-11-27T15:33:06.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.696+0000 7f69601ff700 1 -- 172.21.15.8:0/1326140508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6958061700 con 0x7f6958100290 2023-11-27T15:33:06.699 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.696+0000 7f694effd700 1 -- 172.21.15.8:0/1326140508 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6954003dd0 con 0x7f6958100290 2023-11-27T15:33:06.700 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2023-11-27T15:33:06.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.698+0000 7f69601ff700 1 -- 172.21.15.8:0/1326140508 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f69440a1150 msgr2=0x7f69440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.698+0000 7f69601ff700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f69440a1150 0x7f69440a3600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f694800b2e0 tx=0x7f6948005fb0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.698+0000 7f69601ff700 1 -- 172.21.15.8:0/1326140508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6958100290 msgr2=0x7f69580748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.698+0000 7f69601ff700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6958100290 0x7f69580748a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f695400e970 tx=0x7f695400ec80 comp rx=0 tx=0).stop 2023-11-27T15:33:06.703 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.699+0000 7f69601ff700 1 -- 172.21.15.8:0/1326140508 shutdown_connections 2023-11-27T15:33:06.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.699+0000 7f69601ff700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f6958101e10 0x7f6958073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.699+0000 7f69601ff700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6958100290 0x7f69580748a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.699+0000 7f69601ff700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f69440a1150 0x7f69440a3600 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.704 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.699+0000 7f69601ff700 1 --2- 172.21.15.8:0/1326140508 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f6958101480 0x7f6958072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.699+0000 7f69601ff700 1 -- 172.21.15.8:0/1326140508 >> 172.21.15.8:0/1326140508 conn(0x7f69580fb870 msgr2=0x7f69581049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:06.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.699+0000 7f69601ff700 1 -- 172.21.15.8:0/1326140508 shutdown_connections 2023-11-27T15:33:06.705 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.699+0000 7f69601ff700 1 -- 172.21.15.8:0/1326140508 wait complete. 2023-11-27T15:33:06.722 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-27T15:33:06.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:06 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/724856406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:33:06.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:06 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2779838584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:33:06.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:06 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/724856406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:33:06.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:06 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2779838584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:33:06.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:06 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/724856406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:33:06.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:06 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2779838584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:33:06.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.926+0000 7fa77efff700 1 -- 172.21.15.8:0/1450447743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa778101db0 msgr2=0x7fa778108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.926+0000 7fa77efff700 1 --2- 172.21.15.8:0/1450447743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa778101db0 0x7fa778108670 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa77000b0a0 tx=0x7fa77000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.930 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.926+0000 7fa77efff700 1 -- 172.21.15.8:0/1450447743 shutdown_connections 2023-11-27T15:33:06.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.926+0000 7fa77efff700 1 --2- 172.21.15.8:0/1450447743 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa778101db0 0x7fa778108670 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.926+0000 7fa77efff700 1 --2- 172.21.15.8:0/1450447743 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa778100230 0x7fa778100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.926+0000 7fa77efff700 1 --2- 172.21.15.8:0/1450447743 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa778101420 0x7fa778101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.926+0000 7fa77efff700 1 -- 172.21.15.8:0/1450447743 >> 172.21.15.8:0/1450447743 conn(0x7fa7780fb810 msgr2=0x7fa7780fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:06.931 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.926+0000 7fa77efff700 1 -- 172.21.15.8:0/1450447743 shutdown_connections 2023-11-27T15:33:06.932 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.926+0000 7fa77efff700 1 -- 172.21.15.8:0/1450447743 wait complete. 2023-11-27T15:33:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.927+0000 7fa77efff700 1 Processor -- start 2023-11-27T15:33:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.927+0000 7fa77efff700 1 -- start start 2023-11-27T15:33:06.935 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa77efff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa778100230 0x7fa7781960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa77efff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa778101420 0x7fa7781965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa77efff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa778101db0 0x7fa77819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa77efff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77819dc20 con 0x7fa778100230 2023-11-27T15:33:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa77efff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77819dd60 con 0x7fa778101db0 2023-11-27T15:33:06.936 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa77efff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77819def0 con 0x7fa778101420 2023-11-27T15:33:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa77cd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa778100230 0x7fa7781960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa77cd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa778100230 0x7fa7781960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53186/0 (socket says 172.21.15.8:53186) 2023-11-27T15:33:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa77cd9b700 1 -- 172.21.15.8:0/4004874915 learned_addr learned my addr 172.21.15.8:0/4004874915 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa777fff700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa778101420 0x7fa7781965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.937 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.928+0000 7fa77d59c700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa778101db0 0x7fa77819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa77cd9b700 1 -- 172.21.15.8:0/4004874915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa778101420 msgr2=0x7fa7781965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa77cd9b700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa778101420 0x7fa7781965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa77cd9b700 1 -- 172.21.15.8:0/4004874915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa778101db0 msgr2=0x7fa77819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa77cd9b700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa778101db0 0x7fa77819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:06.938 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa77cd9b700 1 -- 172.21.15.8:0/4004874915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa770009d00 con 0x7fa778100230 2023-11-27T15:33:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa77d59c700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa778101db0 0x7fa77819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:33:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa777fff700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa778101420 0x7fa7781965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:33:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa77cd9b700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa778100230 0x7fa7781960a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e970 tx=0x7fa76c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa775ffb700 1 -- 172.21.15.8:0/4004874915 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c009790 con 0x7fa778100230 2023-11-27T15:33:06.939 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa775ffb700 1 -- 172.21.15.8:0/4004874915 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa76c00c7e0 con 0x7fa778100230 2023-11-27T15:33:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa77efff700 1 -- 172.21.15.8:0/4004874915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77810bc00 con 0x7fa778100230 2023-11-27T15:33:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa775ffb700 1 -- 172.21.15.8:0/4004874915 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c003c70 con 0x7fa778100230 2023-11-27T15:33:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.929+0000 7fa77efff700 1 -- 172.21.15.8:0/4004874915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77810ba70 con 0x7fa778100230 2023-11-27T15:33:06.940 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.930+0000 7fa77efff700 1 -- 172.21.15.8:0/4004874915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77819e690 con 0x7fa778100230 2023-11-27T15:33:06.941 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.932+0000 7fa775ffb700 1 -- 172.21.15.8:0/4004874915 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa76c00cc50 con 0x7fa778100230 2023-11-27T15:33:06.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.934+0000 7fa775ffb700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7680a11d0 0x7fa7680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:06.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.934+0000 7fa775ffb700 1 -- 172.21.15.8:0/4004874915 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa76c014070 con 0x7fa778100230 2023-11-27T15:33:06.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.934+0000 7fa777fff700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7680a11d0 0x7fa7680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:06.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.934+0000 7fa777fff700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7680a11d0 0x7fa7680a3680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa764011490 tx=0x7fa764009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:06.942 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:06.937+0000 7fa775ffb700 1 -- 172.21.15.8:0/4004874915 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa76c0961a0 con 0x7fa778100230 2023-11-27T15:33:07.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.164+0000 7fa77efff700 1 -- 172.21.15.8:0/4004874915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa778107090 con 0x7fa778100230 2023-11-27T15:33:07.168 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.164+0000 7fa775ffb700 1 -- 172.21.15.8:0/4004874915 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) v1 ==== 81+0+579 (secure 0 0 0) 0x7fa76c003dd0 con 0x7fa778100230 2023-11-27T15:33:07.168 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/2324276308 2023-11-28T15:16:38.242903+0000 2023-11-27T15:33:07.168 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2225314492 2023-11-28T15:16:38.242903+0000 2023-11-27T15:33:07.169 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3792241060 2023-11-28T15:16:38.242903+0000 2023-11-27T15:33:07.169 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3188200818 2023-11-28T15:16:11.844441+0000 2023-11-27T15:33:07.169 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/128817483 2023-11-28T15:16:11.844441+0000 2023-11-27T15:33:07.169 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2304079767 2023-11-28T15:16:11.844441+0000 2023-11-27T15:33:07.170 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/560372725 2023-11-28T15:16:11.844441+0000 2023-11-27T15:33:07.170 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/2324276308 2023-11-28T15:16:38.242903+0000 2023-11-27T15:33:07.170 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/560372725 2023-11-28T15:16:11.844441+0000 2023-11-27T15:33:07.170 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3849980484 2023-11-28T15:16:38.242903+0000 2023-11-27T15:33:07.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 -- 172.21.15.8:0/4004874915 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7680a11d0 msgr2=0x7fa7680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:07.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7680a11d0 0x7fa7680a3680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa764011490 tx=0x7fa764009260 comp rx=0 tx=0).stop 2023-11-27T15:33:07.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 -- 172.21.15.8:0/4004874915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa778100230 msgr2=0x7fa7781960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:07.172 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa778100230 0x7fa7781960a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e970 tx=0x7fa76c00ec80 comp rx=0 tx=0).stop 2023-11-27T15:33:07.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 -- 172.21.15.8:0/4004874915 shutdown_connections 2023-11-27T15:33:07.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa778101db0 0x7fa77819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:07.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa778100230 0x7fa7781960a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:07.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7680a11d0 0x7fa7680a3680 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:07.173 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 --2- 172.21.15.8:0/4004874915 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa778101420 0x7fa7781965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:07.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 -- 172.21.15.8:0/4004874915 >> 172.21.15.8:0/4004874915 conn(0x7fa7780fb810 msgr2=0x7fa7780fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:07.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 -- 172.21.15.8:0/4004874915 shutdown_connections 2023-11-27T15:33:07.174 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:07.167+0000 7fa77efff700 1 -- 172.21.15.8:0/4004874915 wait complete. 2023-11-27T15:33:07.174 INFO:teuthology.orchestra.run.smithi008.stderr:listed 10 entries 2023-11-27T15:33:07.189 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-27T15:33:07.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:33:07.190 DEBUG:teuthology.orchestra.run.smithi008:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-27T15:33:07.313 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi008.front.sepia.ceph.com 2023-11-27T15:33:07.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:33:07.314 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-27T15:33:07.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:33:07.526 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2023-11-27T15:33:07.616 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-27T15:33:07.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:33:07.617 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-27T15:33:07.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:33:07.692 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link delete ceph-brx 2023-11-27T15:33:07.806 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-11-27T15:33:07.817 INFO:tasks.cephadm:Teardown begin 2023-11-27T15:33:07.817 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-27T15:33:07.817 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:33:07.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:07 smithi060 ceph-mon[131518]: pgmap v578: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:33:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:07 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1326140508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:33:07.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:07 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/4004874915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:33:07.825 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:07 smithi008 ceph-mon[128788]: pgmap v578: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:33:07.825 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:07 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1326140508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:33:07.826 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:07 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4004874915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:33:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:07 smithi029 ceph-mon[131166]: pgmap v578: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:33:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:07 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1326140508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:33:07.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:07 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/4004874915' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:33:08.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.020+0000 7f3accfff700 1 -- 172.21.15.8:0/2397915358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ac8101e50 msgr2=0x7f3ac8108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:08.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.020+0000 7f3accfff700 1 --2- 172.21.15.8:0/2397915358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ac8101e50 0x7f3ac8108710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009f90 tx=0x7f3ac000b330 comp rx=0 tx=0).stop 2023-11-27T15:33:08.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.020+0000 7f3accfff700 1 -- 172.21.15.8:0/2397915358 shutdown_connections 2023-11-27T15:33:08.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.020+0000 7f3accfff700 1 --2- 172.21.15.8:0/2397915358 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac81002d0 0x7f3ac81006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.020+0000 7f3accfff700 1 --2- 172.21.15.8:0/2397915358 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ac8101e50 0x7f3ac8108710 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.024 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.020+0000 7f3accfff700 1 --2- 172.21.15.8:0/2397915358 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ac81014c0 0x7f3ac8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.020+0000 7f3accfff700 1 -- 172.21.15.8:0/2397915358 >> 172.21.15.8:0/2397915358 conn(0x7f3ac80fb8b0 msgr2=0x7f3ac80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:08.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.020+0000 7f3accfff700 1 -- 172.21.15.8:0/2397915358 shutdown_connections 2023-11-27T15:33:08.025 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.020+0000 7f3accfff700 1 -- 172.21.15.8:0/2397915358 wait complete. 2023-11-27T15:33:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.022+0000 7f3accfff700 1 Processor -- start 2023-11-27T15:33:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.022+0000 7f3accfff700 1 -- start start 2023-11-27T15:33:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.022+0000 7f3accfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ac81002d0 0x7f3ac8196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.022+0000 7f3accfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac81014c0 0x7f3ac8196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.022+0000 7f3accfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ac8101e50 0x7f3ac819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.022+0000 7f3accfff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac819dcb0 con 0x7f3ac8101e50 2023-11-27T15:33:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.022+0000 7f3accfff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac819ddf0 con 0x7f3ac81002d0 2023-11-27T15:33:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.022+0000 7f3accfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac819df80 con 0x7f3ac81014c0 2023-11-27T15:33:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3ac6d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ac8101e50 0x7f3ac819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3ac659c700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ac81002d0 0x7f3ac8196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3ac6d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ac8101e50 0x7f3ac819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53196/0 (socket says 172.21.15.8:53196) 2023-11-27T15:33:08.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3ac6d9d700 1 -- 172.21.15.8:0/786730438 learned_addr learned my addr 172.21.15.8:0/786730438 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:08.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3abffff700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac81014c0 0x7f3ac8196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:08.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3ac6d9d700 1 -- 172.21.15.8:0/786730438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac81014c0 msgr2=0x7f3ac8196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:08.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3ac6d9d700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac81014c0 0x7f3ac8196670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3ac6d9d700 1 -- 172.21.15.8:0/786730438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ac81002d0 msgr2=0x7f3ac8196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:08.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3ac6d9d700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ac81002d0 0x7f3ac8196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3ac6d9d700 1 -- 172.21.15.8:0/786730438 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab0009520 con 0x7f3ac8101e50 2023-11-27T15:33:08.029 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3ac6d9d700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ac8101e50 0x7f3ac819d6e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0007930 tx=0x7f3ac0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:08.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.023+0000 7f3abf7fe700 1 -- 172.21.15.8:0/786730438 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000e040 con 0x7f3ac8101e50 2023-11-27T15:33:08.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.024+0000 7f3accfff700 1 -- 172.21.15.8:0/786730438 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac0009c40 con 0x7f3ac8101e50 2023-11-27T15:33:08.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.024+0000 7f3abf7fe700 1 -- 172.21.15.8:0/786730438 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ac000bda0 con 0x7f3ac8101e50 2023-11-27T15:33:08.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.024+0000 7f3abf7fe700 1 -- 172.21.15.8:0/786730438 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00052a0 con 0x7f3ac8101e50 2023-11-27T15:33:08.030 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.024+0000 7f3accfff700 1 -- 172.21.15.8:0/786730438 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac810bca0 con 0x7f3ac8101e50 2023-11-27T15:33:08.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.025+0000 7f3accfff700 1 -- 172.21.15.8:0/786730438 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac810be30 con 0x7f3ac8101e50 2023-11-27T15:33:08.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.027+0000 7f3abf7fe700 1 -- 172.21.15.8:0/786730438 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3ac001d470 con 0x7f3ac8101e50 2023-11-27T15:33:08.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.028+0000 7f3abf7fe700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ab40a11d0 0x7f3ab40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:08.031 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.028+0000 7f3abf7fe700 1 -- 172.21.15.8:0/786730438 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3ac00ce2b0 con 0x7f3ac8101e50 2023-11-27T15:33:08.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.028+0000 7f3ac659c700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ab40a11d0 0x7f3ab40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:08.032 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.029+0000 7f3ac659c700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ab40a11d0 0x7f3ab40a3680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8101320 tx=0x7f3ab8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:08.035 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.032+0000 7f3abf7fe700 1 -- 172.21.15.8:0/786730438 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3ac0097700 con 0x7f3ac8101e50 2023-11-27T15:33:08.294 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.291+0000 7f3accfff700 1 -- 172.21.15.8:0/786730438 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ac8107770 con 0x7f3ac8101e50 2023-11-27T15:33:08.295 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.292+0000 7f3abf7fe700 1 -- 172.21.15.8:0/786730438 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+10781 (secure 0 0 0) 0x7f3ac0008f30 con 0x7f3ac8101e50 2023-11-27T15:33:08.295 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:33:08.296 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":15717,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1212732321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1212732321},{"type":"v1","addr":"172.21.15.29:6835","nonce":1212732321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:23:27.167939+0000","modified":"2023-11-27T15:31:18.377201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24595,"mds_1":24593,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24593":{"gid":24593,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24595":{"gid":24595,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25696":{"gid":25696,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/2258758502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2258758502},{"type":"v1","addr":"172.21.15.60:6839","nonce":2258758502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:33:08.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.294+0000 7f3accfff700 1 -- 172.21.15.8:0/786730438 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ab40a11d0 msgr2=0x7f3ab40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:08.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.294+0000 7f3accfff700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ab40a11d0 0x7f3ab40a3680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8101320 tx=0x7f3ab8009310 comp rx=0 tx=0).stop 2023-11-27T15:33:08.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.294+0000 7f3accfff700 1 -- 172.21.15.8:0/786730438 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ac8101e50 msgr2=0x7f3ac819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:08.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.294+0000 7f3accfff700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ac8101e50 0x7f3ac819d6e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0007930 tx=0x7f3ac0004d10 comp rx=0 tx=0).stop 2023-11-27T15:33:08.298 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.294+0000 7f3accfff700 1 -- 172.21.15.8:0/786730438 shutdown_connections 2023-11-27T15:33:08.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.294+0000 7f3accfff700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ac8101e50 0x7f3ac819d6e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.294+0000 7f3accfff700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3ac81002d0 0x7f3ac8196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.294+0000 7f3accfff700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ab40a11d0 0x7f3ab40a3680 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.294+0000 7f3accfff700 1 --2- 172.21.15.8:0/786730438 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3ac81014c0 0x7f3ac8196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.294+0000 7f3accfff700 1 -- 172.21.15.8:0/786730438 >> 172.21.15.8:0/786730438 conn(0x7f3ac80fb8b0 msgr2=0x7f3ac81048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:08.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.295+0000 7f3accfff700 1 -- 172.21.15.8:0/786730438 shutdown_connections 2023-11-27T15:33:08.299 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.295+0000 7f3accfff700 1 -- 172.21.15.8:0/786730438 wait complete. 2023-11-27T15:33:08.300 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 38 2023-11-27T15:33:08.318 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:33:08.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.564+0000 7f3cc25ff700 1 -- 172.21.15.8:0/2899876265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cbc101e50 msgr2=0x7f3cbc108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:08.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.564+0000 7f3cc25ff700 1 --2- 172.21.15.8:0/2899876265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cbc101e50 0x7f3cbc108710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400b0a0 tx=0x7f3cb400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.564+0000 7f3cc25ff700 1 -- 172.21.15.8:0/2899876265 shutdown_connections 2023-11-27T15:33:08.568 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.564+0000 7f3cc25ff700 1 --2- 172.21.15.8:0/2899876265 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cbc1002d0 0x7f3cbc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.564+0000 7f3cc25ff700 1 --2- 172.21.15.8:0/2899876265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cbc101e50 0x7f3cbc108710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.565+0000 7f3cc25ff700 1 --2- 172.21.15.8:0/2899876265 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cbc1014c0 0x7f3cbc101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.565+0000 7f3cc25ff700 1 -- 172.21.15.8:0/2899876265 >> 172.21.15.8:0/2899876265 conn(0x7f3cbc0fb8b0 msgr2=0x7f3cbc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:08.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.565+0000 7f3cc25ff700 1 -- 172.21.15.8:0/2899876265 shutdown_connections 2023-11-27T15:33:08.569 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.565+0000 7f3cc25ff700 1 -- 172.21.15.8:0/2899876265 wait complete. 2023-11-27T15:33:08.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.566+0000 7f3cc25ff700 1 Processor -- start 2023-11-27T15:33:08.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.566+0000 7f3cc25ff700 1 -- start start 2023-11-27T15:33:08.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.566+0000 7f3cc25ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cbc1002d0 0x7f3cbc196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:08.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.566+0000 7f3cc25ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cbc1014c0 0x7f3cbc196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:08.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.566+0000 7f3cc25ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cbc101e50 0x7f3cbc19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:08.571 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.566+0000 7f3cc25ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc19dcd0 con 0x7f3cbc101e50 2023-11-27T15:33:08.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.566+0000 7f3cc25ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc19de10 con 0x7f3cbc1002d0 2023-11-27T15:33:08.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.566+0000 7f3cc25ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc19dfa0 con 0x7f3cbc1014c0 2023-11-27T15:33:08.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cbbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cbc1002d0 0x7f3cbc196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:08.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cbbfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cbc1002d0 0x7f3cbc196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:55408/0 (socket says 172.21.15.8:55408) 2023-11-27T15:33:08.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cbbfff700 1 -- 172.21.15.8:0/540773537 learned_addr learned my addr 172.21.15.8:0/540773537 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:08.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cc0b9c700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cbc101e50 0x7f3cbc19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:08.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cbb7fe700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cbc1014c0 0x7f3cbc196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:08.572 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cbbfff700 1 -- 172.21.15.8:0/540773537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cbc1014c0 msgr2=0x7f3cbc196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:08.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cbbfff700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cbc1014c0 0x7f3cbc196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cbbfff700 1 -- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cbc101e50 msgr2=0x7f3cbc19d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:08.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cbbfff700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cbc101e50 0x7f3cbc19d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cbbfff700 1 -- 172.21.15.8:0/540773537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4009d00 con 0x7f3cbc1002d0 2023-11-27T15:33:08.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cbb7fe700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cbc1014c0 0x7f3cbc196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:33:08.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.567+0000 7f3cc0b9c700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cbc101e50 0x7f3cbc19d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:33:08.573 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.568+0000 7f3cbbfff700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cbc1002d0 0x7f3cbc196150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00e970 tx=0x7f3cac00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:08.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.570+0000 7f3cb97fa700 1 -- 172.21.15.8:0/540773537 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac009790 con 0x7f3cbc1002d0 2023-11-27T15:33:08.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.570+0000 7f3cb97fa700 1 -- 172.21.15.8:0/540773537 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cac0044e0 con 0x7f3cbc1002d0 2023-11-27T15:33:08.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.570+0000 7f3cc25ff700 1 -- 172.21.15.8:0/540773537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cbc10bca0 con 0x7f3cbc1002d0 2023-11-27T15:33:08.574 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.570+0000 7f3cb97fa700 1 -- 172.21.15.8:0/540773537 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac003e30 con 0x7f3cbc1002d0 2023-11-27T15:33:08.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.570+0000 7f3cc25ff700 1 -- 172.21.15.8:0/540773537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cbc10bb10 con 0x7f3cbc1002d0 2023-11-27T15:33:08.575 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.572+0000 7f3cc25ff700 1 -- 172.21.15.8:0/540773537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cbc1077d0 con 0x7f3cbc1002d0 2023-11-27T15:33:08.576 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.573+0000 7f3cb97fa700 1 -- 172.21.15.8:0/540773537 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3cac003f90 con 0x7f3cbc1002d0 2023-11-27T15:33:08.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.575+0000 7f3cb97fa700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ca40a1150 0x7f3ca40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:08.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.575+0000 7f3cb97fa700 1 -- 172.21.15.8:0/540773537 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f3cac014070 con 0x7f3cbc1002d0 2023-11-27T15:33:08.578 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.575+0000 7f3cbb7fe700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ca40a1150 0x7f3ca40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:08.579 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.575+0000 7f3cbb7fe700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ca40a1150 0x7f3ca40a3600 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009cb0 tx=0x7f3cb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:08.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.579+0000 7f3cb97fa700 1 -- 172.21.15.8:0/540773537 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3cac096360 con 0x7f3cbc1002d0 2023-11-27T15:33:08.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:08 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/786730438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:33:08.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.833+0000 7f3cc25ff700 1 -- 172.21.15.8:0/540773537 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cbc061980 con 0x7f3cbc1002d0 2023-11-27T15:33:08.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.834+0000 7f3cb97fa700 1 -- 172.21.15.8:0/540773537 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+10781 (secure 0 0 0) 0x7f3cac099980 con 0x7f3cbc1002d0 2023-11-27T15:33:08.838 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:33:08.838 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6837/2810841935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6836","nonce":2810841935},{"type":"v1","addr":"172.21.15.8:6837","nonce":2810841935}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14538,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6839/2208262973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6838","nonce":2208262973},{"type":"v1","addr":"172.21.15.8:6839","nonce":2208262973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":15717,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6835/1212732321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6834","nonce":1212732321},{"type":"v1","addr":"172.21.15.29:6835","nonce":1212732321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":24470,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.29:6833/192235656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6832","nonce":192235656},{"type":"v1","addr":"172.21.15.29:6833","nonce":192235656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24478,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.60:6833/3958511263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":3958511263},{"type":"v1","addr":"172.21.15.60:6833","nonce":3958511263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6835/2969037056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6834","nonce":2969037056},{"type":"v1","addr":"172.21.15.8:6835","nonce":2969037056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:23:27.167939+0000","modified":"2023-11-27T15:31:18.377201+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24595,"mds_1":24593,"mds_2":24539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24535":{"gid":24535,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6835/612130958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6834","nonce":612130958},{"type":"v1","addr":"172.21.15.60:6835","nonce":612130958}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.29:6837/1651468308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6836","nonce":1651468308},{"type":"v1","addr":"172.21.15.29:6837","nonce":1651468308}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24547":{"gid":24547,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6837/287097286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6836","nonce":287097286},{"type":"v1","addr":"172.21.15.60:6837","nonce":287097286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24593":{"gid":24593,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.29:6839/2619830143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6838","nonce":2619830143},{"type":"v1","addr":"172.21.15.29:6839","nonce":2619830143}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24595":{"gid":24595,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.8:6841/110161953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6840","nonce":110161953},{"type":"v1","addr":"172.21.15.8:6841","nonce":110161953}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_25696":{"gid":25696,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.60:6839/2258758502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6838","nonce":2258758502},{"type":"v1","addr":"172.21.15.60:6839","nonce":2258758502}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:33:08.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.836+0000 7f3cc25ff700 1 -- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ca40a1150 msgr2=0x7f3ca40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:08.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.836+0000 7f3cc25ff700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ca40a1150 0x7f3ca40a3600 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009cb0 tx=0x7f3cb0009260 comp rx=0 tx=0).stop 2023-11-27T15:33:08.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.836+0000 7f3cc25ff700 1 -- 172.21.15.8:0/540773537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cbc1002d0 msgr2=0x7f3cbc196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:08.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.836+0000 7f3cc25ff700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cbc1002d0 0x7f3cbc196150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00e970 tx=0x7f3cac00ed30 comp rx=0 tx=0).stop 2023-11-27T15:33:08.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.836+0000 7f3cc25ff700 1 -- 172.21.15.8:0/540773537 shutdown_connections 2023-11-27T15:33:08.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.836+0000 7f3cc25ff700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3cbc101e50 0x7f3cbc19d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.836+0000 7f3cc25ff700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3cbc1002d0 0x7f3cbc196150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.836+0000 7f3cc25ff700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3ca40a1150 0x7f3ca40a3600 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.836+0000 7f3cc25ff700 1 --2- 172.21.15.8:0/540773537 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3cbc1014c0 0x7f3cbc196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:08.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.836+0000 7f3cc25ff700 1 -- 172.21.15.8:0/540773537 >> 172.21.15.8:0/540773537 conn(0x7f3cbc0fb8b0 msgr2=0x7f3cbc104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:08.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.837+0000 7f3cc25ff700 1 -- 172.21.15.8:0/540773537 shutdown_connections 2023-11-27T15:33:08.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:08.837+0000 7f3cc25ff700 1 -- 172.21.15.8:0/540773537 wait complete. 2023-11-27T15:33:08.842 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 38 2023-11-27T15:33:08.856 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:08 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/786730438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:33:08.859 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:33:08.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:08 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/786730438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:33:09.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.061+0000 7fa7af1ff700 1 -- 172.21.15.8:0/845391008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a81002d0 msgr2=0x7fa7a81006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:09.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.061+0000 7fa7af1ff700 1 --2- 172.21.15.8:0/845391008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a81002d0 0x7fa7a81006e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0098f0 tx=0x7fa79c009c00 comp rx=0 tx=0).stop 2023-11-27T15:33:09.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.061+0000 7fa7af1ff700 1 -- 172.21.15.8:0/845391008 shutdown_connections 2023-11-27T15:33:09.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.061+0000 7fa7af1ff700 1 --2- 172.21.15.8:0/845391008 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a81002d0 0x7fa7a81006e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.061+0000 7fa7af1ff700 1 --2- 172.21.15.8:0/845391008 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a8101e50 0x7fa7a8108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.065 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.061+0000 7fa7af1ff700 1 --2- 172.21.15.8:0/845391008 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7a81014c0 0x7fa7a8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.061+0000 7fa7af1ff700 1 -- 172.21.15.8:0/845391008 >> 172.21.15.8:0/845391008 conn(0x7fa7a80fb8b0 msgr2=0x7fa7a80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:09.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.061+0000 7fa7af1ff700 1 -- 172.21.15.8:0/845391008 shutdown_connections 2023-11-27T15:33:09.066 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.061+0000 7fa7af1ff700 1 -- 172.21.15.8:0/845391008 wait complete. 2023-11-27T15:33:09.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.062+0000 7fa7af1ff700 1 Processor -- start 2023-11-27T15:33:09.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7af1ff700 1 -- start start 2023-11-27T15:33:09.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7af1ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a81002d0 0x7fa7a8196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:09.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7af1ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81014c0 0x7fa7a81965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:09.068 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7af1ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7a8101e50 0x7fa7a819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7af1ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a819dc00 con 0x7fa7a8101e50 2023-11-27T15:33:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7af1ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a819dd40 con 0x7fa7a81014c0 2023-11-27T15:33:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7af1ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a819ded0 con 0x7fa7a81002d0 2023-11-27T15:33:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7ad79c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7a8101e50 0x7fa7a819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7ad79c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7a8101e50 0x7fa7a819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53230/0 (socket says 172.21.15.8:53230) 2023-11-27T15:33:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7ad79c700 1 -- 172.21.15.8:0/3011092917 learned_addr learned my addr 172.21.15.8:0/3011092917 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7acf9b700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a81002d0 0x7fa7a8196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.063+0000 7fa7a7fff700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81014c0 0x7fa7a81965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:09.069 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7ad79c700 1 -- 172.21.15.8:0/3011092917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a81002d0 msgr2=0x7fa7a8196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:09.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7ad79c700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a81002d0 0x7fa7a8196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7ad79c700 1 -- 172.21.15.8:0/3011092917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81014c0 msgr2=0x7fa7a81965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:09.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7ad79c700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81014c0 0x7fa7a81965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7ad79c700 1 -- 172.21.15.8:0/3011092917 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940095f0 con 0x7fa7a8101e50 2023-11-27T15:33:09.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7ad79c700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7a8101e50 0x7fa7a819d630 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000ef30 tx=0x7fa7a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:09.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7a5ffb700 1 -- 172.21.15.8:0/3011092917 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0018070 con 0x7fa7a8101e50 2023-11-27T15:33:09.070 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7a5ffb700 1 -- 172.21.15.8:0/3011092917 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7a000f040 con 0x7fa7a8101e50 2023-11-27T15:33:09.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7af1ff700 1 -- 172.21.15.8:0/3011092917 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79c009520 con 0x7fa7a8101e50 2023-11-27T15:33:09.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7a5ffb700 1 -- 172.21.15.8:0/3011092917 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a000cd00 con 0x7fa7a8101e50 2023-11-27T15:33:09.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.064+0000 7fa7af1ff700 1 -- 172.21.15.8:0/3011092917 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a810bc20 con 0x7fa7a8101e50 2023-11-27T15:33:09.071 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.065+0000 7fa7af1ff700 1 -- 172.21.15.8:0/3011092917 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a81078e0 con 0x7fa7a8101e50 2023-11-27T15:33:09.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.067+0000 7fa7a5ffb700 1 -- 172.21.15.8:0/3011092917 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa7a0003a00 con 0x7fa7a8101e50 2023-11-27T15:33:09.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.068+0000 7fa7a5ffb700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7980a1150 0x7fa7980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:09.072 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.069+0000 7fa7a5ffb700 1 -- 172.21.15.8:0/3011092917 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7fa7a00cc7c0 con 0x7fa7a8101e50 2023-11-27T15:33:09.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.069+0000 7fa7acf9b700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7980a1150 0x7fa7980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:09.073 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.069+0000 7fa7acf9b700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7980a1150 0x7fa7980a3600 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009fd0 tx=0x7fa79c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:09.076 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.073+0000 7fa7a5ffb700 1 -- 172.21.15.8:0/3011092917 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa7a0095c10 con 0x7fa7a8101e50 2023-11-27T15:33:09.285 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.282+0000 7fa7af1ff700 1 -- 172.21.15.8:0/3011092917 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7a8067af0 con 0x7fa7a8101e50 2023-11-27T15:33:09.287 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.284+0000 7fa7a5ffb700 1 -- 172.21.15.8:0/3011092917 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24392 (secure 0 0 0) 0x7fa7a0099230 con 0x7fa7a8101e50 2023-11-27T15:33:09.288 INFO:teuthology.orchestra.run.smithi008.stdout: 2023-11-27T15:33:09.288 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":94,"fsid":"9f287ba8-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:15:42.569283+0000","modified":"2023-11-27T15:30:30.970908+0000","last_up_change":"2023-11-27T15:22:35.496996+0000","last_in_change":"2023-11-27T15:22:17.939269+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:19:02.859722+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:23:25.166148+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:23:26.584539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:23:30.130965+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f832011f-25d1-4f98-adf6-6c2225afa50c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6803","nonce":225604885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6805","nonce":225604885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6809","nonce":225604885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":225604885},{"type":"v1","addr":"172.21.15.8:6807","nonce":225604885}]},"public_addr":"172.21.15.8:6803/225604885","cluster_addr":"172.21.15.8:6805/225604885","heartbeat_back_addr":"172.21.15.8:6809/225604885","heartbeat_front_addr":"172.21.15.8:6807/225604885","state":["exists","up"]},{"osd":1,"uuid":"6b3d6a3d-e153-4cc3-9551-81eb07f635c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6800","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6801","nonce":3314882267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6802","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6803","nonce":3314882267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6806","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6807","nonce":3314882267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6804","nonce":3314882267},{"type":"v1","addr":"172.21.15.29:6805","nonce":3314882267}]},"public_addr":"172.21.15.29:6801/3314882267","cluster_addr":"172.21.15.29:6803/3314882267","heartbeat_back_addr":"172.21.15.29:6807/3314882267","heartbeat_front_addr":"172.21.15.29:6805/3314882267","state":["exists","up"]},{"osd":2,"uuid":"c6d3ae94-3e3f-4c0a-b3fb-7a3fb2ee445b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6801","nonce":3602448664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6803","nonce":3602448664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6807","nonce":3602448664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3602448664},{"type":"v1","addr":"172.21.15.60:6805","nonce":3602448664}]},"public_addr":"172.21.15.60:6801/3602448664","cluster_addr":"172.21.15.60:6803/3602448664","heartbeat_back_addr":"172.21.15.60:6807/3602448664","heartbeat_front_addr":"172.21.15.60:6805/3602448664","state":["exists","up"]},{"osd":3,"uuid":"7b26fe4c-ca46-4542-9906-25396da80c99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6811","nonce":709452183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6813","nonce":709452183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6817","nonce":709452183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":709452183},{"type":"v1","addr":"172.21.15.8:6815","nonce":709452183}]},"public_addr":"172.21.15.8:6811/709452183","cluster_addr":"172.21.15.8:6813/709452183","heartbeat_back_addr":"172.21.15.8:6817/709452183","heartbeat_front_addr":"172.21.15.8:6815/709452183","state":["exists","up"]},{"osd":4,"uuid":"77ced007-63e0-4416-a2b1-bebc48d33c03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6808","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6809","nonce":3472444298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6810","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6811","nonce":3472444298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6814","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6815","nonce":3472444298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6812","nonce":3472444298},{"type":"v1","addr":"172.21.15.29:6813","nonce":3472444298}]},"public_addr":"172.21.15.29:6809/3472444298","cluster_addr":"172.21.15.29:6811/3472444298","heartbeat_back_addr":"172.21.15.29:6815/3472444298","heartbeat_front_addr":"172.21.15.29:6813/3472444298","state":["exists","up"]},{"osd":5,"uuid":"e4c2fbe2-63ef-4d43-a04f-38bef2ac6839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6809","nonce":1250742602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6811","nonce":1250742602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6815","nonce":1250742602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":1250742602},{"type":"v1","addr":"172.21.15.60:6813","nonce":1250742602}]},"public_addr":"172.21.15.60:6809/1250742602","cluster_addr":"172.21.15.60:6811/1250742602","heartbeat_back_addr":"172.21.15.60:6815/1250742602","heartbeat_front_addr":"172.21.15.60:6813/1250742602","state":["exists","up"]},{"osd":6,"uuid":"309035f0-69df-47d3-b854-c1b480c200c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6819","nonce":1285430534}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6821","nonce":1285430534}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6825","nonce":1285430534}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":1285430534},{"type":"v1","addr":"172.21.15.8:6823","nonce":1285430534}]},"public_addr":"172.21.15.8:6819/1285430534","cluster_addr":"172.21.15.8:6821/1285430534","heartbeat_back_addr":"172.21.15.8:6825/1285430534","heartbeat_front_addr":"172.21.15.8:6823/1285430534","state":["exists","up"]},{"osd":7,"uuid":"2fcef10a-546a-4928-9341-7d31b79942be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6816","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6817","nonce":2172314430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6818","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6819","nonce":2172314430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6822","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6823","nonce":2172314430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6820","nonce":2172314430},{"type":"v1","addr":"172.21.15.29:6821","nonce":2172314430}]},"public_addr":"172.21.15.29:6817/2172314430","cluster_addr":"172.21.15.29:6819/2172314430","heartbeat_back_addr":"172.21.15.29:6823/2172314430","heartbeat_front_addr":"172.21.15.29:6821/2172314430","state":["exists","up"]},{"osd":8,"uuid":"addef24e-8aa7-4f3d-be94-9decd5f22f87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6817","nonce":2261251565}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6819","nonce":2261251565}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6823","nonce":2261251565}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":2261251565},{"type":"v1","addr":"172.21.15.60:6821","nonce":2261251565}]},"public_addr":"172.21.15.60:6817/2261251565","cluster_addr":"172.21.15.60:6819/2261251565","heartbeat_back_addr":"172.21.15.60:6823/2261251565","heartbeat_front_addr":"172.21.15.60:6821/2261251565","state":["exists","up"]},{"osd":9,"uuid":"6a8dc678-dc37-44a0-9fd5-a4a98ad5b6f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6827","nonce":2087690159}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6829","nonce":2087690159}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6832","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6833","nonce":2087690159}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6830","nonce":2087690159},{"type":"v1","addr":"172.21.15.8:6831","nonce":2087690159}]},"public_addr":"172.21.15.8:6827/2087690159","cluster_addr":"172.21.15.8:6829/2087690159","heartbeat_back_addr":"172.21.15.8:6833/2087690159","heartbeat_front_addr":"172.21.15.8:6831/2087690159","state":["exists","up"]},{"osd":10,"uuid":"5dc133e8-62fc-4f5d-bde0-8454b3292bca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6824","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6825","nonce":3342205417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6826","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6827","nonce":3342205417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6830","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6831","nonce":3342205417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.29:6828","nonce":3342205417},{"type":"v1","addr":"172.21.15.29:6829","nonce":3342205417}]},"public_addr":"172.21.15.29:6825/3342205417","cluster_addr":"172.21.15.29:6827/3342205417","heartbeat_back_addr":"172.21.15.29:6831/3342205417","heartbeat_front_addr":"172.21.15.29:6829/3342205417","state":["exists","up"]},{"osd":11,"uuid":"ac26363a-c419-4b6d-8dc7-5e337cc9b565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6825","nonce":1307710245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6827","nonce":1307710245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6831","nonce":1307710245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":1307710245},{"type":"v1","addr":"172.21.15.60:6829","nonce":1307710245}]},"public_addr":"172.21.15.60:6825/1307710245","cluster_addr":"172.21.15.60:6827/1307710245","heartbeat_back_addr":"172.21.15.60:6831/1307710245","heartbeat_front_addr":"172.21.15.60:6829/1307710245","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:13.496741+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:35.660884+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:58.786651+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.570050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:47.960301+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:09.365731+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:36.887318+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:20:59.408384+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:22.033522+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:21:49.427107+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:12.252265+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:22:33.832761+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6801/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/2225314492":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3792241060":"2023-11-28T15:16:38.242903+0000","172.21.15.8:0/3188200818":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/128817483":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/2304079767":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:6800/2324276308":"2023-11-28T15:16:38.242903+0000","172.21.15.8:6801/560372725":"2023-11-28T15:16:11.844441+0000","172.21.15.8:0/3849980484":"2023-11-28T15:16:38.242903+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:33:09.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.286+0000 7fa7af1ff700 1 -- 172.21.15.8:0/3011092917 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7980a1150 msgr2=0x7fa7980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:09.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.286+0000 7fa7af1ff700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7980a1150 0x7fa7980a3600 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009fd0 tx=0x7fa79c019040 comp rx=0 tx=0).stop 2023-11-27T15:33:09.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.286+0000 7fa7af1ff700 1 -- 172.21.15.8:0/3011092917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7a8101e50 msgr2=0x7fa7a819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:09.290 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.286+0000 7fa7af1ff700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7a8101e50 0x7fa7a819d630 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000ef30 tx=0x7fa7a000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.287+0000 7fa7af1ff700 1 -- 172.21.15.8:0/3011092917 shutdown_connections 2023-11-27T15:33:09.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.287+0000 7fa7af1ff700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fa7980a1150 0x7fa7980a3600 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.287+0000 7fa7af1ff700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fa7a81002d0 0x7fa7a8196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.287+0000 7fa7af1ff700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa7a8101e50 0x7fa7a819d630 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.287+0000 7fa7af1ff700 1 --2- 172.21.15.8:0/3011092917 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fa7a81014c0 0x7fa7a81965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.287+0000 7fa7af1ff700 1 -- 172.21.15.8:0/3011092917 >> 172.21.15.8:0/3011092917 conn(0x7fa7a80fb8b0 msgr2=0x7fa7a8104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:09.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.287+0000 7fa7af1ff700 1 -- 172.21.15.8:0/3011092917 shutdown_connections 2023-11-27T15:33:09.291 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.287+0000 7fa7af1ff700 1 -- 172.21.15.8:0/3011092917 wait complete. 2023-11-27T15:33:09.309 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-27T15:33:09.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.510+0000 7f7e8ffff700 1 -- 172.21.15.8:0/197578429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e88100230 msgr2=0x7f7e88100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:09.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.510+0000 7f7e8ffff700 1 --2- 172.21.15.8:0/197578429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e88100230 0x7f7e88100640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e840098f0 tx=0x7f7e84009c00 comp rx=0 tx=0).stop 2023-11-27T15:33:09.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.510+0000 7f7e8ffff700 1 -- 172.21.15.8:0/197578429 shutdown_connections 2023-11-27T15:33:09.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.510+0000 7f7e8ffff700 1 --2- 172.21.15.8:0/197578429 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e88100230 0x7f7e88100640 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.515 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.510+0000 7f7e8ffff700 1 --2- 172.21.15.8:0/197578429 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e88101db0 0x7f7e88108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.510+0000 7f7e8ffff700 1 --2- 172.21.15.8:0/197578429 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e88101420 0x7f7e88101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.510+0000 7f7e8ffff700 1 -- 172.21.15.8:0/197578429 >> 172.21.15.8:0/197578429 conn(0x7f7e880fb810 msgr2=0x7f7e880fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:09.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.511+0000 7f7e8ffff700 1 -- 172.21.15.8:0/197578429 shutdown_connections 2023-11-27T15:33:09.516 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.511+0000 7f7e8ffff700 1 -- 172.21.15.8:0/197578429 wait complete. 2023-11-27T15:33:09.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8ffff700 1 Processor -- start 2023-11-27T15:33:09.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8ffff700 1 -- start start 2023-11-27T15:33:09.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e88100230 0x7f7e88195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:09.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e88101420 0x7f7e881964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:09.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8ffff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e88101db0 0x7f7e8819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:09.518 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8819db30 con 0x7f7e88100230 2023-11-27T15:33:09.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8819dc70 con 0x7f7e88101db0 2023-11-27T15:33:09.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8ffff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8819de00 con 0x7f7e88101420 2023-11-27T15:33:09.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8dd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e88100230 0x7f7e88195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:09.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8dd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e88100230 0x7f7e88195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53246/0 (socket says 172.21.15.8:53246) 2023-11-27T15:33:09.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8dd9b700 1 -- 172.21.15.8:0/4278051259 learned_addr learned my addr 172.21.15.8:0/4278051259 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:09.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.512+0000 7f7e8d59a700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e88101420 0x7f7e881964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:09.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.513+0000 7f7e8e59c700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e88101db0 0x7f7e8819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:09.519 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.513+0000 7f7e8dd9b700 1 -- 172.21.15.8:0/4278051259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e88101420 msgr2=0x7f7e881964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:09.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.513+0000 7f7e8dd9b700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e88101420 0x7f7e881964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.513+0000 7f7e8dd9b700 1 -- 172.21.15.8:0/4278051259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e88101db0 msgr2=0x7f7e8819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:09.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.513+0000 7f7e8dd9b700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e88101db0 0x7f7e8819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:09.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.513+0000 7f7e8dd9b700 1 -- 172.21.15.8:0/4278051259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e80009d00 con 0x7f7e88100230 2023-11-27T15:33:09.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.513+0000 7f7e8dd9b700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e88100230 0x7f7e88195fb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009fd0 tx=0x7f7e8400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:09.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.513+0000 7f7e7effd700 1 -- 172.21.15.8:0/4278051259 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8401c070 con 0x7f7e88100230 2023-11-27T15:33:09.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.513+0000 7f7e8ffff700 1 -- 172.21.15.8:0/4278051259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e84009520 con 0x7f7e88100230 2023-11-27T15:33:09.520 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.514+0000 7f7e7effd700 1 -- 172.21.15.8:0/4278051259 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e8400f7d0 con 0x7f7e88100230 2023-11-27T15:33:09.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.514+0000 7f7e7effd700 1 -- 172.21.15.8:0/4278051259 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84003a40 con 0x7f7e88100230 2023-11-27T15:33:09.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.514+0000 7f7e8ffff700 1 -- 172.21.15.8:0/4278051259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8810bb90 con 0x7f7e88100230 2023-11-27T15:33:09.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.515+0000 7f7e8ffff700 1 -- 172.21.15.8:0/4278051259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8810bd20 con 0x7f7e88100230 2023-11-27T15:33:09.521 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.516+0000 7f7e7effd700 1 -- 172.21.15.8:0/4278051259 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7e84020030 con 0x7f7e88100230 2023-11-27T15:33:09.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.518+0000 7f7e7effd700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e740a1150 0x7f7e740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:09.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.518+0000 7f7e7effd700 1 -- 172.21.15.8:0/4278051259 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9049+0+0 (secure 0 0 0) 0x7f7e840cdc00 con 0x7f7e88100230 2023-11-27T15:33:09.522 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.518+0000 7f7e8d59a700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e740a1150 0x7f7e740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:09.523 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.519+0000 7f7e8d59a700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e740a1150 0x7f7e740a3600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009c40 tx=0x7f7e78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:09.526 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.522+0000 7f7e7effd700 1 -- 172.21.15.8:0/4278051259 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7e84097050 con 0x7f7e88100230 2023-11-27T15:33:09.782 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:09.778+0000 7f7e8ffff700 1 -- 172.21.15.8:0/4278051259 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7e88107820 con 0x7f7e88100230 2023-11-27T15:33:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:09 smithi060 ceph-mon[131518]: pgmap v579: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-27T15:33:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:09 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/540773537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:33:09.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:09 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3011092917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:33:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:09 smithi008 ceph-mon[128788]: pgmap v579: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-27T15:33:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:09 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/540773537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:33:09.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:09 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3011092917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:33:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:09 smithi029 ceph-mon[131166]: pgmap v579: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 6 op/s 2023-11-27T15:33:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:09 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/540773537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:33:09.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:09 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3011092917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:33:10.581 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.577+0000 7f7e7effd700 1 -- 172.21.15.8:0/4278051259 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7e8409a670 con 0x7f7e88100230 2023-11-27T15:33:10.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 -- 172.21.15.8:0/4278051259 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e740a1150 msgr2=0x7f7e740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:10.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e740a1150 0x7f7e740a3600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009c40 tx=0x7f7e78009260 comp rx=0 tx=0).stop 2023-11-27T15:33:10.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 -- 172.21.15.8:0/4278051259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e88100230 msgr2=0x7f7e88195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:10.584 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e88100230 0x7f7e88195fb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009fd0 tx=0x7f7e8400b790 comp rx=0 tx=0).stop 2023-11-27T15:33:10.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 -- 172.21.15.8:0/4278051259 shutdown_connections 2023-11-27T15:33:10.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f7e88101db0 0x7f7e8819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:10.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e88100230 0x7f7e88195fb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:10.585 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f7e740a1150 0x7f7e740a3600 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:10.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 --2- 172.21.15.8:0/4278051259 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f7e88101420 0x7f7e881964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:10.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 -- 172.21.15.8:0/4278051259 >> 172.21.15.8:0/4278051259 conn(0x7f7e880fb810 msgr2=0x7f7e88104960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:10.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 -- 172.21.15.8:0/4278051259 shutdown_connections 2023-11-27T15:33:10.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.580+0000 7f7e8ffff700 1 -- 172.21.15.8:0/4278051259 wait complete. 2023-11-27T15:33:10.586 INFO:teuthology.orchestra.run.smithi008.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-27T15:33:10.608 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-27T15:33:10.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/4278051259' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:33:10.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-mon[131518]: osdmap e95: 12 total, 12 up, 12 in 2023-11-27T15:33:10.824 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-c[157452]: ignoring --setuser ceph since I am not root 2023-11-27T15:33:10.824 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-c[157452]: ignoring --setgroup ceph since I am not root 2023-11-27T15:33:10.824 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-mds[157483]: -- 172.21.15.60:0/3562067579 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b4f483f520 con 0x55b4f4814c00 2023-11-27T15:33:10.824 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-c[157452]: starting mds.c at 2023-11-27T15:33:10.825 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-f[157111]: ignoring --setuser ceph since I am not root 2023-11-27T15:33:10.825 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-f[157111]: ignoring --setgroup ceph since I am not root 2023-11-27T15:33:10.825 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-mds[157135]: -- 172.21.15.60:0/2475161413 <== mon.2 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55caf3e19520 con 0x55caf3dee800 2023-11-27T15:33:10.825 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-f[157111]: starting mds.f at 2023-11-27T15:33:10.825 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[160783]: ignoring --setuser ceph since I am not root 2023-11-27T15:33:10.826 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[160783]: ignoring --setgroup ceph since I am not root 2023-11-27T15:33:10.826 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-mds[160808]: -- 172.21.15.60:0/1917576982 <== mon.2 v2:172.21.15.29:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55963f949520 con 0x55963f91f000 2023-11-27T15:33:10.826 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:33:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[160783]: starting mds.i at 2023-11-27T15:33:10.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.826+0000 7f8733dff700 1 -- 172.21.15.8:0/3324366247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f872c101480 msgr2=0x7f872c1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:10.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.826+0000 7f8733dff700 1 --2- 172.21.15.8:0/3324366247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f872c101480 0x7f872c1018d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f871c0098f0 tx=0x7f871c009c00 comp rx=0 tx=0).stop 2023-11-27T15:33:10.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.826+0000 7f8733dff700 1 -- 172.21.15.8:0/3324366247 shutdown_connections 2023-11-27T15:33:10.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.826+0000 7f8733dff700 1 --2- 172.21.15.8:0/3324366247 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c101e10 0x7f872c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:10.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.826+0000 7f8733dff700 1 --2- 172.21.15.8:0/3324366247 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f872c100290 0x7f872c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:10.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.826+0000 7f8733dff700 1 --2- 172.21.15.8:0/3324366247 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f872c101480 0x7f872c1018d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:10.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.826+0000 7f8733dff700 1 -- 172.21.15.8:0/3324366247 >> 172.21.15.8:0/3324366247 conn(0x7f872c0fb870 msgr2=0x7f872c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:10.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.826+0000 7f8733dff700 1 -- 172.21.15.8:0/3324366247 shutdown_connections 2023-11-27T15:33:10.832 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.826+0000 7f8733dff700 1 -- 172.21.15.8:0/3324366247 wait complete. 2023-11-27T15:33:10.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.827+0000 7f8733dff700 1 Processor -- start 2023-11-27T15:33:10.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.827+0000 7f8733dff700 1 -- start start 2023-11-27T15:33:10.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f8733dff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f872c100290 0x7f872c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:10.834 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f8733dff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f872c101480 0x7f872c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:10.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f8733dff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c101e10 0x7f872c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:10.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f8733dff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c19fde0 con 0x7f872c100290 2023-11-27T15:33:10.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f8733dff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c19ff20 con 0x7f872c101480 2023-11-27T15:33:10.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f8733dff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c1a00b0 con 0x7f872c101e10 2023-11-27T15:33:10.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f8731b9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f872c100290 0x7f872c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:10.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f873139a700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f872c101480 0x7f872c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:10.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f8731b9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f872c100290 0x7f872c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49848/0 (socket says 172.21.15.8:49848) 2023-11-27T15:33:10.835 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f8731b9b700 1 -- 172.21.15.8:0/3932215059 learned_addr learned my addr 172.21.15.8:0/3932215059 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:10.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f873239c700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c101e10 0x7f872c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:10.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f873139a700 1 -- 172.21.15.8:0/3932215059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c101e10 msgr2=0x7f872c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:10.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f873139a700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c101e10 0x7f872c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:10.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f873139a700 1 -- 172.21.15.8:0/3932215059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f872c100290 msgr2=0x7f872c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:10.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.828+0000 7f873139a700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f872c100290 0x7f872c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:10.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.829+0000 7f873139a700 1 -- 172.21.15.8:0/3932215059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87280095f0 con 0x7f872c101480 2023-11-27T15:33:10.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.829+0000 7f8731b9b700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f872c100290 0x7f872c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:33:10.836 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.829+0000 7f873139a700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f872c101480 0x7f872c1987a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f871c009230 tx=0x7f871c00b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:10.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.829+0000 7f8722ffd700 1 -- 172.21.15.8:0/3932215059 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c01c070 con 0x7f872c101480 2023-11-27T15:33:10.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.829+0000 7f8733dff700 1 -- 172.21.15.8:0/3932215059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c009520 con 0x7f872c101480 2023-11-27T15:33:10.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.829+0000 7f8733dff700 1 -- 172.21.15.8:0/3932215059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872c10bc50 con 0x7f872c101480 2023-11-27T15:33:10.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.829+0000 7f8722ffd700 1 -- 172.21.15.8:0/3932215059 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f871c00f7f0 con 0x7f872c101480 2023-11-27T15:33:10.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.829+0000 7f8722ffd700 1 -- 172.21.15.8:0/3932215059 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c003970 con 0x7f872c101480 2023-11-27T15:33:10.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.830+0000 7f8733dff700 1 -- 172.21.15.8:0/3932215059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8710005310 con 0x7f872c101480 2023-11-27T15:33:10.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.832+0000 7f8722ffd700 1 -- 172.21.15.8:0/3932215059 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f871c020030 con 0x7f872c101480 2023-11-27T15:33:10.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.833+0000 7f8722ffd700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f87180a1150 0x7f87180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:10.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.833+0000 7f8722ffd700 1 -- 172.21.15.8:0/3932215059 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7f871c0cdb90 con 0x7f872c101480 2023-11-27T15:33:10.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.834+0000 7f8731b9b700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f87180a1150 0x7f87180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:10.839 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.834+0000 7f8731b9b700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f87180a1150 0x7f87180a3600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8728005950 tx=0x7f8728009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:10.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:10.837+0000 7f8722ffd700 1 -- 172.21.15.8:0/3932215059 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f871c096ee0 con 0x7f872c101480 2023-11-27T15:33:10.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:10 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4278051259' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:33:10.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:10 smithi008 ceph-mon[128788]: osdmap e95: 12 total, 12 up, 12 in 2023-11-27T15:33:10.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-a[128764]: 2023-11-27T15:33:10.567+0000 7fddf1597700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:33:10.863 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:33:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-a[163286]: ignoring --setuser ceph since I am not root 2023-11-27T15:33:10.863 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:33:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-a[163286]: ignoring --setgroup ceph since I am not root 2023-11-27T15:33:10.863 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:33:10 smithi008 ceph-mds[163310]: -- 172.21.15.8:0/2747279945 <== mon.1 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f69456b520 con 0x55f694543000 2023-11-27T15:33:10.863 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:33:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-a[163286]: starting mds.a at 2023-11-27T15:33:10.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:10 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/4278051259' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:33:10.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:10 smithi029 ceph-mon[131166]: osdmap e95: 12 total, 12 up, 12 in 2023-11-27T15:33:10.897 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:33:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-b[156484]: ignoring --setuser ceph since I am not root 2023-11-27T15:33:10.897 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:33:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-b[156484]: ignoring --setgroup ceph since I am not root 2023-11-27T15:33:10.898 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:33:10 smithi029 ceph-mds[156508]: -- 172.21.15.29:0/556225931 <== mon.1 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ed7e1b7520 con 0x55ed7e18c800 2023-11-27T15:33:10.898 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:33:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-b[156484]: starting mds.b at 2023-11-27T15:33:10.898 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:33:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-h[156142]: ignoring --setuser ceph since I am not root 2023-11-27T15:33:10.898 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:33:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-h[156142]: ignoring --setgroup ceph since I am not root 2023-11-27T15:33:10.898 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:33:10 smithi029 ceph-mds[156166]: -- 172.21.15.29:0/485126569 <== mon.1 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56086716f520 con 0x560867144800 2023-11-27T15:33:10.898 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:33:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-h[156142]: starting mds.h at 2023-11-27T15:33:11.100 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.096+0000 7f8733dff700 1 -- 172.21.15.8:0/3932215059 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8710005f60 con 0x7f872c101480 2023-11-27T15:33:11.582 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.578+0000 7f8722ffd700 1 -- 172.21.15.8:0/3932215059 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f871c00b920 con 0x7f872c101480 2023-11-27T15:33:11.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.581+0000 7f8733dff700 1 -- 172.21.15.8:0/3932215059 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f87180a1150 msgr2=0x7f87180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:11.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.581+0000 7f8733dff700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f87180a1150 0x7f87180a3600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8728005950 tx=0x7f8728009310 comp rx=0 tx=0).stop 2023-11-27T15:33:11.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.581+0000 7f8733dff700 1 -- 172.21.15.8:0/3932215059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f872c101480 msgr2=0x7f872c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:11.586 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.581+0000 7f8733dff700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f872c101480 0x7f872c1987a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f871c009230 tx=0x7f871c00b730 comp rx=0 tx=0).stop 2023-11-27T15:33:11.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.581+0000 7f8733dff700 1 -- 172.21.15.8:0/3932215059 shutdown_connections 2023-11-27T15:33:11.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.581+0000 7f8733dff700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f87180a1150 0x7f87180a3600 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:11.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.581+0000 7f8733dff700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f872c101e10 0x7f872c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:11.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.581+0000 7f8733dff700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f872c100290 0x7f872c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:11.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.581+0000 7f8733dff700 1 --2- 172.21.15.8:0/3932215059 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f872c101480 0x7f872c1987a0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f871c009230 tx=0x7f871c00b730 comp rx=0 tx=0).stop 2023-11-27T15:33:11.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.581+0000 7f8733dff700 1 -- 172.21.15.8:0/3932215059 >> 172.21.15.8:0/3932215059 conn(0x7f872c0fb870 msgr2=0x7f872c0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:11.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.582+0000 7f8733dff700 1 -- 172.21.15.8:0/3932215059 shutdown_connections 2023-11-27T15:33:11.587 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.582+0000 7f8733dff700 1 -- 172.21.15.8:0/3932215059 wait complete. 2023-11-27T15:33:11.608 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-27T15:33:11.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.815+0000 7fb3199ff700 1 -- 172.21.15.8:0/337153412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb314100290 msgr2=0x7fb3141006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:11.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.815+0000 7fb3199ff700 1 --2- 172.21.15.8:0/337153412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb314100290 0x7fb3141006a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb304009990 tx=0x7fb304009ca0 comp rx=0 tx=0).stop 2023-11-27T15:33:11.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.816+0000 7fb3199ff700 1 -- 172.21.15.8:0/337153412 shutdown_connections 2023-11-27T15:33:11.820 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.816+0000 7fb3199ff700 1 --2- 172.21.15.8:0/337153412 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb314101e10 0x7fb3141086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:11.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.816+0000 7fb3199ff700 1 --2- 172.21.15.8:0/337153412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb314100290 0x7fb3141006a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:11.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.816+0000 7fb3199ff700 1 --2- 172.21.15.8:0/337153412 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb314101480 0x7fb3141018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:11.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.816+0000 7fb3199ff700 1 -- 172.21.15.8:0/337153412 >> 172.21.15.8:0/337153412 conn(0x7fb3140fb870 msgr2=0x7fb3140fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:11.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.816+0000 7fb3199ff700 1 -- 172.21.15.8:0/337153412 shutdown_connections 2023-11-27T15:33:11.821 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.816+0000 7fb3199ff700 1 -- 172.21.15.8:0/337153412 wait complete. 2023-11-27T15:33:11.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.817+0000 7fb3199ff700 1 Processor -- start 2023-11-27T15:33:11.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.817+0000 7fb3199ff700 1 -- start start 2023-11-27T15:33:11.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.817+0000 7fb3199ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb314100290 0x7fb314074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:11.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.817+0000 7fb3199ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb314101480 0x7fb314072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:11.823 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.817+0000 7fb3199ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb314101e10 0x7fb314073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:11.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.817+0000 7fb3199ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb314073bc0 con 0x7fb314100290 2023-11-27T15:33:11.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.817+0000 7fb3199ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb314073d30 con 0x7fb314101480 2023-11-27T15:33:11.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.817+0000 7fb3199ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3141a00a0 con 0x7fb314101e10 2023-11-27T15:33:11.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.817+0000 7fb312ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb314100290 0x7fb314074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:11.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb312ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb314100290 0x7fb314074850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49862/0 (socket says 172.21.15.8:49862) 2023-11-27T15:33:11.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb312ffd700 1 -- 172.21.15.8:0/1557759250 learned_addr learned my addr 172.21.15.8:0/1557759250 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:11.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb3137fe700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb314101e10 0x7fb314073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:11.824 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb30bfff700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb314101480 0x7fb314072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:11.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb312ffd700 1 -- 172.21.15.8:0/1557759250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb314101e10 msgr2=0x7fb314073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:11.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb312ffd700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb314101e10 0x7fb314073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:11.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb312ffd700 1 -- 172.21.15.8:0/1557759250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb314101480 msgr2=0x7fb314072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:11.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb312ffd700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb314101480 0x7fb314072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:11.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb312ffd700 1 -- 172.21.15.8:0/1557759250 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3040095f0 con 0x7fb314100290 2023-11-27T15:33:11.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb30bfff700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb314101480 0x7fb314072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:33:11.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb312ffd700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb314100290 0x7fb314074850 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb3040098e0 tx=0x7fb304004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:11.825 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb3189fd700 1 -- 172.21.15.8:0/1557759250 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30401d070 con 0x7fb314100290 2023-11-27T15:33:11.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb3189fd700 1 -- 172.21.15.8:0/1557759250 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3040054f0 con 0x7fb314100290 2023-11-27T15:33:11.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb3189fd700 1 -- 172.21.15.8:0/1557759250 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304004ca0 con 0x7fb314100290 2023-11-27T15:33:11.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.818+0000 7fb3199ff700 1 -- 172.21.15.8:0/1557759250 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31410bc40 con 0x7fb314100290 2023-11-27T15:33:11.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.819+0000 7fb3199ff700 1 -- 172.21.15.8:0/1557759250 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31410bab0 con 0x7fb314100290 2023-11-27T15:33:11.826 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.820+0000 7fb3199ff700 1 -- 172.21.15.8:0/1557759250 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb314107950 con 0x7fb314100290 2023-11-27T15:33:11.827 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.821+0000 7fb3189fd700 1 -- 172.21.15.8:0/1557759250 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb30400fb90 con 0x7fb314100290 2023-11-27T15:33:11.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.823+0000 7fb3189fd700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb3000a11d0 0x7fb3000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:11.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.823+0000 7fb3189fd700 1 -- 172.21.15.8:0/1557759250 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9307+0+0 (secure 0 0 0) 0x7fb3040cdff0 con 0x7fb314100290 2023-11-27T15:33:11.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.824+0000 7fb30bfff700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb3000a11d0 0x7fb3000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:11.828 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.824+0000 7fb30bfff700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb3000a11d0 0x7fb3000a3680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009cb0 tx=0x7fb2fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:11.831 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:11.827+0000 7fb3189fd700 1 -- 172.21.15.8:0/1557759250 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb30405a730 con 0x7fb314100290 2023-11-27T15:33:11.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:11 smithi008 ceph-mon[128788]: pgmap v581: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:33:11.862 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:11 smithi008 ceph-mon[128788]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-27T15:33:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:11 smithi008 ceph-mon[128788]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:33:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:11 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/4278051259' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-27T15:33:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:11 smithi008 ceph-mon[128788]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-27T15:33:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:11 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3932215059' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:33:11.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:11 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:33:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:11 smithi029 ceph-mon[131166]: pgmap v581: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:33:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:11 smithi029 ceph-mon[131166]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-27T15:33:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:11 smithi029 ceph-mon[131166]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:33:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:11 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/4278051259' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-27T15:33:11.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:11 smithi029 ceph-mon[131166]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-27T15:33:11.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:11 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3932215059' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:33:11.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:11 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:33:12.039 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.035+0000 7fb3199ff700 1 -- 172.21.15.8:0/1557759250 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb31402cfe0 con 0x7fb314100290 2023-11-27T15:33:12.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:11 smithi060 ceph-mon[131518]: pgmap v581: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2023-11-27T15:33:12.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:11 smithi060 ceph-mon[131518]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-27T15:33:12.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:11 smithi060 ceph-mon[131518]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:33:12.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:11 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/4278051259' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-27T15:33:12.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:11 smithi060 ceph-mon[131518]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-27T15:33:12.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:11 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3932215059' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:33:12.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:11 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:33:12.588 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.584+0000 7fb3189fd700 1 -- 172.21.15.8:0/1557759250 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb30409adf0 con 0x7fb314100290 2023-11-27T15:33:12.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.588+0000 7fb3199ff700 1 -- 172.21.15.8:0/1557759250 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb3000a11d0 msgr2=0x7fb3000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:12.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.588+0000 7fb3199ff700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb3000a11d0 0x7fb3000a3680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009cb0 tx=0x7fb2fc009260 comp rx=0 tx=0).stop 2023-11-27T15:33:12.592 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.588+0000 7fb3199ff700 1 -- 172.21.15.8:0/1557759250 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb314100290 msgr2=0x7fb314074850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:12.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.588+0000 7fb3199ff700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb314100290 0x7fb314074850 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb3040098e0 tx=0x7fb304004730 comp rx=0 tx=0).stop 2023-11-27T15:33:12.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.588+0000 7fb3199ff700 1 -- 172.21.15.8:0/1557759250 shutdown_connections 2023-11-27T15:33:12.593 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.588+0000 7fb3199ff700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7fb3000a11d0 0x7fb3000a3680 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:12.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.588+0000 7fb3199ff700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7fb314101e10 0x7fb314073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:12.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.588+0000 7fb3199ff700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb314100290 0x7fb314074850 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:12.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.588+0000 7fb3199ff700 1 --2- 172.21.15.8:0/1557759250 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7fb314101480 0x7fb314072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:12.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.588+0000 7fb3199ff700 1 -- 172.21.15.8:0/1557759250 >> 172.21.15.8:0/1557759250 conn(0x7fb3140fb870 msgr2=0x7fb314104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:12.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.589+0000 7fb3199ff700 1 -- 172.21.15.8:0/1557759250 shutdown_connections 2023-11-27T15:33:12.594 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.589+0000 7fb3199ff700 1 -- 172.21.15.8:0/1557759250 wait complete. 2023-11-27T15:33:12.595 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_metadata' removed 2023-11-27T15:33:12.619 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-27T15:33:12.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.832+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2696952629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dd8073550 msgr2=0x7f1dd80739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:12.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.832+0000 7f1dde3ff700 1 --2- 172.21.15.8:0/2696952629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dd8073550 0x7f1dd80739c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009990 tx=0x7f1dcc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:33:12.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.832+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2696952629 shutdown_connections 2023-11-27T15:33:12.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.832+0000 7f1dde3ff700 1 --2- 172.21.15.8:0/2696952629 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd810c860 0x7f1dd810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:12.837 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.832+0000 7f1dde3ff700 1 --2- 172.21.15.8:0/2696952629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dd8074b30 0x7f1dd8072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:12.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.832+0000 7f1dde3ff700 1 --2- 172.21.15.8:0/2696952629 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dd8073550 0x7f1dd80739c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:12.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.832+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2696952629 >> 172.21.15.8:0/2696952629 conn(0x7f1dd80fb870 msgr2=0x7f1dd80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:12.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.832+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2696952629 shutdown_connections 2023-11-27T15:33:12.838 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.832+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2696952629 wait complete. 2023-11-27T15:33:12.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.833+0000 7f1dde3ff700 1 Processor -- start 2023-11-27T15:33:12.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dde3ff700 1 -- start start 2023-11-27T15:33:12.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dde3ff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dd8073550 0x7f1dd819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:12.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dde3ff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd8074b30 0x7f1dd819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:12.840 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dde3ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dd810c860 0x7f1dd81a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:12.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dde3ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd81a4180 con 0x7f1dd810c860 2023-11-27T15:33:12.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dde3ff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd81a42c0 con 0x7f1dd8073550 2023-11-27T15:33:12.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dde3ff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd81a4450 con 0x7f1dd8074b30 2023-11-27T15:33:12.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dd7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dd8073550 0x7f1dd819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:12.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dd7fff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dd8073550 0x7f1dd819c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.29:3300/0 says I am v2:172.21.15.8:41520/0 (socket says 172.21.15.8:41520) 2023-11-27T15:33:12.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dd77fe700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd8074b30 0x7f1dd819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:12.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1dd7fff700 1 -- 172.21.15.8:0/2438348148 learned_addr learned my addr 172.21.15.8:0/2438348148 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:12.841 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.834+0000 7f1ddc99c700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dd810c860 0x7f1dd81a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:12.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dd7fff700 1 -- 172.21.15.8:0/2438348148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd8074b30 msgr2=0x7f1dd819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:12.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dd7fff700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd8074b30 0x7f1dd819cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:12.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dd7fff700 1 -- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dd810c860 msgr2=0x7f1dd81a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:12.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dd7fff700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dd810c860 0x7f1dd81a3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:12.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dd7fff700 1 -- 172.21.15.8:0/2438348148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc0095f0 con 0x7f1dd8073550 2023-11-27T15:33:12.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1ddc99c700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dd810c860 0x7f1dd81a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:33:12.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dd7fff700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dd8073550 0x7f1dd819c600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800e970 tx=0x7f1dc800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:12.842 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dd57fa700 1 -- 172.21.15.8:0/2438348148 <== mon.2 v2:172.21.15.29:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8009790 con 0x7f1dd8073550 2023-11-27T15:33:12.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dd57fa700 1 -- 172.21.15.8:0/2438348148 <== mon.2 v2:172.21.15.29:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc800eec0 con 0x7f1dd8073550 2023-11-27T15:33:12.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2438348148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd8112270 con 0x7f1dd8073550 2023-11-27T15:33:12.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dd57fa700 1 -- 172.21.15.8:0/2438348148 <== mon.2 v2:172.21.15.29:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8005590 con 0x7f1dd8073550 2023-11-27T15:33:12.843 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.835+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2438348148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd81120e0 con 0x7f1dd8073550 2023-11-27T15:33:12.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.836+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2438348148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd81023b0 con 0x7f1dd8073550 2023-11-27T15:33:12.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.838+0000 7f1dd57fa700 1 -- 172.21.15.8:0/2438348148 <== mon.2 v2:172.21.15.29:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1dc800cbd0 con 0x7f1dd8073550 2023-11-27T15:33:12.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.839+0000 7f1dd57fa700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dc00a11d0 0x7f1dc00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:12.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.839+0000 7f1dd57fa700 1 -- 172.21.15.8:0/2438348148 <== mon.2 v2:172.21.15.29:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8753+0+0 (secure 0 0 0) 0x7f1dc8014070 con 0x7f1dd8073550 2023-11-27T15:33:12.844 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.840+0000 7f1dd77fe700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dc00a11d0 0x7f1dc00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:12.845 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.840+0000 7f1dd77fe700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dc00a11d0 0x7f1dc00a3680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00b2e0 tx=0x7f1dcc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:12.847 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:12.843+0000 7f1dd57fa700 1 -- 172.21.15.8:0/2438348148 <== mon.2 v2:172.21.15.29:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1dc8096250 con 0x7f1dd8073550 2023-11-27T15:33:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-27T15:33:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-27T15:33:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: Cluster is now healthy 2023-11-27T15:33:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.29:6837/2020747937,v1:172.21.15.29:6839/2020747937] up:boot 2023-11-27T15:33:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.60:6834/1636030811,v1:172.21.15.60:6835/1636030811] up:boot 2023-11-27T15:33:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.60:6836/347253850,v1:172.21.15.60:6838/347253850] up:boot 2023-11-27T15:33:12.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.60:6837/953399008,v1:172.21.15.60:6839/953399008] up:boot 2023-11-27T15:33:12.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.29:6836/489653786,v1:172.21.15.29:6838/489653786] up:boot 2023-11-27T15:33:12.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: mds.? [v2:172.21.15.8:6840/3104455932,v1:172.21.15.8:6841/3104455932] up:boot 2023-11-27T15:33:12.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:33:12.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: fsmap 12 up:standby 2023-11-27T15:33:12.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:33:12.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:33:12.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:33:12.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:33:12.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:33:12.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:33:12.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:12 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1557759250' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-27T15:33:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-27T15:33:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: Cluster is now healthy 2023-11-27T15:33:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.29:6837/2020747937,v1:172.21.15.29:6839/2020747937] up:boot 2023-11-27T15:33:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.60:6834/1636030811,v1:172.21.15.60:6835/1636030811] up:boot 2023-11-27T15:33:12.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.60:6836/347253850,v1:172.21.15.60:6838/347253850] up:boot 2023-11-27T15:33:12.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.60:6837/953399008,v1:172.21.15.60:6839/953399008] up:boot 2023-11-27T15:33:12.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.29:6836/489653786,v1:172.21.15.29:6838/489653786] up:boot 2023-11-27T15:33:12.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: mds.? [v2:172.21.15.8:6840/3104455932,v1:172.21.15.8:6841/3104455932] up:boot 2023-11-27T15:33:12.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:33:12.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: fsmap 12 up:standby 2023-11-27T15:33:12.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:33:12.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:33:12.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:33:12.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:33:12.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:33:12.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:33:12.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:12 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1557759250' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:13.061 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.057+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2438348148 --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1dd8061b00 con 0x7f1dd8073550 2023-11-27T15:33:13.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-27T15:33:13.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-27T15:33:13.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: Cluster is now healthy 2023-11-27T15:33:13.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.29:6837/2020747937,v1:172.21.15.29:6839/2020747937] up:boot 2023-11-27T15:33:13.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.60:6834/1636030811,v1:172.21.15.60:6835/1636030811] up:boot 2023-11-27T15:33:13.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.60:6836/347253850,v1:172.21.15.60:6838/347253850] up:boot 2023-11-27T15:33:13.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.60:6837/953399008,v1:172.21.15.60:6839/953399008] up:boot 2023-11-27T15:33:13.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.29:6836/489653786,v1:172.21.15.29:6838/489653786] up:boot 2023-11-27T15:33:13.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: mds.? [v2:172.21.15.8:6840/3104455932,v1:172.21.15.8:6841/3104455932] up:boot 2023-11-27T15:33:13.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:33:13.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: fsmap 12 up:standby 2023-11-27T15:33:13.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:33:13.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:33:13.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:33:13.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:33:13.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:33:13.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:33:13.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:12 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1557759250' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:13.596 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.592+0000 7f1dd57fa700 1 -- 172.21.15.8:0/2438348148 <== mon.2 v2:172.21.15.29:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1dc8004ef0 con 0x7f1dd8073550 2023-11-27T15:33:13.600 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.596+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dc00a11d0 msgr2=0x7f1dc00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:13.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.596+0000 7f1dde3ff700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dc00a11d0 0x7f1dc00a3680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00b2e0 tx=0x7f1dcc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.596+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2438348148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dd8073550 msgr2=0x7f1dd819c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:13.601 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.596+0000 7f1dde3ff700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dd8073550 0x7f1dd819c600 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800e970 tx=0x7f1dc800ec80 comp rx=0 tx=0).stop 2023-11-27T15:33:13.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.597+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2438348148 shutdown_connections 2023-11-27T15:33:13.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.597+0000 7f1dde3ff700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dd810c860 0x7f1dd81a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.597+0000 7f1dde3ff700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f1dd8073550 0x7f1dd819c600 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.602 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.597+0000 7f1dde3ff700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f1dc00a11d0 0x7f1dc00a3680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.597+0000 7f1dde3ff700 1 --2- 172.21.15.8:0/2438348148 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f1dd8074b30 0x7f1dd819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.597+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2438348148 >> 172.21.15.8:0/2438348148 conn(0x7f1dd80fb870 msgr2=0x7f1dd80fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:13.603 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.598+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2438348148 shutdown_connections 2023-11-27T15:33:13.604 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.598+0000 7f1dde3ff700 1 -- 172.21.15.8:0/2438348148 wait complete. 2023-11-27T15:33:13.604 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_data' removed 2023-11-27T15:33:13.635 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-27T15:33:13.860 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.855+0000 7f3a6bfff700 1 -- 172.21.15.8:0/1357445055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a64101480 msgr2=0x7f3a641018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:13.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.855+0000 7f3a6bfff700 1 --2- 172.21.15.8:0/1357445055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a64101480 0x7f3a641018d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009990 tx=0x7f3a54009ca0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.855+0000 7f3a6bfff700 1 -- 172.21.15.8:0/1357445055 shutdown_connections 2023-11-27T15:33:13.861 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.855+0000 7f3a6bfff700 1 --2- 172.21.15.8:0/1357445055 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a64101e10 0x7f3a641086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.855+0000 7f3a6bfff700 1 --2- 172.21.15.8:0/1357445055 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a64100290 0x7f3a641006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.855+0000 7f3a6bfff700 1 --2- 172.21.15.8:0/1357445055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a64101480 0x7f3a641018d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.856+0000 7f3a6bfff700 1 -- 172.21.15.8:0/1357445055 >> 172.21.15.8:0/1357445055 conn(0x7f3a640fb870 msgr2=0x7f3a640fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:13.862 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.856+0000 7f3a6bfff700 1 -- 172.21.15.8:0/1357445055 shutdown_connections 2023-11-27T15:33:13.863 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.856+0000 7f3a6bfff700 1 -- 172.21.15.8:0/1357445055 wait complete. 2023-11-27T15:33:13.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6bfff700 1 Processor -- start 2023-11-27T15:33:13.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6bfff700 1 -- start start 2023-11-27T15:33:13.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a64100290 0x7f3a64198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:13.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a64101480 0x7f3a64198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:13.865 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a64101e10 0x7f3a6419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:13.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6bfff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6419fd90 con 0x7f3a64101480 2023-11-27T15:33:13.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6bfff700 1 -- --> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6419fed0 con 0x7f3a64101e10 2023-11-27T15:33:13.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6bfff700 1 -- --> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a641a0060 con 0x7f3a64100290 2023-11-27T15:33:13.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6959a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a64101480 0x7f3a64198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:13.866 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a69d9b700 1 --2- >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a64100290 0x7f3a64198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:13.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6959a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a64101480 0x7f3a64198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49894/0 (socket says 172.21.15.8:49894) 2023-11-27T15:33:13.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.857+0000 7f3a6959a700 1 -- 172.21.15.8:0/3889629263 learned_addr learned my addr 172.21.15.8:0/3889629263 (peer_addr_for_me v2:172.21.15.8:0/0) 2023-11-27T15:33:13.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a6a59c700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a64101e10 0x7f3a6419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:13.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a6959a700 1 -- 172.21.15.8:0/3889629263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a64100290 msgr2=0x7f3a64198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:13.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a6959a700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a64100290 0x7f3a64198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.867 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a6959a700 1 -- 172.21.15.8:0/3889629263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a64101e10 msgr2=0x7f3a6419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:13.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a6959a700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a64101e10 0x7f3a6419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:13.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a6959a700 1 -- 172.21.15.8:0/3889629263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60009520 con 0x7f3a64101480 2023-11-27T15:33:13.868 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a6959a700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a64101480 0x7f3a64198750 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400b620 tx=0x7f3a54005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:13.869 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: pgmap v582: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2023-11-27T15:33:13.869 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/1557759250' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:33:13.869 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: osdmap e96: 12 total, 12 up, 12 in 2023-11-27T15:33:13.869 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:33:13.869 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/2438348148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:13.870 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:13.870 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.870 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.870 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.870 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.871 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:13 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a5affd700 1 -- 172.21.15.8:0/3889629263 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5401d070 con 0x7f3a64101480 2023-11-27T15:33:13.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a5affd700 1 -- 172.21.15.8:0/3889629263 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a54022470 con 0x7f3a64101480 2023-11-27T15:33:13.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a5affd700 1 -- 172.21.15.8:0/3889629263 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54017650 con 0x7f3a64101480 2023-11-27T15:33:13.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.858+0000 7f3a6bfff700 1 -- 172.21.15.8:0/3889629263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a540095f0 con 0x7f3a64101480 2023-11-27T15:33:13.873 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.859+0000 7f3a6bfff700 1 -- 172.21.15.8:0/3889629263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6410bc50 con 0x7f3a64101480 2023-11-27T15:33:13.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.860+0000 7f3a6bfff700 1 -- 172.21.15.8:0/3889629263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6410bde0 con 0x7f3a64101480 2023-11-27T15:33:13.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.862+0000 7f3a5affd700 1 -- 172.21.15.8:0/3889629263 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3a540225e0 con 0x7f3a64101480 2023-11-27T15:33:13.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.864+0000 7f3a5affd700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a500a1150 0x7f3a500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:33:13.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.864+0000 7f3a5affd700 1 -- 172.21.15.8:0/3889629263 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8252+0+0 (secure 0 0 0) 0x7f3a540cdeb0 con 0x7f3a64101480 2023-11-27T15:33:13.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.864+0000 7f3a69d9b700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a500a1150 0x7f3a500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:33:13.874 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.865+0000 7f3a69d9b700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a500a1150 0x7f3a500a3600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a641012e0 tx=0x7f3a60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:33:13.875 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:13.868+0000 7f3a5affd700 1 -- 172.21.15.8:0/3889629263 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3a54097620 con 0x7f3a64101480 2023-11-27T15:33:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: pgmap v582: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2023-11-27T15:33:13.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/1557759250' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:33:13.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: osdmap e96: 12 total, 12 up, 12 in 2023-11-27T15:33:13.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:33:13.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/2438348148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:13.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:13.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.900 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:13.900 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:13 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:14.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: pgmap v582: 129 pgs: 129 active+clean; 369 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2023-11-27T15:33:14.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/1557759250' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:33:14.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: osdmap e96: 12 total, 12 up, 12 in 2023-11-27T15:33:14.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:33:14.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/2438348148' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:14.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:14.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:14.075 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:14.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:14.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:14.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:14.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:14.076 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:14.077 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:13 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:33:14.085 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.081+0000 7f3a6bfff700 1 -- 172.21.15.8:0/3889629263 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3a64061700 con 0x7f3a64101480 2023-11-27T15:33:14.672 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.667+0000 7f3a5affd700 1 -- 172.21.15.8:0/3889629263 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) v1 ==== 178+0+0 (secure 0 0 0) 0x7f3a5409ac40 con 0x7f3a64101480 2023-11-27T15:33:14.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 -- 172.21.15.8:0/3889629263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a500a1150 msgr2=0x7f3a500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:14.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a500a1150 0x7f3a500a3600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3a641012e0 tx=0x7f3a60009260 comp rx=0 tx=0).stop 2023-11-27T15:33:14.677 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 -- 172.21.15.8:0/3889629263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a64101480 msgr2=0x7f3a64198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:33:14.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a64101480 0x7f3a64198750 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400b620 tx=0x7f3a54005740 comp rx=0 tx=0).stop 2023-11-27T15:33:14.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 -- 172.21.15.8:0/3889629263 shutdown_connections 2023-11-27T15:33:14.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.8:6800/1756612424,v1:172.21.15.8:6801/1756612424] conn(0x7f3a500a1150 0x7f3a500a3600 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:14.678 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] conn(0x7f3a64100290 0x7f3a64198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:14.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.29:3300/0,v1:172.21.15.29:6789/0] conn(0x7f3a64101e10 0x7f3a6419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:14.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 --2- 172.21.15.8:0/3889629263 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a64101480 0x7f3a64198750 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:33:14.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 -- 172.21.15.8:0/3889629263 >> 172.21.15.8:0/3889629263 conn(0x7f3a640fb870 msgr2=0x7f3a64106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:33:14.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 -- 172.21.15.8:0/3889629263 shutdown_connections 2023-11-27T15:33:14.679 INFO:teuthology.orchestra.run.smithi008.stderr:2023-11-27T15:33:14.672+0000 7f3a6bfff700 1 -- 172.21.15.8:0/3889629263 wait complete. 2023-11-27T15:33:14.680 INFO:teuthology.orchestra.run.smithi008.stderr:pool 'cephfs_data_ec' removed 2023-11-27T15:33:14.709 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:33:14.803 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:33:14.841 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:33:14.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi060 to 197.2M 2023-11-27T15:33:14.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: Adjusting osd_memory_target on smithi029 to 197.2M 2023-11-27T15:33:14.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: Unable to set osd_memory_target on smithi060 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-11-27T15:33:14.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: Unable to set osd_memory_target on smithi029 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-11-27T15:33:14.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:33:14.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: osdmap e97: 12 total, 12 up, 12 in 2023-11-27T15:33:14.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:33:14.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:33:14.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:33:14.864 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:33:14.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: Service mgr has no related daemon already placed. 2023-11-27T15:33:14.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: Service mon has no related daemon already placed. 2023-11-27T15:33:14.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: Service mds.all has no related daemon already placed. 2023-11-27T15:33:14.865 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:14 smithi008 ceph-mon[128788]: from='client.? 172.21.15.8:0/3889629263' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:14.875 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi060 to 197.2M 2023-11-27T15:33:14.875 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: Adjusting osd_memory_target on smithi029 to 197.2M 2023-11-27T15:33:14.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: Unable to set osd_memory_target on smithi060 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-11-27T15:33:14.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: Unable to set osd_memory_target on smithi029 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-11-27T15:33:14.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:33:14.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: osdmap e97: 12 total, 12 up, 12 in 2023-11-27T15:33:14.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:33:14.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:33:14.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:33:14.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:33:14.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: Service mgr has no related daemon already placed. 2023-11-27T15:33:14.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: Service mon has no related daemon already placed. 2023-11-27T15:33:14.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: Service mds.all has no related daemon already placed. 2023-11-27T15:33:14.878 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:14 smithi060 ceph-mon[131518]: from='client.? 172.21.15.8:0/3889629263' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:14.881 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-11-27T15:33:14.881 DEBUG:teuthology.orchestra.run.smithi008:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:33:14.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi060 to 197.2M 2023-11-27T15:33:14.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: Adjusting osd_memory_target on smithi029 to 197.2M 2023-11-27T15:33:14.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: Unable to set osd_memory_target on smithi060 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-11-27T15:33:14.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: Unable to set osd_memory_target on smithi029 to 206875648: error parsing value: Value '206875648' is below minimum 939524096 2023-11-27T15:33:14.897 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:33:14.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: osdmap e97: 12 total, 12 up, 12 in 2023-11-27T15:33:14.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:33:14.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:33:14.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:33:14.898 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: from='mgr.14152 172.21.15.8:0/1943199397' entity='mgr.x' 2023-11-27T15:33:14.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: Service mgr has no related daemon already placed. 2023-11-27T15:33:14.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: Service mon has no related daemon already placed. 2023-11-27T15:33:14.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: Service mds.all has no related daemon already placed. 2023-11-27T15:33:14.899 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:14 smithi029 ceph-mon[131166]: from='client.? 172.21.15.8:0/3889629263' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:33:14.925 DEBUG:teuthology.orchestra.run.smithi029:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:33:14.945 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:33:14.967 INFO:tasks.cephadm:Stopping all daemons... 2023-11-27T15:33:14.967 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-11-27T15:33:14.968 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.a 2023-11-27T15:33:15.363 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:15 smithi008 systemd[1]: Stopping Ceph mon.a for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:15 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-a[128764]: 2023-11-27T15:33:15.593+0000 7fddf459d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:33:15.863 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:15 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-a[128764]: 2023-11-27T15:33:15.593+0000 7fddf459d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-11-27T15:33:16.593 INFO:journalctl@ceph.mon.a.smithi008.stdout:Nov 27 15:33:16 smithi008 bash[285507]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-a 2023-11-27T15:33:16.617 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.a.service' 2023-11-27T15:33:16.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:33:16.719 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-11-27T15:33:16.719 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-11-27T15:33:16.719 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.b 2023-11-27T15:33:17.147 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:16 smithi029 systemd[1]: Stopping Ceph mon.b for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:17.460 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:17 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-b[131142]: 2023-11-27T15:33:17.173+0000 7feb92a15700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:33:17.460 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:17 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-b[131142]: 2023-11-27T15:33:17.173+0000 7feb92a15700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-11-27T15:33:17.721 INFO:journalctl@ceph.mon.b.smithi029.stdout:Nov 27 15:33:17 smithi029 bash[160572]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-b 2023-11-27T15:33:17.728 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.b.service' 2023-11-27T15:33:17.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:33:17.780 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-11-27T15:33:17.780 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-11-27T15:33:17.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.c 2023-11-27T15:33:18.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:17 smithi060 systemd[1]: Stopping Ceph mon.c for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:18.420 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:18 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-c[131494]: 2023-11-27T15:33:18.150+0000 7f448b277700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:33:18.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:18 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-c[131494]: 2023-11-27T15:33:18.150+0000 7f448b277700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-11-27T15:33:18.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Nov 27 15:33:18 smithi060 bash[162560]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mon-c 2023-11-27T15:33:18.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mon.c.service' 2023-11-27T15:33:18.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:33:18.910 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-11-27T15:33:18.911 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-11-27T15:33:18.911 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.x 2023-11-27T15:33:19.362 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:33:19 smithi008 systemd[1]: Stopping Ceph mgr.x for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:19.703 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:33:19 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:33:19.441+0000 7f95622e2700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:33:19.703 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:33:19 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x[129037]: 2023-11-27T15:33:19.441+0000 7f95622e2700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-11-27T15:33:20.113 INFO:journalctl@ceph.mgr.x.smithi008.stdout:Nov 27 15:33:19 smithi008 bash[285654]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-x 2023-11-27T15:33:20.268 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.x.service' 2023-11-27T15:33:20.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:33:20.357 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-11-27T15:33:20.357 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-11-27T15:33:20.357 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.y 2023-11-27T15:33:20.647 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:33:20 smithi029 systemd[1]: Stopping Ceph mgr.y for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:21.073 INFO:journalctl@ceph.mgr.y.smithi029.stdout:Nov 27 15:33:21 smithi029 bash[160722]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-y 2023-11-27T15:33:21.265 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.y.service' 2023-11-27T15:33:21.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:33:21.304 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-11-27T15:33:21.304 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-11-27T15:33:21.304 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.z 2023-11-27T15:33:21.740 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:33:21 smithi060 systemd[1]: Stopping Ceph mgr.z for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:22.018 INFO:journalctl@ceph.mgr.z.smithi060.stdout:Nov 27 15:33:21 smithi060 bash[162716]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mgr-z 2023-11-27T15:33:22.208 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mgr.z.service' 2023-11-27T15:33:22.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:33:22.247 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-11-27T15:33:22.247 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-11-27T15:33:22.247 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.0 2023-11-27T15:33:22.613 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:33:22 smithi008 systemd[1]: Stopping Ceph osd.0 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:23.113 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:33:22 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-0[141167]: 2023-11-27T15:33:22.791+0000 7f3ddee52700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:33:23.113 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:33:22 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-0[141167]: 2023-11-27T15:33:22.791+0000 7f3ddee52700 -1 osd.0 98 *** Got signal Terminated *** 2023-11-27T15:33:23.113 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:33:22 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-0[141167]: 2023-11-27T15:33:22.791+0000 7f3ddee52700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:33:29.113 INFO:journalctl@ceph.osd.0.smithi008.stdout:Nov 27 15:33:28 smithi008 bash[285807]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-0 2023-11-27T15:33:31.830 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.0.service' 2023-11-27T15:33:31.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:33:31.925 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-11-27T15:33:31.925 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-11-27T15:33:31.926 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.1 2023-11-27T15:33:32.302 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:33:31 smithi029 systemd[1]: Stopping Ceph osd.1 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:32.647 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:33:32 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-1[136259]: 2023-11-27T15:33:32.297+0000 7f7a5f94d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:33:32.647 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:33:32 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-1[136259]: 2023-11-27T15:33:32.297+0000 7f7a5f94d700 -1 osd.1 98 *** Got signal Terminated *** 2023-11-27T15:33:32.648 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:33:32 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-1[136259]: 2023-11-27T15:33:32.297+0000 7f7a5f94d700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:33:38.397 INFO:journalctl@ceph.osd.1.smithi029.stdout:Nov 27 15:33:38 smithi029 bash[160865]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-1 2023-11-27T15:33:39.452 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.1.service' 2023-11-27T15:33:39.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:33:39.495 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-11-27T15:33:39.495 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-11-27T15:33:39.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.2 2023-11-27T15:33:39.825 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:33:39 smithi060 systemd[1]: Stopping Ceph osd.2 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:40.324 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:33:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-2[137357]: 2023-11-27T15:33:39.863+0000 7fc824344700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:33:40.324 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:33:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-2[137357]: 2023-11-27T15:33:39.863+0000 7fc824344700 -1 osd.2 98 *** Got signal Terminated *** 2023-11-27T15:33:40.324 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:33:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-2[137357]: 2023-11-27T15:33:39.863+0000 7fc824344700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:33:45.574 INFO:journalctl@ceph.osd.2.smithi060.stdout:Nov 27 15:33:45 smithi060 bash[162863]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-2 2023-11-27T15:33:47.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.2.service' 2023-11-27T15:33:47.258 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:33:47.259 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-11-27T15:33:47.259 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-11-27T15:33:47.259 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.3 2023-11-27T15:33:47.613 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:33:47 smithi008 systemd[1]: Stopping Ceph osd.3 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:48.113 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:33:47 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-3[146762]: 2023-11-27T15:33:47.781+0000 7f5119c09700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:33:48.113 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:33:47 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-3[146762]: 2023-11-27T15:33:47.781+0000 7f5119c09700 -1 osd.3 98 *** Got signal Terminated *** 2023-11-27T15:33:48.113 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:33:47 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-3[146762]: 2023-11-27T15:33:47.781+0000 7f5119c09700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:33:49.109 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:48 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:48.757+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:49.363 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:33:49 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-3[146762]: 2023-11-27T15:33:49.105+0000 7f51141fb700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:26.058563+0000 front 2023-11-27T15:33:26.058640+0000 (oldest deadline 2023-11-27T15:33:48.358659+0000) 2023-11-27T15:33:50.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:49 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:49.765+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:50.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:49.859+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:33:50.613 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:33:50 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-3[146762]: 2023-11-27T15:33:50.134+0000 7f51141fb700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:26.058563+0000 front 2023-11-27T15:33:26.058640+0000 (oldest deadline 2023-11-27T15:33:48.358659+0000) 2023-11-27T15:33:50.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:33:50 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:33:50.378+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:33:51.104 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:50 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:50.771+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:51.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:50.826+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:33:51.363 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:33:51 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-3[146762]: 2023-11-27T15:33:51.100+0000 7f51141fb700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:26.058563+0000 front 2023-11-27T15:33:26.058640+0000 (oldest deadline 2023-11-27T15:33:48.358659+0000) 2023-11-27T15:33:51.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:33:51 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:33:51.371+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:33:51.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:33:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:33:51.568+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:33:52.094 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:51 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:51.799+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:52.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:51.864+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:33:52.363 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:33:52 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-3[146762]: 2023-11-27T15:33:52.090+0000 7f51141fb700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:26.058563+0000 front 2023-11-27T15:33:26.058640+0000 (oldest deadline 2023-11-27T15:33:48.358659+0000) 2023-11-27T15:33:52.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:33:52 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:33:52.327+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:33:52.894 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:33:52 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:33:52.594+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:33:53.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:52 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:52.775+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:53.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:52 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:52.891+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:33:53.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:33:53 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:33:53.374+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:33:53.862 INFO:journalctl@ceph.osd.3.smithi008.stdout:Nov 27 15:33:53 smithi008 bash[286081]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-3 2023-11-27T15:33:53.863 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:53 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:53.752+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:53.938 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:33:53 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:33:53.589+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:33:54.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:53 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:53.936+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:33:54.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:33:54 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:33:54.351+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:33:54.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:33:54 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:33:54.561+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:33:55.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:54 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:54.757+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:55.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:54 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:54.892+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:33:55.575 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.3.service' 2023-11-27T15:33:55.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:33:55 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:33:55.388+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:33:55.666 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:33:55.667 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-11-27T15:33:55.667 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-11-27T15:33:55.667 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.4 2023-11-27T15:33:55.863 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:55 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:55.770+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:55.935 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:33:55 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:33:55.599+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:33:56.040 INFO:journalctl@ceph.osd.4.smithi029.stdout:Nov 27 15:33:55 smithi029 systemd[1]: Stopping Ceph osd.4 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:33:56.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:55 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:55.932+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:33:56.397 INFO:journalctl@ceph.osd.4.smithi029.stdout:Nov 27 15:33:56 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-4[141760]: 2023-11-27T15:33:56.036+0000 7f3b24d6f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:33:56.397 INFO:journalctl@ceph.osd.4.smithi029.stdout:Nov 27 15:33:56 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-4[141760]: 2023-11-27T15:33:56.036+0000 7f3b24d6f700 -1 osd.4 98 *** Got signal Terminated *** 2023-11-27T15:33:56.397 INFO:journalctl@ceph.osd.4.smithi029.stdout:Nov 27 15:33:56 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-4[141760]: 2023-11-27T15:33:56.036+0000 7f3b24d6f700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:33:56.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:33:56 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:33:56.403+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:33:56.954 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:33:56 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:33:56.634+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:33:57.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:56 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:56.789+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:57.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:56 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:56.952+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:33:57.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:33:57 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:33:57.446+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:33:57.919 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:33:57 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:33:57.626+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:33:58.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:57 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:57.808+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:58.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:57 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:57.916+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:33:58.896 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:33:58 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:33:58.410+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:33:58.930 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:33:58 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:33:58.620+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:33:59.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:58 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:58.797+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:33:59.323 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:58 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:58.926+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:33:59.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:58 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:58.926+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:33.059681+0000 front 2023-11-27T15:33:33.059601+0000 (oldest deadline 2023-11-27T15:33:58.359640+0000) 2023-11-27T15:33:59.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:33:59 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:33:59.440+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:33:59.963 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:33:59 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:33:59.664+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:00.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:59 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:59.750+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:00.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:33:59 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:33:59.750+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:00.323 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:59 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:59.959+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:34:00.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:33:59 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:33:59.959+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:33.059681+0000 front 2023-11-27T15:33:33.059601+0000 (oldest deadline 2023-11-27T15:33:58.359640+0000) 2023-11-27T15:34:00.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:00 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:00.414+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:00.970 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:00 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:00.647+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:00.970 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:00 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:00.647+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:01.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:00 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:00.791+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:01.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:00 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:00.791+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:01.323 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:00 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:00.967+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:34:01.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:00 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:00.967+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:33.059681+0000 front 2023-11-27T15:33:33.059601+0000 (oldest deadline 2023-11-27T15:33:58.359640+0000) 2023-11-27T15:34:01.397 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:01 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:01.143+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:01.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:01 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:01.375+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:01.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:01 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:01.375+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:01.897 INFO:journalctl@ceph.osd.4.smithi029.stdout:Nov 27 15:34:01 smithi029 bash[161138]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-4 2023-11-27T15:34:02.074 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:01 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:01.938+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:34:02.074 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:01 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:01.938+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:33.059681+0000 front 2023-11-27T15:33:33.059601+0000 (oldest deadline 2023-11-27T15:33:58.359640+0000) 2023-11-27T15:34:02.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:01 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:01.694+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:02.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:01 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:01.694+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:02.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:01 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:01.764+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:02.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:01 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:01.764+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:02.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:02 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:02.182+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:02.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:02 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:02.397+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:02.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:02 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:02.397+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:02.968 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:02 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:02.725+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:02.969 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:02 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:02.649+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:02.969 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:02 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:02.649+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:02.987 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.4.service' 2023-11-27T15:34:03.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:34:03.030 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-11-27T15:34:03.031 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-11-27T15:34:03.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.5 2023-11-27T15:34:03.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:02 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:02.813+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:03.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:02 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:02.813+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:03.323 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:02 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:02.964+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:34:03.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:02 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:02.964+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:33.059681+0000 front 2023-11-27T15:33:33.059601+0000 (oldest deadline 2023-11-27T15:33:58.359640+0000) 2023-11-27T15:34:03.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:03 smithi060 systemd[1]: Stopping Ceph osd.5 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:34:03.397 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:03 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:03.183+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:03.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:03 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:03.297+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:03.668 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:03 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:03.385+0000 7f66186f0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:34:03.669 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:03 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:03.385+0000 7f66186f0700 -1 osd.5 98 *** Got signal Terminated *** 2023-11-27T15:34:03.669 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:03 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:03.385+0000 7f66186f0700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:34:03.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:03 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:03.420+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:03.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:03 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:03.420+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:03.988 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:03 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:03.726+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:03.989 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:03 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:03.664+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:03.989 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:03 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:03.665+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:04.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:03 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:03.823+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:04.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:03 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:03.823+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:04.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:03 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:03.984+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:34:04.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:03 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:03.984+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:33.059681+0000 front 2023-11-27T15:33:33.059601+0000 (oldest deadline 2023-11-27T15:33:58.359640+0000) 2023-11-27T15:34:04.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:04 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:04.292+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:04.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:04 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:04.207+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:04.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:04 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:04.438+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:04.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:04 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:04.438+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:04.965 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:04 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:04.698+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:04.965 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:04 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:04.638+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:04.965 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:04 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:04.638+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:05.290 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:04 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:04.860+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:05.290 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:04 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:04.860+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:05.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:04 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:04.961+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:34:05.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:04 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:04.961+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:33.059681+0000 front 2023-11-27T15:33:33.059601+0000 (oldest deadline 2023-11-27T15:33:58.359640+0000) 2023-11-27T15:34:05.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:05 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:05.286+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:05.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:05 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:05.253+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:05.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:05 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:05.449+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:05.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:05 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:05.449+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:05.955 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:05 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:05.686+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:05.956 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:05 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:05.650+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:05.956 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:05 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:05.650+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:06.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:05 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:05.856+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:06.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:05 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:05.856+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:06.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:05 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:05.952+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:34:06.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:05 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:05.952+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:33.059681+0000 front 2023-11-27T15:33:33.059601+0000 (oldest deadline 2023-11-27T15:33:58.359640+0000) 2023-11-27T15:34:06.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:06 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:06.308+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:06.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:06 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:06.234+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:06.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:06 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:06.474+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:06.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:06 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:06.474+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:06.968 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:06 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:06.671+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:06.968 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:06 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:06.671+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:06.968 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:06 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:06.645+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:06.968 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:06 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:06.645+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:07.302 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:06 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:06.892+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:07.302 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:06 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:06.892+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:07.303 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:06 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:06.892+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:07.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:06 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:06.964+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:34:07.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:06 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:06.964+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:33.059681+0000 front 2023-11-27T15:33:33.059601+0000 (oldest deadline 2023-11-27T15:33:58.359640+0000) 2023-11-27T15:34:07.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:07 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:07.299+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:07.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:07 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:07.234+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:07.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:07 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:07.234+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:07.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:07 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:07.477+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:07.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:07 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:07.477+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:07.986 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:07 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:07.654+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:07.987 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:07 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:07.654+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:07.987 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:07 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:07.652+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:07.987 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:07 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:07.652+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:08.293 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:07 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:07.871+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:08.293 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:07 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:07.871+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:08.293 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:07 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:07.871+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:08.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:07 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:07.983+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.657414+0000 front 2023-11-27T15:33:25.657435+0000 (oldest deadline 2023-11-27T15:33:49.157399+0000) 2023-11-27T15:34:08.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:07 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:07.983+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:33.059681+0000 front 2023-11-27T15:33:33.059601+0000 (oldest deadline 2023-11-27T15:33:58.359640+0000) 2023-11-27T15:34:08.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:07 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5[142838]: 2023-11-27T15:34:07.983+0000 7f6611ce0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:44.761228+0000 front 2023-11-27T15:33:44.761287+0000 (oldest deadline 2023-11-27T15:34:07.061325+0000) 2023-11-27T15:34:08.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:08 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:08.290+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:08.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:08 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:08.223+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:08.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:08 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:08.223+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:08.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:08 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:08.454+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:08.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:08 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:08.454+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:09.070 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:08 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:08.681+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:09.070 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:08 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:08.681+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:09.071 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:08 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:08.656+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:09.071 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:08 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:08.656+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:09.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:08 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:08.859+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:09.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:08 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:08.859+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:09.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:08 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:08.859+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:09.324 INFO:journalctl@ceph.osd.5.smithi060.stdout:Nov 27 15:34:09 smithi060 bash[163134]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-5 2023-11-27T15:34:09.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:09 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:09.333+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:09.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:09 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:09.333+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.785134+0000 front 2023-11-27T15:33:42.784963+0000 (oldest deadline 2023-11-27T15:34:08.684953+0000) 2023-11-27T15:34:09.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:09 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:09.217+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:09.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:09 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:09.217+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:09.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:09 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:09.419+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:09.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:09 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:09.419+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:10.058 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:09 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:09.655+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:10.058 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:09 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:09.655+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:10.059 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:09 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:09.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:10.059 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:09 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:09.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:10.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:09 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:09.849+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:10.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:09 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:09.849+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:10.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:09 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:09.849+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:10.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.5.service' 2023-11-27T15:34:10.492 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:34:10.492 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-11-27T15:34:10.493 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-11-27T15:34:10.493 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.6 2023-11-27T15:34:10.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:10.197+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:10.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:10.197+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:10.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:10.404+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:10.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:10.404+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:10.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:10 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:10.404+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:10.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:10.631+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:10.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:10.631+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:10.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:10.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:10.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:10.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:10.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:10 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:10.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:10.858 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:10.379+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:10.858 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:10.379+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.785134+0000 front 2023-11-27T15:33:42.784963+0000 (oldest deadline 2023-11-27T15:34:08.684953+0000) 2023-11-27T15:34:10.858 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:10 smithi008 systemd[1]: Stopping Ceph osd.6 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:34:11.113 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:11 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:11.027+0000 7f73c2c34700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:34:11.113 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:11 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:11.027+0000 7f73c2c34700 -1 osd.6 98 *** Got signal Terminated *** 2023-11-27T15:34:11.113 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:11 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:11.027+0000 7f73c2c34700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:34:11.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:10.854+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:11.114 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:10.854+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:11.114 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:10 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:10.854+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:11.612 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:11 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:11.329+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:11.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:11 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:11.329+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.785134+0000 front 2023-11-27T15:33:42.784963+0000 (oldest deadline 2023-11-27T15:34:08.684953+0000) 2023-11-27T15:34:11.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:11 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:11.229+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:11.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:11 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:11.229+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:11.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:11 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:11.388+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:11.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:11 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:11.388+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:11.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:11 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:11.388+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:12.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:11 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:11.620+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:12.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:11 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:11.620+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:12.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:11 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:11.577+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:12.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:11 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:11.577+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:12.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:11 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:11.577+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:12.362 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:11 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:11.901+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:12.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:11 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:11.901+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:12.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:11 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:11.901+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:12.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:12 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:12.265+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:12.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:12 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:12.265+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:12.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:12 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:12.416+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:12.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:12 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:12.416+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:12.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:12 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:12.416+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:12.863 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:12 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:12.371+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:12.863 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:12 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:12.371+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.785134+0000 front 2023-11-27T15:33:42.784963+0000 (oldest deadline 2023-11-27T15:34:08.684953+0000) 2023-11-27T15:34:13.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:12 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:12.661+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:13.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:12 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:12.661+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:13.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:12 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:12.586+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:13.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:12 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:12.586+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:13.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:12 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:12.586+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:13.361 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:12 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:12.879+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:13.361 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:12 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:12.879+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:13.361 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:12 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:12.879+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:13.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:13 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:13.358+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:13.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:13 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:13.358+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.785134+0000 front 2023-11-27T15:33:42.784963+0000 (oldest deadline 2023-11-27T15:34:08.684953+0000) 2023-11-27T15:34:13.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:13 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:13.288+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:13.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:13 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:13.288+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:13.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:13 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:13.458+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:13.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:13 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:13.458+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:13.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:13 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:13.458+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:14.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:13 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:13.685+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:14.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:13 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:13.685+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:14.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:13 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:13.618+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:14.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:13 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:13.618+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:14.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:13 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:13.618+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:14.330 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:13 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:13.921+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:14.330 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:13 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:13.921+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:14.330 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:13 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:13.921+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:14.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:14 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:14.326+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:14.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:14 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:14.326+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.785134+0000 front 2023-11-27T15:33:42.784963+0000 (oldest deadline 2023-11-27T15:34:08.684953+0000) 2023-11-27T15:34:14.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:14 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:14.287+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:14.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:14 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:14.287+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:14.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:14 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:14.474+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:14.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:14 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:14.474+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:14.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:14 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:14.474+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:14.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:14 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:14.474+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:15.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:14 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:14.636+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:15.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:14 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:14.636+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:15.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:14 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:14.662+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:15.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:14 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:14.663+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:15.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:14 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:14.663+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:15.296 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:14 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:14.918+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:15.297 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:14 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:14.918+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:15.297 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:14 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:14.918+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:15.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:15 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:15.293+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.884370+0000 front 2023-11-27T15:33:36.884419+0000 (oldest deadline 2023-11-27T15:34:02.784214+0000) 2023-11-27T15:34:15.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:15 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:15.293+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.785134+0000 front 2023-11-27T15:33:42.784963+0000 (oldest deadline 2023-11-27T15:34:08.684953+0000) 2023-11-27T15:34:15.613 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:15 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6[152306]: 2023-11-27T15:34:15.293+0000 7f73bd226700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.986067+0000 front 2023-11-27T15:33:50.986129+0000 (oldest deadline 2023-11-27T15:34:15.086162+0000) 2023-11-27T15:34:15.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:15 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:15.275+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:15.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:15 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:15.275+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:15.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:15 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:15.436+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:15.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:15 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:15.436+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:15.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:15 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:15.436+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:15.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:15 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:15.436+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:16.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:15 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:15.626+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:16.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:15 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:15.626+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:16.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:15 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:15.670+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:16.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:15 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:15.670+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:16.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:15 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:15.670+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:16.362 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:15 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:15.900+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:16.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:15 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:15.900+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:16.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:15 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:15.900+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:16.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:15 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:15.900+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:16.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:16 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:16.238+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:16.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:16 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:16.238+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:16.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:16 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:16.457+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:16.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:16 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:16.457+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:16.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:16 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:16.457+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:16.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:16 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:16.457+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:17.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:16 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:16.620+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:17.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:16 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:16.620+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:17.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:16 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:16.653+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:17.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:16 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:16.653+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:17.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:16 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:16.653+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:17.113 INFO:journalctl@ceph.osd.6.smithi008.stdout:Nov 27 15:34:16 smithi008 bash[286356]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-6 2023-11-27T15:34:17.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:16.919+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:17.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:16.919+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:17.114 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:16.919+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:17.114 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:16 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:16.919+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:17.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:17 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:17.272+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:17.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:17 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:17.272+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:17.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:17 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:17.447+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:17.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:17 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:17.447+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:17.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:17 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:17.447+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:17.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:17 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:17.447+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:18.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:17 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:17.594+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:18.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:17 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:17.594+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:18.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:17 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:17.680+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:18.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:17 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:17.680+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:18.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:17 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:17.680+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:18.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:17 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:17.878+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:18.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:17 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:17.878+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:18.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:17 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:17.878+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:18.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:17 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:17.878+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:18.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:18 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:18.289+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:18.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:18 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:18.289+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:18.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:18 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:18.400+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:18.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:18 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:18.400+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:18.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:18 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:18.400+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:18.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:18 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:18.400+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:18.910 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.6.service' 2023-11-27T15:34:19.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:34:19.000 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-11-27T15:34:19.001 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-11-27T15:34:19.001 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.7 2023-11-27T15:34:19.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:18 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:18.604+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:19.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:18 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:18.604+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:19.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:18 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:18.604+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:19.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:18 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:18.635+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:19.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:18 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:18.635+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:19.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:18 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:18.635+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:19.293 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:19 smithi029 systemd[1]: Stopping Ceph osd.7 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:34:19.362 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:18 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:18.888+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:19.362 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:18 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:18.888+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:19.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:18 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:18.888+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:19.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:18 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:18.888+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:19.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:19.289+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:19.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:19.289+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:19.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:19.289+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:47.965475+0000 front 2023-11-27T15:33:47.965362+0000 (oldest deadline 2023-11-27T15:34:18.565925+0000) 2023-11-27T15:34:19.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:19.366+0000 7fd6b6620700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:34:19.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:19.366+0000 7fd6b6620700 -1 osd.7 98 *** Got signal Terminated *** 2023-11-27T15:34:19.648 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:19.366+0000 7fd6b6620700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:34:19.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:19.380+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:19.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:19.380+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:19.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:19.380+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:19.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:19.380+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:20.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:19 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:19.653+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:20.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:19 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:19.653+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:20.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:19 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:19.653+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:20.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:19 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:19.613+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:20.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:19 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:19.613+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:20.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:19 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:19.613+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:20.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:19 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:19.864+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:20.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:19 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:19.864+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:20.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:19 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:19.864+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:20.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:19 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:19.864+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:20.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:20 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:20.324+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:20.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:20 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:20.324+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:20.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:20 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:20.324+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:47.965475+0000 front 2023-11-27T15:33:47.965362+0000 (oldest deadline 2023-11-27T15:34:18.565925+0000) 2023-11-27T15:34:20.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:20 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:20.398+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:20.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:20 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:20.398+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:20.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:20 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:20.398+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:20.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:20 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:20.398+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:21.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:20 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:20.695+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:21.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:20 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:20.695+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:21.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:20 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:20.695+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:21.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:20 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:20.584+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:21.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:20 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:20.584+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:21.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:20 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:20.584+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:21.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:20 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:20.820+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:21.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:20 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:20.820+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:21.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:20 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:20.820+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:21.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:20 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:20.820+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:21.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:21 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:21.291+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:21.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:21 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:21.291+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:21.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:21 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:21.291+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:47.965475+0000 front 2023-11-27T15:33:47.965362+0000 (oldest deadline 2023-11-27T15:34:18.565925+0000) 2023-11-27T15:34:21.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:21 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:21.382+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:21.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:21 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:21.382+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:21.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:21 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:21.382+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:21.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:21 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:21.382+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:21.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:21 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:21.534+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:21.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:21 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:21.534+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:21.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:21 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:21.534+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:21.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:21 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:21.721+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:21.825 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:21 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:21.722+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:21.825 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:21 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:21.722+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:22.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:21 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:21.814+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:22.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:21 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:21.814+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:22.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:21 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:21.814+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:22.114 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:21 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:21.814+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:22.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:22 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:22.412+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:22.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:22 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:22.412+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:22.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:22 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:22.412+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:22.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:22 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:22.412+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:22.648 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:22 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:22.337+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:22.648 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:22 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:22.337+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:22.648 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:22 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:22.337+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:47.965475+0000 front 2023-11-27T15:33:47.965362+0000 (oldest deadline 2023-11-27T15:34:18.565925+0000) 2023-11-27T15:34:22.648 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:22 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:22.337+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.566299+0000 front 2023-11-27T15:33:58.566327+0000 (oldest deadline 2023-11-27T15:34:22.066445+0000) 2023-11-27T15:34:22.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:22 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:22.696+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:22.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:22 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:22.696+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:22.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:22 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:22.696+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:22.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:22 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:22.539+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:22.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:22 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:22.539+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:22.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:22 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:22.539+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:23.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:22 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:22.843+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:23.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:22 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:22.843+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:23.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:22 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:22.843+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:23.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:22 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:22.843+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:23.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:23.339+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.762951+0000 front 2023-11-27T15:33:35.763063+0000 (oldest deadline 2023-11-27T15:34:01.063204+0000) 2023-11-27T15:34:23.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:23.339+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:41.063742+0000 front 2023-11-27T15:33:41.063754+0000 (oldest deadline 2023-11-27T15:34:06.963858+0000) 2023-11-27T15:34:23.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:23.339+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:47.965475+0000 front 2023-11-27T15:33:47.965362+0000 (oldest deadline 2023-11-27T15:34:18.565925+0000) 2023-11-27T15:34:23.647 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7[147190]: 2023-11-27T15:34:23.339+0000 7fd6aec0e700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.566299+0000 front 2023-11-27T15:33:58.566327+0000 (oldest deadline 2023-11-27T15:34:22.066445+0000) 2023-11-27T15:34:23.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:23.398+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:23.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:23.398+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:23.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:23.398+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:23.649 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:23.398+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:23.649 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:23.398+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:23.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:23 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:23.714+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:23.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:23 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:23.714+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:23.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:23 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:23.714+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:23.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:23 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:23.714+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.070772+0000 front 2023-11-27T15:33:58.070945+0000 (oldest deadline 2023-11-27T15:34:23.370660+0000) 2023-11-27T15:34:23.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:23 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:23.532+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:23.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:23 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:23.532+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:23.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:23 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:23.532+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:24.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:23 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:23.879+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:24.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:23 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:23.879+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:24.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:23 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:23.879+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:24.363 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:23 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:23.879+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:24.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:24 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:24.694+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:24.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:24 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:24.694+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:24.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:24 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:24.694+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:24.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:24 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:24.694+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.070772+0000 front 2023-11-27T15:33:58.070945+0000 (oldest deadline 2023-11-27T15:34:23.370660+0000) 2023-11-27T15:34:24.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:24 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:24.568+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:24.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:24 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:24.568+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:24.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:24 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:24.568+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:24.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:24 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:24.568+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:24.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:24 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:24.415+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:24.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:24 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:24.415+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:24.898 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:24 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:24.415+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:24.898 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:24 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:24.415+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:24.898 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:24 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:24.415+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:25.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:24 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:24.840+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:25.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:24 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:24.840+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:25.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:24 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:24.840+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:25.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:24 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:24.840+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:25.246 INFO:journalctl@ceph.osd.7.smithi029.stdout:Nov 27 15:34:25 smithi029 bash[161400]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-7 2023-11-27T15:34:25.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:25 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:25.370+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:25.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:25 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:25.370+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:25.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:25 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:25.370+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:25.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:25 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:25.370+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:25.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:25 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:25.370+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:25.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:25 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:25.664+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:25.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:25 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:25.664+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:25.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:25 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:25.664+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:25.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:25 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:25.664+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.070772+0000 front 2023-11-27T15:33:58.070945+0000 (oldest deadline 2023-11-27T15:34:23.370660+0000) 2023-11-27T15:34:25.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:25 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:25.531+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:25.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:25 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:25.531+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:25.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:25 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:25.531+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:25.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:25 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:25.531+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:26.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:25 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:25.829+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:26.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:25 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:25.829+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:26.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:25 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:25.829+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:26.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:25 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:25.829+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:26.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:25 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:25.829+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:26.616 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:26 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:26.394+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:26.616 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:26 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:26.394+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:26.616 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:26 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:26.394+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:26.617 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:26 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:26.394+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:26.617 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:26 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:26.394+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:26.625 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.7.service' 2023-11-27T15:34:26.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:34:26.667 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-11-27T15:34:26.668 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-11-27T15:34:26.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.8 2023-11-27T15:34:26.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:26 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:26.516+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:26.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:26 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:26.516+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:26.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:26 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:26.516+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:26.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:26 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:26.516+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:26.825 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:26 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:26.693+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:26.825 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:26 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:26.693+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:26.826 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:26 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:26.693+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:26.826 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:26 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:26.693+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.070772+0000 front 2023-11-27T15:33:58.070945+0000 (oldest deadline 2023-11-27T15:34:23.370660+0000) 2023-11-27T15:34:26.826 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:26 smithi060 systemd[1]: Stopping Ceph osd.8 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:34:27.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:26 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:26.859+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:27.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:26 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:26.859+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:27.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:26 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:26.859+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:27.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:26 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:26.859+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:27.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:26 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:26.859+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:27.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:27.024+0000 7fafa915e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:34:27.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:27.024+0000 7fafa915e700 -1 osd.8 98 *** Got signal Terminated *** 2023-11-27T15:34:27.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:27.024+0000 7fafa915e700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:34:27.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:27 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:27.382+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:27.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:27 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:27.382+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:27.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:27 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:27.382+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:27.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:27 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:27.382+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:27.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:27 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:27.382+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:27.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:27.529+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:27.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:27.529+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:27.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:27.529+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:27.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:27.529+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:27.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:27.717+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:27.825 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:27.717+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:27.825 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:27.717+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:27.825 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:27 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:27.717+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.070772+0000 front 2023-11-27T15:33:58.070945+0000 (oldest deadline 2023-11-27T15:34:23.370660+0000) 2023-11-27T15:34:28.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:27 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:27.818+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:28.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:27 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:27.818+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:28.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:27 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:27.818+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:28.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:27 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:27.818+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:28.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:27 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:27.818+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:28.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:28 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:28.370+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:28.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:28 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:28.370+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:28.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:28 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:28.370+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:28.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:28 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:28.370+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:28.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:28 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:28.370+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:28.823 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:28 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:28.509+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:28.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:28 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:28.509+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:28.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:28 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:28.509+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:28.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:28 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:28.509+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:28.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:28 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:28.726+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:28.824 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:28 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:28.726+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:28.825 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:28 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:28.726+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:28.825 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:28 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:28.726+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.070772+0000 front 2023-11-27T15:33:58.070945+0000 (oldest deadline 2023-11-27T15:34:23.370660+0000) 2023-11-27T15:34:29.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:28 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:28.843+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:29.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:28 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:28.843+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:29.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:28 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:28.843+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:29.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:28 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:28.843+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:29.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:28 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:28.843+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:29.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:29 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:29.322+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:29.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:29 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:29.322+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:29.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:29 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:29.322+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:29.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:29 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:29.322+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:29.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:29 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:29.322+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:29.780 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:29 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:29.477+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:29.780 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:29 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:29.477+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:29.780 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:29 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:29.477+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:29.780 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:29 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:29.477+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:29.780 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:29 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:29.477+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:30.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:29 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:29.776+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:30.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:29 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:29.776+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:30.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:29 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:29.776+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:30.074 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:29 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:29.776+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.070772+0000 front 2023-11-27T15:33:58.070945+0000 (oldest deadline 2023-11-27T15:34:23.370660+0000) 2023-11-27T15:34:30.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:29 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:29.825+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:30.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:29 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:29.825+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:30.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:29 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:29.825+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:30.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:29 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:29.825+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:30.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:29 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:29.825+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:30.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:30 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:30.306+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:30.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:30 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:30.306+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:30.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:30 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:30.306+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:30.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:30 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:30.306+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:30.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:30 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:30.306+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:30.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:30.500+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:30.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:30.500+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:30.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:30.500+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:30.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:30.500+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:30.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:30.500+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:31.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:30 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:30.826+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:31.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:30 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:30.826+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:31.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:30 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:30.826+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:31.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:30 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:30.826+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:31.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:30 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:30.826+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:31.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:30.825+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:31.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:30.825+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:31.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:30.825+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:31.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:30.825+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.070772+0000 front 2023-11-27T15:33:58.070945+0000 (oldest deadline 2023-11-27T15:34:23.370660+0000) 2023-11-27T15:34:31.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:30.825+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:04.471428+0000 front 2023-11-27T15:34:04.471490+0000 (oldest deadline 2023-11-27T15:34:30.371517+0000) 2023-11-27T15:34:31.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:31 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:31.270+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:31.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:31 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:31.270+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:31.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:31 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:31.270+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:31.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:31 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:31.270+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:31.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:31 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:31.270+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:31.823 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:31 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:31.487+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:31.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:31 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:31.487+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:31.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:31 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:31.487+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:31.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:31 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:31.487+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:31.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:31 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:31.487+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:32.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:31 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:31.840+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:32.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:31 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:31.840+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:32.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:31 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:31.840+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:32.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:31 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:31.840+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:32.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:31 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:31.840+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:32.323 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:31 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:31.866+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.467088+0000 front 2023-11-27T15:33:36.467074+0000 (oldest deadline 2023-11-27T15:34:02.367162+0000) 2023-11-27T15:34:32.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:31 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:31.866+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:42.367695+0000 front 2023-11-27T15:33:42.367781+0000 (oldest deadline 2023-11-27T15:34:05.867827+0000) 2023-11-27T15:34:32.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:31 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:31.866+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:52.770270+0000 front 2023-11-27T15:33:52.770280+0000 (oldest deadline 2023-11-27T15:34:18.070118+0000) 2023-11-27T15:34:32.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:31 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:31.866+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.070772+0000 front 2023-11-27T15:33:58.070945+0000 (oldest deadline 2023-11-27T15:34:23.370660+0000) 2023-11-27T15:34:32.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:31 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8[148312]: 2023-11-27T15:34:31.866+0000 7fafa3750700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:04.471428+0000 front 2023-11-27T15:34:04.471490+0000 (oldest deadline 2023-11-27T15:34:30.371517+0000) 2023-11-27T15:34:32.646 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:32 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:32.241+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:32.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:32 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:32.241+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:32.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:32 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:32.241+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:32.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:32 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:32.241+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:32.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:32 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:32.241+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:32.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:32 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:32.489+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:32.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:32 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:32.489+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:32.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:32 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:32.489+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:32.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:32 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:32.489+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:32.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:32 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:32.489+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:33.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:32 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:32.821+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:33.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:32 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:32.821+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:33.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:32 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:32.821+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:33.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:32 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:32.821+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:33.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:32 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:32.821+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:33.324 INFO:journalctl@ceph.osd.8.smithi060.stdout:Nov 27 15:34:32 smithi060 bash[163402]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-8 2023-11-27T15:34:33.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:33 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:33.213+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:33.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:33 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:33.213+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:33.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:33 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:33.213+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:33.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:33 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:33.213+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:33.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:33 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:33.213+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:33.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:33 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:33.213+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:33.823 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:33 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:33.487+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:33.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:33 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:33.487+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:33.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:33 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:33.487+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:33.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:33 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:33.487+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:33.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:33 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:33.487+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:34.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:33 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:33.819+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:34.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:33 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:33.819+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:34.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:33 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:33.819+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:34.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:33 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:33.819+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:34.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:33 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:33.819+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:34.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:33 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:33.819+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.013429+0000 front 2023-11-27T15:34:07.013357+0000 (oldest deadline 2023-11-27T15:34:32.913322+0000) 2023-11-27T15:34:34.185 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.8.service' 2023-11-27T15:34:34.225 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:34:34.226 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-11-27T15:34:34.226 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-11-27T15:34:34.226 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.9 2023-11-27T15:34:34.613 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:34 smithi008 systemd[1]: Stopping Ceph osd.9 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:34:34.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:34 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:34.253+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:34.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:34 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:34.253+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:34.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:34 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:34.253+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:34.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:34 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:34.253+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:34.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:34 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:34.253+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:34.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:34 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:34.253+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:34.823 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:34 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:34.501+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:34.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:34 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:34.501+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:34.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:34 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:34.501+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:34.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:34 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:34.501+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:34.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:34 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:34.501+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:35.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:34.758+0000 7f6a2c2ac700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:34:35.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:34.758+0000 7f6a2c2ac700 -1 osd.9 98 *** Got signal Terminated *** 2023-11-27T15:34:35.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:34.758+0000 7f6a2c2ac700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:34:35.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:34.773+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:35.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:34.773+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:35.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:34.773+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:35.114 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:34.773+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:35.114 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:34.773+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:35.114 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:34 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:34.773+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.013429+0000 front 2023-11-27T15:34:07.013357+0000 (oldest deadline 2023-11-27T15:34:32.913322+0000) 2023-11-27T15:34:35.651 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:35 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:35.286+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:35.672 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:35 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:35.286+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:35.672 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:35 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:35.286+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:35.672 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:35 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:35.286+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:35.672 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:35 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:35.286+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:35.673 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:35 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:35.286+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:35.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:35 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:35.511+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:35.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:35 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:35.511+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:35.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:35 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:35.511+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:35.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:35 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:35.511+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:35.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:35 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:35.511+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:36.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:35 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:35.729+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:36.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:35 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:35.729+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:36.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:35 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:35.729+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:36.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:35 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:35.729+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:36.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:35 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:35.729+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:36.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:35 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:35.729+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.013429+0000 front 2023-11-27T15:34:07.013357+0000 (oldest deadline 2023-11-27T15:34:32.913322+0000) 2023-11-27T15:34:36.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:36 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:36.256+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:36.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:36 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:36.256+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:36.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:36 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:36.256+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:36.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:36 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:36.256+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:36.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:36 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:36.256+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:36.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:36 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:36.256+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:36.823 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:36.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:36.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:36.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:36.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:36.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:36.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:36.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:36.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:36 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:36.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:37.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:36 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:36.724+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:37.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:36 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:36.724+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:37.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:36 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:36.724+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:37.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:36 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:36.724+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:37.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:36 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:36.724+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:37.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:36 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:36.724+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.013429+0000 front 2023-11-27T15:34:07.013357+0000 (oldest deadline 2023-11-27T15:34:32.913322+0000) 2023-11-27T15:34:37.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:37 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:37.212+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:37.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:37 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:37.212+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:37.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:37 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:37.212+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:37.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:37 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:37.212+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:37.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:37 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:37.212+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:37.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:37 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:37.212+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:37.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:37 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:37.500+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:37.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:37 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:37.500+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:37.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:37 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:37.500+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:37.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:37 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:37.500+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:37.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:37 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:37.500+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:38.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:37 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:37.756+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:38.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:37 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:37.756+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:38.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:37 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:37.756+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:38.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:37 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:37.756+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:38.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:37 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:37.756+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:38.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:37 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:37.756+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.013429+0000 front 2023-11-27T15:34:07.013357+0000 (oldest deadline 2023-11-27T15:34:32.913322+0000) 2023-11-27T15:34:38.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:38 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:38.174+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:38.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:38 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:38.174+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:38.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:38 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:38.174+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:38.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:38 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:38.174+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:38.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:38 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:38.174+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:38.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:38 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:38.174+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:38.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:38 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:38.174+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:38.823 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:38 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:38.479+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:38.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:38 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:38.479+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:38.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:38 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:38.479+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:38.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:38 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:38.479+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:38.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:38 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:38.479+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:39.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:38 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:38.707+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:39.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:38 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:38.707+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:39.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:38 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:38.707+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:39.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:38 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:38.707+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:39.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:38 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:38.707+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:39.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:38 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:38.707+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.013429+0000 front 2023-11-27T15:34:07.013357+0000 (oldest deadline 2023-11-27T15:34:32.913322+0000) 2023-11-27T15:34:39.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:39 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:39.155+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:39.647 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:39 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:39.155+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:39.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:39 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:39.155+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:39.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:39 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:39.155+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:39.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:39 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:39.155+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:39.648 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:39 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:39.155+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:39.649 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:39 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:39.155+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:39.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:39.523+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:39.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:39.523+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:39.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:39.523+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:39.826 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:39.523+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:39.826 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:39.523+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:39.826 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:39.523+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:40.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:39 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:39.670+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:23.204168+0000 front 2023-11-27T15:33:23.204193+0000 (oldest deadline 2023-11-27T15:33:48.504464+0000) 2023-11-27T15:34:40.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:39 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:39.670+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:34.906995+0000 front 2023-11-27T15:33:34.906955+0000 (oldest deadline 2023-11-27T15:33:59.607053+0000) 2023-11-27T15:34:40.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:39 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:39.670+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.108653+0000 front 2023-11-27T15:33:43.108679+0000 (oldest deadline 2023-11-27T15:34:06.608950+0000) 2023-11-27T15:34:40.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:39 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:39.670+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:50.110080+0000 front 2023-11-27T15:33:50.110040+0000 (oldest deadline 2023-11-27T15:34:15.410053+0000) 2023-11-27T15:34:40.113 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:39 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:39.670+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:34:00.011691+0000 front 2023-11-27T15:34:00.011836+0000 (oldest deadline 2023-11-27T15:34:25.311853+0000) 2023-11-27T15:34:40.114 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:39 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9[157800]: 2023-11-27T15:34:39.670+0000 7f6a2589c700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.013429+0000 front 2023-11-27T15:34:07.013357+0000 (oldest deadline 2023-11-27T15:34:32.913322+0000) 2023-11-27T15:34:40.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:40 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:40.136+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:40.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:40 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:40.136+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:40.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:40 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:40.136+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:40.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:40 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:40.136+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:40.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:40 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:40.136+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:40.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:40 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:40.136+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:40.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:40 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:40.136+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:40.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:40 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:40.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:40.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:40 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:40.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:40.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:40 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:40.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:40.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:40 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:40.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:40.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:40 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:40.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:40.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:40 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:40.522+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:41.112 INFO:journalctl@ceph.osd.9.smithi008.stdout:Nov 27 15:34:40 smithi008 bash[286638]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-9 2023-11-27T15:34:41.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:41 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:41.102+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:41.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:41 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:41.102+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:41.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:41 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:41.102+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:41.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:41 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:41.102+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:41.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:41 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:41.102+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:41.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:41 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:41.102+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:41.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:41 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:41.102+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:41.823 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:41 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:41.558+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:41.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:41 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:41.558+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:41.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:41 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:41.558+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:41.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:41 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:41.558+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:41.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:41 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:41.558+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:41.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:41 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:41.558+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:42.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:42 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:42.095+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:42.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:42 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:42.095+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:42.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:42 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:42.095+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:42.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:42 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:42.095+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:42.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:42 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:42.095+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:42.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:42 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:42.095+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:42.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:42 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:42.095+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:42.969 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.9.service' 2023-11-27T15:34:43.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:34:43.059 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-11-27T15:34:43.059 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-11-27T15:34:43.060 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.10 2023-11-27T15:34:43.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:42 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:42.585+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:43.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:42 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:42.585+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:43.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:42 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:42.585+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:43.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:42 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:42.585+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:43.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:42 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:42.585+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:43.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:42 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:42.585+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:43.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 systemd[1]: Stopping Ceph osd.10 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:34:43.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:43.130+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:43.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:43.130+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:43.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:43.130+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:43.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:43.130+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:43.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:43.130+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:43.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:43.130+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:43.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:43.130+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:43.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:43.422+0000 7f32da909700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:34:43.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:43.422+0000 7f32da909700 -1 osd.10 98 *** Got signal Terminated *** 2023-11-27T15:34:43.897 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:43 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:43.422+0000 7f32da909700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:34:44.073 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:43 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:43.620+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:44.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:43 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:43.620+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:44.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:43 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:43.620+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:44.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:43 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:43.620+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:44.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:43 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:43.620+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:44.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:43 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:43.620+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:44.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:44 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:44.091+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:44.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:44 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:44.091+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:44.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:44 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:44.091+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:44.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:44 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:44.091+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:44.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:44 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:44.091+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:44.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:44 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:44.091+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:44.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:44 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:44.091+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:45.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:44 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:44.642+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:45.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:44 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:44.642+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:45.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:44 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:44.642+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:45.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:44 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:44.642+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:45.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:44 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:44.642+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:45.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:44 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:44.642+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:45.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:45 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:45.112+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:45.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:45 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:45.112+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:45.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:45 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:45.112+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:45.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:45 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:45.112+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:45.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:45 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:45.112+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:45.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:45 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:45.112+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:45.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:45 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:45.112+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:46.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:45.672+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:46.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:45.672+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:46.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:45.672+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:46.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:45.672+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:46.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:45.672+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:46.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:45 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:45.672+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:46.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:46 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:46.108+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:46.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:46 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:46.108+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:46.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:46 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:46.108+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:46.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:46 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:46.108+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:46.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:46 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:46.108+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:46.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:46 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:46.108+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:46.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:46 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:46.108+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:47.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:46 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:46.713+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:47.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:46 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:46.713+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:47.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:46 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:46.713+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:47.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:46 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:46.713+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:47.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:46 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:46.713+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:47.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:46 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:46.713+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:47.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:47.068+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:47.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:47.068+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:47.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:47.068+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:47.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:47.068+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:47.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:47.068+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:47.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:47.068+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:47.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:47 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:47.068+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:48.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:47 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:47.757+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:48.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:47 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:47.757+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:48.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:47 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:47.757+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:48.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:47 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:47.757+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:48.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:47 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:47.757+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:48.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:47 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:47.757+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:48.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:48.022+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.220145+0000 front 2023-11-27T15:33:25.220119+0000 (oldest deadline 2023-11-27T15:33:49.919912+0000) 2023-11-27T15:34:48.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:48.022+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:35.021938+0000 front 2023-11-27T15:33:35.021965+0000 (oldest deadline 2023-11-27T15:34:00.922211+0000) 2023-11-27T15:34:48.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:48.022+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:43.823541+0000 front 2023-11-27T15:33:43.823659+0000 (oldest deadline 2023-11-27T15:34:09.623977+0000) 2023-11-27T15:34:48.397 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:48.022+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6814 osd.3 since back 2023-11-27T15:33:49.624487+0000 front 2023-11-27T15:33:49.624471+0000 (oldest deadline 2023-11-27T15:34:14.324536+0000) 2023-11-27T15:34:48.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:48.022+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:57.825375+0000 front 2023-11-27T15:33:57.825411+0000 (oldest deadline 2023-11-27T15:34:22.425993+0000) 2023-11-27T15:34:48.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:48.022+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:08.227556+0000 front 2023-11-27T15:34:08.227474+0000 (oldest deadline 2023-11-27T15:34:32.327417+0000) 2023-11-27T15:34:48.398 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:48 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10[152618]: 2023-11-27T15:34:48.022+0000 7f32d36f8700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:12.328094+0000 front 2023-11-27T15:34:12.328143+0000 (oldest deadline 2023-11-27T15:34:37.627944+0000) 2023-11-27T15:34:49.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:48 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:48.724+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:49.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:48 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:48.724+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:49.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:48 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:48.724+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:49.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:48 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:48.724+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:49.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:48 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:48.724+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:49.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:48 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:48.724+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:49.204 INFO:journalctl@ceph.osd.10.smithi029.stdout:Nov 27 15:34:49 smithi029 bash[161664]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-10 2023-11-27T15:34:50.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:49.703+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:50.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:49.703+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:50.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:49.703+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:50.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:49.703+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:50.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:49.703+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:50.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:49.703+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:50.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:49 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:49.703+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6820 osd.7 since back 2023-11-27T15:34:18.904968+0000 front 2023-11-27T15:34:18.905025+0000 (oldest deadline 2023-11-27T15:34:49.505534+0000) 2023-11-27T15:34:50.582 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.10.service' 2023-11-27T15:34:50.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:34:50.623 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-11-27T15:34:50.623 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-11-27T15:34:50.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.11 2023-11-27T15:34:50.974 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 systemd[1]: Stopping Ceph osd.11 for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:34:50.975 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:50.665+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:50.975 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:50.665+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:50.975 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:50.665+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:50.975 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:50.665+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:50.975 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:50.665+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:50.975 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:50.665+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:50.976 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:50.665+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6820 osd.7 since back 2023-11-27T15:34:18.904968+0000 front 2023-11-27T15:34:18.905025+0000 (oldest deadline 2023-11-27T15:34:49.505534+0000) 2023-11-27T15:34:51.323 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:50.971+0000 7f19285c9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:34:51.324 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:50.971+0000 7f19285c9700 -1 osd.11 98 *** Got signal Terminated *** 2023-11-27T15:34:51.324 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:50 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:50.971+0000 7f19285c9700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:34:52.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:51.621+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:52.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:51.621+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:52.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:51.621+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:52.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:51.621+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:52.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:51.621+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:52.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:51.621+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:52.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:51 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:51.621+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6820 osd.7 since back 2023-11-27T15:34:18.904968+0000 front 2023-11-27T15:34:18.905025+0000 (oldest deadline 2023-11-27T15:34:49.505534+0000) 2023-11-27T15:34:53.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:52 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:52.652+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:53.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:52 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:52.652+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:53.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:52 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:52.652+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:53.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:52 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:52.652+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:53.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:52 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:52.652+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:53.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:52 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:52.652+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:53.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:52 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:52.652+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6820 osd.7 since back 2023-11-27T15:34:18.904968+0000 front 2023-11-27T15:34:18.905025+0000 (oldest deadline 2023-11-27T15:34:49.505534+0000) 2023-11-27T15:34:54.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:53 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:53.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:54.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:53 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:53.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:54.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:53 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:53.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:54.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:53 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:53.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:54.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:53 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:53.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:54.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:53 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:53.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:54.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:53 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:53.609+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6820 osd.7 since back 2023-11-27T15:34:18.904968+0000 front 2023-11-27T15:34:18.905025+0000 (oldest deadline 2023-11-27T15:34:49.505534+0000) 2023-11-27T15:34:55.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:54 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:54.588+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:55.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:54 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:54.588+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:55.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:54 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:54.588+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:55.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:54 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:54.588+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:55.074 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:54 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:54.588+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:55.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:54 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:54.588+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:55.075 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:54 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:54.588+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6820 osd.7 since back 2023-11-27T15:34:18.904968+0000 front 2023-11-27T15:34:18.905025+0000 (oldest deadline 2023-11-27T15:34:49.505534+0000) 2023-11-27T15:34:55.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:55 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:55.564+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2023-11-27T15:33:25.796227+0000 front 2023-11-27T15:33:25.796137+0000 (oldest deadline 2023-11-27T15:33:51.096084+0000) 2023-11-27T15:34:55.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:55 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:55.564+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6804 osd.1 since back 2023-11-27T15:33:36.397249+0000 front 2023-11-27T15:33:36.397287+0000 (oldest deadline 2023-11-27T15:34:00.397953+0000) 2023-11-27T15:34:55.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:55 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:55.564+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6804 osd.2 since back 2023-11-27T15:33:40.398358+0000 front 2023-11-27T15:33:40.398384+0000 (oldest deadline 2023-11-27T15:34:10.399254+0000) 2023-11-27T15:34:55.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:55 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:55.564+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6812 osd.4 since back 2023-11-27T15:33:58.501817+0000 front 2023-11-27T15:33:58.501571+0000 (oldest deadline 2023-11-27T15:34:24.401626+0000) 2023-11-27T15:34:55.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:55 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:55.564+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6812 osd.5 since back 2023-11-27T15:34:07.902707+0000 front 2023-11-27T15:34:07.902818+0000 (oldest deadline 2023-11-27T15:34:29.002859+0000) 2023-11-27T15:34:55.824 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:55 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:55.564+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.8:6822 osd.6 since back 2023-11-27T15:34:14.804450+0000 front 2023-11-27T15:34:14.804485+0000 (oldest deadline 2023-11-27T15:34:38.904479+0000) 2023-11-27T15:34:55.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:55 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:55.564+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.29:6820 osd.7 since back 2023-11-27T15:34:18.904968+0000 front 2023-11-27T15:34:18.905025+0000 (oldest deadline 2023-11-27T15:34:49.505534+0000) 2023-11-27T15:34:55.825 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:55 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11[153867]: 2023-11-27T15:34:55.564+0000 7f19213b8700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.60:6820 osd.8 since back 2023-11-27T15:34:30.006494+0000 front 2023-11-27T15:34:30.006520+0000 (oldest deadline 2023-11-27T15:34:54.607092+0000) 2023-11-27T15:34:57.073 INFO:journalctl@ceph.osd.11.smithi060.stdout:Nov 27 15:34:56 smithi060 bash[163663]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-osd-11 2023-11-27T15:34:58.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@osd.11.service' 2023-11-27T15:34:58.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:34:58.141 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-11-27T15:34:58.141 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-11-27T15:34:58.141 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.a 2023-11-27T15:34:58.612 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:34:58 smithi008 systemd[1]: Stopping Ceph mds.a for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:34:59.113 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:34:58 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-a[163286]: 2023-11-27T15:34:58.670+0000 7f75d65bc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:34:59.113 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:34:58 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-a[163286]: 2023-11-27T15:34:58.670+0000 7f75d65bc700 -1 mds.a *** got signal Terminated *** 2023-11-27T15:35:03.362 INFO:journalctl@ceph.mds.a.smithi008.stdout:Nov 27 15:35:02 smithi008 bash[286908]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-a 2023-11-27T15:35:03.506 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.a.service' 2023-11-27T15:35:03.592 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:03.593 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-11-27T15:35:03.593 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-11-27T15:35:03.593 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.d 2023-11-27T15:35:03.863 INFO:journalctl@ceph.mds.d.smithi008.stdout:Nov 27 15:35:03 smithi008 systemd[1]: Stopping Ceph mds.d for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:04.363 INFO:journalctl@ceph.mds.d.smithi008.stdout:Nov 27 15:35:04 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-d[162234]: 2023-11-27T15:35:04.103+0000 7f8355dde700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:04.363 INFO:journalctl@ceph.mds.d.smithi008.stdout:Nov 27 15:35:04 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-d[162234]: 2023-11-27T15:35:04.103+0000 7f8355dde700 -1 mds.d *** got signal Terminated *** 2023-11-27T15:35:08.362 INFO:journalctl@ceph.mds.d.smithi008.stdout:Nov 27 15:35:07 smithi008 bash[287051]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-d 2023-11-27T15:35:08.423 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.d.service' 2023-11-27T15:35:08.506 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:08.506 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-11-27T15:35:08.506 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-11-27T15:35:08.506 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.g 2023-11-27T15:35:08.863 INFO:journalctl@ceph.mds.g.smithi008.stdout:Nov 27 15:35:08 smithi008 systemd[1]: Stopping Ceph mds.g for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:09.362 INFO:journalctl@ceph.mds.g.smithi008.stdout:Nov 27 15:35:09 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-g[162848]: 2023-11-27T15:35:09.011+0000 7ff4a465d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:09.363 INFO:journalctl@ceph.mds.g.smithi008.stdout:Nov 27 15:35:09 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-g[162848]: 2023-11-27T15:35:09.011+0000 7ff4a465d700 -1 mds.g *** got signal Terminated *** 2023-11-27T15:35:11.363 INFO:journalctl@ceph.mds.g.smithi008.stdout:Nov 27 15:35:11 smithi008 bash[287196]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-g 2023-11-27T15:35:11.479 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.g.service' 2023-11-27T15:35:11.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:11.563 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-11-27T15:35:11.564 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-11-27T15:35:11.564 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.j 2023-11-27T15:35:11.862 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:35:11 smithi008 systemd[1]: Stopping Ceph mds.j for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:12.612 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:35:12 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-j[161845]: 2023-11-27T15:35:12.153+0000 7f5eebc13700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:12.613 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:35:12 smithi008 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-j[161845]: 2023-11-27T15:35:12.153+0000 7f5eebc13700 -1 mds.j *** got signal Terminated *** 2023-11-27T15:35:16.639 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:35:16 smithi008 bash[287342]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-j 2023-11-27T15:35:16.877 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.j.service' 2023-11-27T15:35:16.939 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:35:16 smithi008 systemd[1]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.j.service: Succeeded. 2023-11-27T15:35:16.939 INFO:journalctl@ceph.mds.j.smithi008.stdout:Nov 27 15:35:16 smithi008 systemd[1]: Stopped Ceph mds.j for 9f287ba8-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:35:16.961 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:16.962 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-11-27T15:35:16.962 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-11-27T15:35:16.962 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.b 2023-11-27T15:35:17.338 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:35:17 smithi029 systemd[1]: Stopping Ceph mds.b for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:17.647 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:35:17 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-b[156484]: 2023-11-27T15:35:17.334+0000 7f38ff24e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:17.647 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:35:17 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-b[156484]: 2023-11-27T15:35:17.334+0000 7f38ff24e700 -1 mds.b *** got signal Terminated *** 2023-11-27T15:35:18.984 INFO:journalctl@ceph.mds.b.smithi029.stdout:Nov 27 15:35:18 smithi029 bash[161937]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-b 2023-11-27T15:35:19.183 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.b.service' 2023-11-27T15:35:19.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:19.222 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-11-27T15:35:19.223 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-11-27T15:35:19.223 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.e 2023-11-27T15:35:19.613 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:35:19 smithi029 systemd[1]: Stopping Ceph mds.e for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:19.897 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:35:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-e[159308]: 2023-11-27T15:35:19.609+0000 7ff19f92b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:19.897 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:35:19 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-e[159308]: 2023-11-27T15:35:19.609+0000 7ff19f92b700 -1 mds.e *** got signal Terminated *** 2023-11-27T15:35:23.181 INFO:journalctl@ceph.mds.e.smithi029.stdout:Nov 27 15:35:23 smithi029 bash[162078]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-e 2023-11-27T15:35:23.379 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.e.service' 2023-11-27T15:35:23.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:23.414 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-11-27T15:35:23.414 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-11-27T15:35:23.414 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.h 2023-11-27T15:35:23.647 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:35:23 smithi029 systemd[1]: Stopping Ceph mds.h for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:24.147 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:35:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-h[156142]: 2023-11-27T15:35:23.792+0000 7f5e99c66700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:24.147 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:35:23 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-h[156142]: 2023-11-27T15:35:23.792+0000 7f5e99c66700 -1 mds.h *** got signal Terminated *** 2023-11-27T15:35:26.966 INFO:journalctl@ceph.mds.h.smithi029.stdout:Nov 27 15:35:26 smithi029 bash[162218]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-h 2023-11-27T15:35:27.167 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.h.service' 2023-11-27T15:35:27.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:27.204 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-11-27T15:35:27.204 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-11-27T15:35:27.204 DEBUG:teuthology.orchestra.run.smithi029:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.k 2023-11-27T15:35:27.579 INFO:journalctl@ceph.mds.k.smithi029.stdout:Nov 27 15:35:27 smithi029 systemd[1]: Stopping Ceph mds.k for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:27.897 INFO:journalctl@ceph.mds.k.smithi029.stdout:Nov 27 15:35:27 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-k[155449]: 2023-11-27T15:35:27.575+0000 7fb0791b0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:27.897 INFO:journalctl@ceph.mds.k.smithi029.stdout:Nov 27 15:35:27 smithi029 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-k[155449]: 2023-11-27T15:35:27.575+0000 7fb0791b0700 -1 mds.k *** got signal Terminated *** 2023-11-27T15:35:29.535 INFO:journalctl@ceph.mds.k.smithi029.stdout:Nov 27 15:35:29 smithi029 bash[162358]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-k 2023-11-27T15:35:29.757 DEBUG:teuthology.orchestra.run.smithi029:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.k.service' 2023-11-27T15:35:29.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:29.791 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-11-27T15:35:29.791 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-11-27T15:35:29.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.c 2023-11-27T15:35:30.074 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:35:29 smithi060 systemd[1]: Stopping Ceph mds.c for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:30.573 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:35:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-c[157452]: 2023-11-27T15:35:30.140+0000 7fe7fd521700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:30.574 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:35:30 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-c[157452]: 2023-11-27T15:35:30.140+0000 7fe7fd521700 -1 mds.c *** got signal Terminated *** 2023-11-27T15:35:34.575 INFO:journalctl@ceph.mds.c.smithi060.stdout:Nov 27 15:35:34 smithi060 bash[163940]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-c 2023-11-27T15:35:34.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.c.service' 2023-11-27T15:35:34.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:34.659 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-11-27T15:35:34.659 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-11-27T15:35:34.659 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.f 2023-11-27T15:35:35.024 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:35:34 smithi060 systemd[1]: Stopping Ceph mds.f for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:35.324 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:35:35 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-f[157111]: 2023-11-27T15:35:35.020+0000 7f460eb33700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:35.324 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:35:35 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-f[157111]: 2023-11-27T15:35:35.020+0000 7f460eb33700 -1 mds.f *** got signal Terminated *** 2023-11-27T15:35:39.073 INFO:journalctl@ceph.mds.f.smithi060.stdout:Nov 27 15:35:38 smithi060 bash[164085]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-f 2023-11-27T15:35:39.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.f.service' 2023-11-27T15:35:39.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:39.148 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-11-27T15:35:39.148 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-11-27T15:35:39.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i 2023-11-27T15:35:39.517 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:35:39 smithi060 systemd[1]: Stopping Ceph mds.i for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:39.823 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:35:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[160783]: 2023-11-27T15:35:39.513+0000 7f8b5e402700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:39.824 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:35:39 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i[160783]: 2023-11-27T15:35:39.513+0000 7f8b5e402700 -1 mds.i *** got signal Terminated *** 2023-11-27T15:35:43.293 INFO:journalctl@ceph.mds.i.smithi060.stdout:Nov 27 15:35:42 smithi060 bash[164223]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-i 2023-11-27T15:35:43.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.i.service' 2023-11-27T15:35:43.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:43.333 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-11-27T15:35:43.333 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-11-27T15:35:43.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.l 2023-11-27T15:35:43.573 INFO:journalctl@ceph.mds.l.smithi060.stdout:Nov 27 15:35:43 smithi060 systemd[1]: Stopping Ceph mds.l for 9f287ba8-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:35:44.073 INFO:journalctl@ceph.mds.l.smithi060.stdout:Nov 27 15:35:43 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-l[156770]: 2023-11-27T15:35:43.704+0000 7f3fd9a32700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:35:44.074 INFO:journalctl@ceph.mds.l.smithi060.stdout:Nov 27 15:35:43 smithi060 ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-l[156770]: 2023-11-27T15:35:43.704+0000 7f3fd9a32700 -1 mds.l *** got signal Terminated *** 2023-11-27T15:35:47.074 INFO:journalctl@ceph.mds.l.smithi060.stdout:Nov 27 15:35:46 smithi060 bash[164364]: ceph-9f287ba8-8d37-11ee-95a2-87774f69a715-mds-l 2023-11-27T15:35:47.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9f287ba8-8d37-11ee-95a2-87774f69a715@mds.l.service' 2023-11-27T15:35:47.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:35:47.121 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-11-27T15:35:47.121 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 --force --keep-logs 2023-11-27T15:35:51.635 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 --force --keep-logs 2023-11-27T15:35:53.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 --force --keep-logs 2023-11-27T15:35:55.705 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:35:55.787 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:35:55.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:35:55.856 INFO:tasks.cephadm:Archiving crash dumps... 2023-11-27T15:35:55.859 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809/remote/smithi008/crash 2023-11-27T15:35:55.860 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/crash -- . 2023-11-27T15:35:55.946 INFO:teuthology.orchestra.run.smithi008.stderr:tar: /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-11-27T15:35:55.947 INFO:teuthology.orchestra.run.smithi008.stderr:tar: Error is not recoverable: exiting now 2023-11-27T15:35:55.950 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809/remote/smithi029/crash 2023-11-27T15:35:55.951 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/crash -- . 2023-11-27T15:35:55.997 INFO:teuthology.orchestra.run.smithi029.stderr:tar: /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-11-27T15:35:55.998 INFO:teuthology.orchestra.run.smithi029.stderr:tar: Error is not recoverable: exiting now 2023-11-27T15:35:56.000 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809/remote/smithi060/crash 2023-11-27T15:35:56.001 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/crash -- . 2023-11-27T15:35:56.032 INFO:teuthology.orchestra.run.smithi060.stderr:tar: /var/lib/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-11-27T15:35:56.033 INFO:teuthology.orchestra.run.smithi060.stderr:tar: Error is not recoverable: exiting now 2023-11-27T15:35:56.034 INFO:tasks.cephadm:Checking cluster log for badness... 2023-11-27T15:35:56.034 DEBUG:teuthology.orchestra.run.smithi008:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-27T15:35:56.115 INFO:teuthology.orchestra.run.smithi008.stderr:grep: /var/log/ceph/9f287ba8-8d37-11ee-95a2-87774f69a715/ceph.log: No such file or directory 2023-11-27T15:35:56.119 INFO:tasks.cephadm:Compressing logs... 2023-11-27T15:35:56.120 DEBUG:teuthology.orchestra.run.smithi008:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:35:56.124 DEBUG:teuthology.orchestra.run.smithi029:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:35:56.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:35:56.155 INFO:teuthology.orchestra.run.smithi060.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-27T15:35:56.157 INFO:teuthology.orchestra.run.smithi029.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-27T15:35:56.201 INFO:teuthology.orchestra.run.smithi008.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-27T15:39:27.613 INFO:tasks.cephadm:Archiving logs... 2023-11-27T15:39:27.614 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809/remote/smithi008/log 2023-11-27T15:39:27.616 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-27T15:39:41.313 DEBUG:teuthology.misc:Transferring archived files from smithi029:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809/remote/smithi029/log 2023-11-27T15:39:41.314 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-27T15:40:02.664 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809/remote/smithi060/log 2023-11-27T15:40:02.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-27T15:40:10.775 INFO:tasks.cephadm:Removing cluster... 2023-11-27T15:40:10.776 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 --force 2023-11-27T15:40:12.399 DEBUG:teuthology.orchestra.run.smithi029:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 --force 2023-11-27T15:40:13.463 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9f287ba8-8d37-11ee-95a2-87774f69a715 --force 2023-11-27T15:40:14.395 INFO:tasks.cephadm:Removing cephadm ... 2023-11-27T15:40:14.395 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-27T15:40:14.445 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-27T15:40:14.467 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-27T15:40:14.485 INFO:tasks.cephadm:Teardown complete 2023-11-27T15:40:14.485 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-27T15:40:14.499 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-27T15:40:14.499 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-27T15:40:14.504 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-27T15:40:14.512 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-27T15:40:14.648 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-27T15:40:14.659 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-27T15:40:14.659 DEBUG:teuthology.orchestra.run.smithi008:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:40:14.665 DEBUG:teuthology.orchestra.run.smithi029:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:40:14.667 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:40:14.684 INFO:teuthology.orchestra.run.smithi029.stderr:bash: ntpq: command not found 2023-11-27T15:40:14.684 INFO:teuthology.orchestra.run.smithi060.stderr:bash: ntpq: command not found 2023-11-27T15:40:14.688 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:40:14.688 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2023-11-27T15:40:14.688 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 10 -994us[-1028us] +/- 96ms 2023-11-27T15:40:14.689 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv02.front.sepia.ceph.com 3 6 377 7 -3222us[-3256us] +/- 65ms 2023-11-27T15:40:14.689 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +4769us[+4736us] +/- 75ms 2023-11-27T15:40:14.689 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:40:14.690 INFO:teuthology.orchestra.run.smithi029.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:40:14.690 INFO:teuthology.orchestra.run.smithi029.stdout:=============================================================================== 2023-11-27T15:40:14.690 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 7 -940us[ -940us] +/- 96ms 2023-11-27T15:40:14.691 INFO:teuthology.orchestra.run.smithi029.stdout:^* hv02.front.sepia.ceph.com 3 6 377 7 -3174us[-3213us] +/- 65ms 2023-11-27T15:40:14.691 INFO:teuthology.orchestra.run.smithi029.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +4854us[+4815us] +/- 75ms 2023-11-27T15:40:14.691 INFO:teuthology.orchestra.run.smithi029.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:40:14.702 INFO:teuthology.orchestra.run.smithi008.stderr:bash: ntpq: command not found 2023-11-27T15:40:14.722 INFO:teuthology.orchestra.run.smithi008.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:40:14.722 INFO:teuthology.orchestra.run.smithi008.stdout:=============================================================================== 2023-11-27T15:40:14.722 INFO:teuthology.orchestra.run.smithi008.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 -1036us[-1021us] +/- 96ms 2023-11-27T15:40:14.722 INFO:teuthology.orchestra.run.smithi008.stdout:^* hv02.front.sepia.ceph.com 3 6 377 9 -3246us[-3231us] +/- 65ms 2023-11-27T15:40:14.722 INFO:teuthology.orchestra.run.smithi008.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +4739us[+4754us] +/- 75ms 2023-11-27T15:40:14.722 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:40:14.723 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-27T15:40:14.733 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-27T15:40:14.733 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-27T15:40:14.797 DEBUG:teuthology.orchestra.run.smithi008:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-27T15:40:15.152 DEBUG:teuthology.orchestra.run.smithi029:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-27T15:40:15.258 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-27T15:40:15.366 DEBUG:teuthology.orchestra.run.smithi008:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T15:40:15.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:40:15.444 DEBUG:teuthology.orchestra.run.smithi029:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T15:40:15.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:40:15.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T15:40:15.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:40:15.516 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-27T15:40:15.528 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-27T15:40:15.564 INFO:teuthology.task.internal:Duration was 2620.899904 seconds 2023-11-27T15:40:15.565 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-27T15:40:15.634 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-27T15:40:15.634 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-27T15:40:15.639 DEBUG:teuthology.orchestra.run.smithi029:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-27T15:40:15.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-27T15:40:15.692 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T15:40:15.693 INFO:teuthology.orchestra.run.smithi029.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T15:40:15.798 INFO:teuthology.orchestra.run.smithi008.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T15:40:16.199 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-27T15:40:16.200 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi008.front.sepia.ceph.com 2023-11-27T15:40:16.200 DEBUG:teuthology.orchestra.run.smithi008:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-27T15:40:16.287 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi029.front.sepia.ceph.com 2023-11-27T15:40:16.287 DEBUG:teuthology.orchestra.run.smithi029:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-27T15:40:16.315 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2023-11-27T15:40:16.316 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-27T15:40:16.344 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-27T15:40:16.344 DEBUG:teuthology.orchestra.run.smithi008:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:40:16.350 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:40:16.359 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:40:16.479 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-27T15:40:16.480 DEBUG:teuthology.orchestra.run.smithi008:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-27T15:40:16.485 DEBUG:teuthology.orchestra.run.smithi029:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-27T15:40:16.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-27T15:40:16.694 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-27T15:40:16.706 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-27T15:40:16.706 DEBUG:teuthology.orchestra.run.smithi008:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-27T15:40:16.784 DEBUG:teuthology.orchestra.run.smithi029:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-27T15:40:16.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-27T15:40:16.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-27T15:40:16.855 DEBUG:teuthology.orchestra.run.smithi008:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:40:16.860 DEBUG:teuthology.orchestra.run.smithi029:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:40:16.863 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:40:16.889 INFO:teuthology.orchestra.run.smithi029.stdout:kernel.core_pattern = core 2023-11-27T15:40:16.913 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2023-11-27T15:40:16.932 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = core 2023-11-27T15:40:16.980 DEBUG:teuthology.orchestra.run.smithi008:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:40:17.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:40:17.022 DEBUG:teuthology.orchestra.run.smithi029:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:40:17.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:40:17.039 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:40:17.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:40:17.056 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-27T15:40:17.070 INFO:teuthology.task.internal:Transferring archived files... 2023-11-27T15:40:17.071 DEBUG:teuthology.misc:Transferring archived files from smithi008:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809/remote/smithi008 2023-11-27T15:40:17.071 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-27T15:40:17.157 DEBUG:teuthology.misc:Transferring archived files from smithi029:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809/remote/smithi029 2023-11-27T15:40:17.157 DEBUG:teuthology.orchestra.run.smithi029:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-27T15:40:17.199 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809/remote/smithi060 2023-11-27T15:40:17.200 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-27T15:40:17.242 INFO:teuthology.task.internal:Removing archive directory... 2023-11-27T15:40:17.242 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-27T15:40:17.247 DEBUG:teuthology.orchestra.run.smithi029:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-27T15:40:17.250 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-27T15:40:17.293 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-27T15:40:17.305 INFO:teuthology.task.internal:Not uploading archives. 2023-11-27T15:40:17.305 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-27T15:40:17.326 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-27T15:40:17.326 DEBUG:teuthology.orchestra.run.smithi008:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-27T15:40:17.332 DEBUG:teuthology.orchestra.run.smithi029:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-27T15:40:17.334 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-27T15:40:17.348 INFO:teuthology.orchestra.run.smithi029.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 27 15:40 /home/ubuntu/cephtest 2023-11-27T15:40:17.351 INFO:teuthology.orchestra.run.smithi060.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 27 15:40 /home/ubuntu/cephtest 2023-11-27T15:40:17.371 INFO:teuthology.orchestra.run.smithi008.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 27 15:40 /home/ubuntu/cephtest 2023-11-27T15:40:17.374 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-27T15:40:17.385 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-27T15:40:17.432 INFO:teuthology.nuke:Checking targets against current locks 2023-11-27T15:40:17.458 DEBUG:teuthology.nuke:shortname: smithi008 2023-11-27T15:40:17.459 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T15:40:17.482 DEBUG:teuthology.nuke:shortname: smithi029 2023-11-27T15:40:17.483 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T15:40:17.488 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:01.949055', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T15:40:17.505 DEBUG:teuthology.nuke:shortname: smithi060 2023-11-27T15:40:17.505 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T15:40:17.509 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi029.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:01.950693', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T15:40:17.527 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467809', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:48:01.947281', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T15:40:17.537 INFO:teuthology.orchestra.console.smithi008:Power off 2023-11-27T15:40:17.538 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-27T15:40:17.561 DEBUG:teuthology.orchestra.console.smithi008:power off output: Chassis Power Control: Down/Off 2023-11-27T15:40:17.561 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:40:17.580 INFO:teuthology.orchestra.console.smithi029:Power off 2023-11-27T15:40:17.581 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-27T15:40:17.594 INFO:teuthology.orchestra.console.smithi060:Power off 2023-11-27T15:40:17.595 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-27T15:40:17.602 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2023-11-27T15:40:17.603 DEBUG:teuthology.orchestra.console.smithi029:power off output: Chassis Power Control: Down/Off 2023-11-27T15:40:17.604 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:40:17.616 DEBUG:teuthology.orchestra.console.smithi060:power off output: Chassis Power Control: Down/Off 2023-11-27T15:40:17.616 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:40:17.625 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2023-11-27T15:40:17.636 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2023-11-27T15:40:21.604 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:40:21.626 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:40:21.637 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:40:21.715 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2023-11-27T15:40:21.737 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is on 2023-11-27T15:40:21.746 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is on 2023-11-27T15:40:25.721 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:40:25.738 DEBUG:teuthology.orchestra.console.smithi029:pexpect command: ipmitool -H smithi029.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:40:25.748 DEBUG:teuthology.orchestra.console.smithi060:pexpect command: ipmitool -H smithi060.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:40:25.830 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2023-11-27T15:40:25.846 DEBUG:teuthology.orchestra.console.smithi029:check power output: Chassis Power is off 2023-11-27T15:40:25.856 DEBUG:teuthology.orchestra.console.smithi060:check power output: Chassis Power is off 2023-11-27T15:40:25.930 INFO:teuthology.orchestra.console.smithi008:Power off completed 2023-11-27T15:40:25.947 INFO:teuthology.orchestra.console.smithi029:Power off completed 2023-11-27T15:40:25.957 INFO:teuthology.orchestra.console.smithi060:Power off completed 2023-11-27T15:40:26.064 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/fs/misc}} duration: 2620.899904489517 failure_reason: 'Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi008 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh''' flavor: default owner: scheduled_choffman@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=622b1e18819f4d049812695aab7c60e3 status: fail success: false 2023-11-27T15:40:26.064 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-27T15:40:26.142 INFO:teuthology.run:FAIL